The API Gateway is one of the products available on the Pivotal CF Mobile Services. It allows developers to easily create edge gateways proxy calls between devices and your organization internal services.
There are several reasons why one would consider having such services in place:
- Centralized authentication: Your internal services may have different authentication mechanisms, if any, and you may want to expose a API key style of authentication to external consumers.
- Reduce chattiness: Allows mobile devices to reduce the number of calls to your backend by aggregating invocations to multiple services at once.
- Data type conversion: Internal services may have different formats such as REST or SOAP, an edge gateway can serve as a central transformation point.
- Data filtering: You can filter data either based on role based access or perhaps just for the sake of saving bandwidth to certain clients.
- Rate limiting: Control the number of requests clients can make to your services. This can avoid possible DDOS on your services as well as control access to certain billable services such as third-party services you may use.
One of the benefits of using an edge gateway is that you can perform all of the above without having to modify internal services—the gateway will sit between your services and the clients. The picture below demonstrates this:
Pivotal’s API Gateway uses javascript as its core language, but not your usual javascript engine. It’s powered by Nashorn, the latest JDK script engine. This approach allows developers to enjoy the benefits of using a simple script language such as javascript, while still being able to seamless integrate with Spring beans. You can invoke any Java Spring bean from your javascript code.
The picture below depicts the internals of a gateway application. Note that you can add your own java beans as well as javascript code.
Implementing Rate Limit
In this post, we will show how easy is to implement a rate limiting edge gateway service using Pivotal’s API Gateway and Redis.
Rate limiting is an important feature that you should consider when exposing your APIs to external users. It prevents possible DDOS on your internal services by discarding requests from customers which exceeded the maximum stipulated quota. It also allows you to have control of your billing expenses on your IaaS provider by controlling traffic and usage.
When it comes to rate limiting there isn’t a standard on how to deal with it, but several service providers such as twitter and github utilizes a combination of HTTP response headers and an HTTP Response code 429 (too many requests) when the client quota has exceeded.
There are many ways of implementing rate limiting, but in a nutshell we need to be able to:
- Identify the caller (via an API key or IP address)
- Log the number of requests
- Set an expiration time for the request window (1 minute, 1 hour, 1 day)
Rate control using Redis
Redis is a great choice for implementing rate limiting. It offers the capability of setting keys timeouts as well as atomic increase operations on a numeric key.
The pattern is actually described on the redis incr command. We will use a slight variation of that, which allows us to control the window interval as opposed to only using a one second window.
Redis stores one value with a JSON configuration object for each API key in this way:
{
“key”:”mykey”,
“value”:10,
“current”:null,
“reset”:null,
“window”:60
}
key: The API Key associated with this config
value: The request limit
current: The current request count for a given period (not used on configuration)
reset: The Unix epoch in milliseconds of the next reset of the count
window: Controls how long the counter will be held in memory
The above configuration creates a rate limit control that allows 10 requests per minute. Using Spring-Data-Redis, all we need is one Bean to get the information for each API request.
The code below shows the java spring bean we created to keep track of requests per api:
package io.pivotal.api.rate;
import java.io.IOException;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.dao.DataAccessException;
import org.springframework.data.redis.core.RedisOperations;
import org.springframework.data.redis.core.SessionCallback;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
@Component("rateLimiter")
public class RateLimiterImpl {
@Autowired
private StringRedisTemplate template;
@Autowired
private ObjectMapper mapper;
/**
* Reads the current value of a given key on a window of time. It contains both the value and next reset window
* This entry is stored in redis as a key on the format : 'apikey:timewindow'
*
* */
public Rate getCurrent(final Rate rate){
Long now = System.currentTimeMillis();
Long time = (Long)(now/(1000*rate.getWindow()));
final String key = rate.getKey()+":"+time;
List<Object> results = template.execute(new SessionCallback<List<Object>>() {
@Override
public List<Object> execute(RedisOperations ops) throws DataAccessException {
ops.multi();
ops.boundValueOps(key).increment(1L);
ops.expire(key, rate.getWindow(), TimeUnit.SECONDS);
return ops.exec();
}
});
Long current = (Long)results.get(0);
rate.setCurrent(current.intValue());
rate.setReset(time*(rate.getWindow()*1000));
return rate;
}
/**
* Fetches the configured rate for a given API. This entry only contains the key, the window and value
* it does not store the value of the current invocation or the next reset window.
* This entry is represented in redis just by the key 'apikey'
*
* */
public Rate getRate(String apikey){
Rate rate = null;
try {
rate = mapper.readValue(template.opsForValue().get(apikey), Rate.class);
rate = getCurrent(rate);
} catch (IOException e) {
e.printStackTrace();
}
return rate;
}
public void setRate(Rate rate){
try {
template.opsForValue().set(rate.getKey(), mapper.writeValueAsString(rate));
} catch (JsonProcessingException e) {
e.printStackTrace();
}
}
}
For sake of simplicity, we use the same JSON object to represent the configuration for a given API—for example, the number of requests and the window duration—as well as the current value for that instance in time.
We use two separate keys to store this. For the configuration, it’s a key that has not ttl and its stored using the apikey as the key. For the current value, we store using apikey:window as the key and we set a TTL on it.
For each call we create an unique key composed of the API key plus the current time divided by the rate window. This will give an unique key for each window of invocation. On each call we increment the key and increase its time to live by the rate window.
When the time window expires we generate a new key with a zero counter, the old key would still live for as long as the length of the window, but since it would never be refreshed, Redis will eventually expire it.
Javascript meets Java
As mentioned before, one of the greatests features of API Gateway is the ability to mesh Javascript code with Java. Writing your application in Javascript will allow you to leverage the simplicity of the language, and its great support for JSON. This reduces quite significantly the amount of boilerplate that would be needed to write APIs, while allowing you to access your java backend components as first class citizens.
API Gateway has a native spring global function that allows you to invoke any spring bean that is part of your application. Using this feature we can easily access our RateLimiter service bean.
To enable the rate limit for any request to this API endpoint we’ve created a decorator around the original Router that checks for the limits of each request:
var RatedRouter = function(router) {
this.router = router;
}
function getApiKey(req){
var header = req.headers["Authorization"];
return header.split(" ")[1];
}
function checkLimit(req,res){
var api = getApiKey(req);
var rate = spring.getBean("rateLimiter").getRate(api);
var remaining = rate.value-rate.current;
console.log(JSON.stringify(rate));
res.setHeader("X-RateLimit-Limit",rate.value);
res.setHeader("X-RateLimit-Remaining", remaining);
res.setHeader("X-RateLimit-Reset",rate.reset);
return (remaining > 0);
}
RatedRouter.prototype = {
get: function(path, handle){
this.router.get(path, function(req, res){
if(!checkLimit(req,res)){
res.setStatus(429);
}else{
handle(req,res);
}
});
},
post: function(path, handle){
this.router.post(path, function(req, res){
if(!checkLimit(req,res)){
res.setStatus(429);
}else{
handle(req,res);
}
});
},
put: function(path, handle){
this.router.put(path, function(req, res){
if(!checkLimit(req,res)){
res.setStatus(429);
}else{
handle(req,res);
}
});
},
delete : function(path, handle){
this.router.delete(path, function(req, res){
if(!checkLimit(req,res)){
res.setStatus(429);
}else{
handle(req,res);
}
});
}
}
module.exports = RatedRouter
The checkLimit function will retrieve the limits for this request, and add the proper X-Rate-Limit headers to each response.
Each method on this router will check if the limit was reached and return a 429 (Too Many Requests) empty response if true, or just invoke the original handler that was passed to it (with the enhanced headers).
To demonstrate this new rate limited router, we will modify the http client sample (see here) and make sure that the calls to the external endpoint now have rate limiting enabled.
var Router = require("Router");
var RatedRouter = require("modules/RatedRouter")
var _router = new Router();
var appRouter = new RatedRouter(_router);
var fbGraphClient = require('http')({
baseUrl : 'http://graph.facebook.com/',
});
appRouter.get("/fb/pivotal", function(req, res) {
var result = fbGraphClient.getJSON('pivotalsoftware', function(response) {
return {
reponse_code_from_fb : response.statusCode,
data_from_fb : response.body
};
});
res.setBody(result);
});
module.exports = _router;
The new script is exact the same as the one on the samples, but now calls respect a limit that is stored on your redis database.
To test it out, let’s create a script that tries to invoke this endpoint 11 times in windows of less than 60 seconds.
for i in {1..11}; do curl -i -H "Authorization: apikey mykey" http://localhost:8080/api/v1/fb/pivotal; done
HTTP/1.1 200 OK
X-Application-Context: application:9999
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 9
X-RateLimit-Reset: 1410442320000
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Server: Jetty(8.1.15.v20140411)
{hidden response body}
HTTP/1.1 200 OK
X-Application-Context: application:9999
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 8
X-RateLimit-Reset: 1410442320000
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Server: Jetty(8.1.15.v20140411)
.
.
.
HTTP/1.1 429 429
X-Application-Context: application:9999
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1410442320000
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Server: Jetty(8.1.15.v20140411)
As you can see after the 10th call the server stops forwarding requests to the fbClient service and returns 429. The client receives a X-RateLimit-Reset header that informs when it would be ok to call the service again.
Summary
This post was intended to demonstrate how easy it is to add features to your edge gateway when using Pivotal’s API Gateway. With very few lines of code we were able to enable rate limiting to our internal services without having to change any existing component.
The seamless integration between javascript and java puts the best of both languages at your disposal. You can still rely on enterprise grade components built using Spring Framework, while leveraging a dynamic scripting language running on an asynchronous framework that the API Gateway provides.
About the Author
