Routing Requests to Different Regions with Lambda@Edge and Amazon CloudFront

One of the problems facing growing technology companies with online presence is reaching their clients across the globe and simplifying the service. Internet makes it possible for us to acquire services from a provider in the United States while being in Australia. You would think breaking down thousands of miles into a few seconds would make the cut, but in today’s world that can cost you a potential user. Studies suggest that 53% of users leave a site if it takes more than 3 seconds to load. It would seem like a stretch, but that is a reality all companies have to deal with.

Companies maintain data centers in multiple regions to get to their clients in the fastest possible time. Once you have addressed that then comes the issue with data protection laws in different regions. Regulations like GDPR make it imperative to keep certain client data within the region of operation. Federal guidelines can also bar certain industries from using services that store data outside the country. One of the most noticeable cases of recent times was with TikTok where questions got raised about their data accessibility from China. You have to host client data in a way where you are compliant with all such regulations.

Fortunately, with cloud service provides like AWS, Google Cloud and Microsoft Azure, hosting servers in multiple regions becomes seamless. You can replicate your existing systems and copy them over to another region and launch. The complexity of the process is no more enduring than copying a word document to another folder. So, when you are done navigating through all regulatory guidelines, you can quite readily turn yourself into a multi-region service provider. Costs are obviously going to go up, but that is a business decision you need to make.

I wish I could tell you the new journey begins here, but it does not. It only adds more complexity. How do you determine which data center region to send each client to and how can you do it in the most effective manner? That is the big question that still remains. We found ourselves stuck at this juncture some months back when we decided to go multi region. The solution came in the form of Amazon CloudFront bolstered by Lambda@Edge. We had a hard time finding material that addressed our specific pain points. We hope this article well help you avoid the struggles we faced. We will delve into the problem and alternatives first and finish up with the final solution.

Isolated Multi Region Hosting

Simplicity and maintainability are what we strive to do at TaskCall. We make the utmost effort to build systems that will be easy to maintain and be even easier for clients to use. The multi region infrastructure that we deployed is one that is completely separate and isolated from each other. We have two data center regions at the moment – one in the EU and the other in the US. The two regions do not interact with each other. If an account is hosted in one region, the data for that account will always be stored within the region at rest at all times. Even in transit, the data will not leave the region if the client does not use any external integration. We achieve this by having completely separate endpoints for each of our system components. For example, the endpoint for our EU region web application is https://app.taskcallapp.com while that of our US region is https://app.us.taskcallapp.com. Clients are required to choose their data center region when they log in. Having the wrong region selected would return a 404 error. This gives us and our clients the assurance that we will stay compliant with regulations that they are subjected to.

Problem with Hosting Application in External Marketplaces

The problem that seemed to linger on was how to handle custom integrations we have with external vendors like Slack, Microsoft Teams and Google Chat. For each of these vendors and some others as well, we offer our custom application on their marketplaces. These applications run on the vendor’s platform and enable interaction with TaskCall. The limitation of each of these platforms is that they only allow you to provide a single HTTPS endpoint where they will send events to. This was not an issue when we only had one region, but now that we have two completely isolated regions, the question rose about how to direct traffic to the correct region. If we set the endpoint to our EU region and a US region client used our Slack integration, the requests would be sent to our EU region, where the client’s account would not be found. We needed a way to route requests to the right region.

Dirty Solution – Too Many Round Trips

The first solution that we explored was having all requests still be sent to the EU region where we would use a mapping to identify what region the incoming request’s client account was in and send a new request to that region, wait for the response and then respond back again. So, visualize this, if a request generated from California, USA, it would be sent to our data center region in EU. The EU region would internally send the payload to our US data center region for processing. It would process the data and send it back to our EU region. The EU region would then respond back to the client in California. Seems a little redundant, costly, time consuming? Yes, felt the same to us too. We had to drop that.

Even Dirtier Solution – Duplicate Applications

Next we explored the option of building a duplicate application for each platform with a US region endpoint and publishing them in the respective marketplaces. For example, we would have a TaskCall Slack App and a TaskCall (US) Slack App. That meant for each Slack, Microsoft Teams and Google Chat we would have to publish two different applications – 6 in total. And that was just these three platforms. We have many others that would need the same change. This seemed redundant and dirty. If we wanted to make a change we would have to make the same change in both applications and resubmit for review. We would have to rely on our clients to download the correct application for their specific region. This works for some companies, like our competitor PagerDuty, but we felt that it was unnecessarily messy and burdensome. So, if that was not going to work, what then?

Emergence of Amazon CloudFront

Then we started looking into CDNs. CDN stands for Cloud Content Delivery Network. It is used by many sites to respond back to users in the fastest possible time and from the region nearest to them. Two of the most popular CDN providers are Amazon CloudFront and Cloudflare. They are present in hundreds of locations across the world. You simply point your website address to the CDN endpoint and it will route the traffic to the region nearest to the user and respond back. In a lot of cases, CDNs cache the data and simply send back the cached date without even doing the trip to the data center region. Let’s say your website is https://example.com. Instead of pointing that address to your server IP, you will point it to the address minted for you by your CDN provider, like https://cdn1235.net. The CDN handles the routing from there on.

Try CloudFront Functions?

Sounds like that would solve our problem? Not really, but it’s a start. Basic CDN works well with static pages. Static pages don’t change much and so sending cached data is not a problem. In the case where requests are routed to the nearest region, what CDNs do is run a small function at the CDN location to determine where to send the data. So if the request generates from Australia, you can route the request to your data center region in India, while if the request originates from the Canada, you can send them to your data center region in the US. In CloudFront, you can use CloudFront Functions to achieve this, while in Cloudflare you can use Workers to do the same. Since, we are hosted in AWS we wanted to use the AWS service. It also made sense for us cost wise.

If we could run a small piece of code on the CDN provider to determine what region an account belonged to and route requests accordingly, then that would solve our problem. CloudFront Functions are extremely light and are optimized to run very fast. However, they are limited in storage size and cannot access the body of the request and cannot make network requests. We need to get the account ID from the body of the request and make network requests to a known location where our account mappings stay and check which region to route to.

The Savior – Lambda@Edge

The final solution that came to the rescue was Lambda@Edge. Lambda@Edge lets you combine the powerful feature of AWS Lambda and CloudFront. Lambda@Edge has relatively higher runtime (we are talking about 100-300ms more) and is limited to 13 locations as opposed to over 225 basic CloudFront locations, but can be configured to make network requests. After evaluating the differences between CloudFront Functions and Lambda@Edge it became clear to us that the latter is exactly what we needed. On Lambda@Edge we decided to run a light function that would grab the original request, read the body of the request and get the vendor account ID, fetch all the account mappings from a known S3 location and determine which of our data center regions – EU or US – to finally route the request to.

Lambda@Edge pricing is reasonable and designed for growth. You are billed for Lambda@Edge based on the time the function runs for. For CloudFront the first 1 TB of data transfer, the first 10 million requests and the first 2 million function invocations are all free. We did the math, and it was almost negligible for us.

On the contrary, we saved the trouble of having to duplicate our applications for each vendor platform. Our clients only interact with one application while the work of where to route to is all done in the background without them even noticing a difference. We are compliant with data transfer regulations and our clients are all happy. So, if you find yourself in a similar predicament, dig into CloudFront and Lambda@Edge. It might have the answer for you.