Tutorial for building a Web Application with Amazon S3, Lambda, DynamoDB and API Gateway
I recently attended Serverless Day at the AWS Loft in downtown San Francisco. During the workshop section we built a serverless web application for requesting Unicorns to come pick us up. The AWS team provided excellent documentation on Github and Rahul Sareen gave a one of the best presentations I have heard at a tech event overviewing Serverless application architecture. (Slides for that presentation are available here).
In the workshop portion we created and deployed a website that utilized S3 for hosting, DynamoDB for a database, API Gateway for RESTful endpoints and Lambda functions as our backend server processing.
This tutorial covers my notes from building out the application and using some of these services for the first time on Serverless Day 2017. More detailed notes for following along are available on the github and the Wild Rydes demo application is live at http://www.wildrydes.com/.
The application we are going to create in this tutorial is called Wild Rydes. The application is a fictional service for ordering unicorns to come pick us up. Users can login to the application and request unicorns from their current location. The application then dispatches a unicorn to pick up the user.
Without further ado, let’s get started.
As with most AWS tutorials, the first step is to create an IAM user that will create and provision our AWS resources. I have a user set up that has AdminAccess. It is considered best practice to login using such an user rather than logging into and managing your AWS resources using your root account credentials. If you have no idea what I’m talking about I suggest checking out the A Cloud Guru course for passing the AWS Certified Developer - Associate exam. Chapter 3 provides easy to follow video instructions on setting up users for your AWS account.
If you are not so inclined, the AWS team also provides detailed instructions for creating an IAM user with the specific permissions (
AWSLambdaBasicExecutionRole) to write to DynamoDB and CloudWatch. If you associate your Lambda function with a user that has admin access your Lambda function will be able to access any service.
You also want to make sure that when you install the AWS CLI it is associated with the user you created. When creating a new IAM user you get one chance to download the key-value pair for that user. In the command line type
aws configure and you can set your public and secret API keys for the CLI.
Managing user access is important for account security and provisioning access to our AWS resources. We ran into some errors getting things set up and all of the errors were related to IAM so make sure you have permissions to do what you are trying to do! (pro tip:
aws configure helps)
The first step is to create an S3 bucket and enable the static web hosting option for that bucket. The AWS team provides details instructions on how to do this here.
When static website hosting is enabled for an S3 bucket, the contents of the index.html file within that bucket will be publicly accessible to the internet following this URL structure:
http://BUCKET_NAME.s3-website-REGION.amazonaws.com/ where BUCKET_NAME is the globally unique name you gave your bucket and REGION is the region you created the bucket in (such as
us-east-1 for Virginia or
us-west-2 for Oregon).
Since this tutorial focuses on AWS infrastructure instead of static website coding, we copy the files for Wild Rydes from the AWS team. This code is open source and available here
The command to copy the contents of their bucket into our bucket is as follows:
aws s3 sync s3://wildrydes-us-east-1/WebApplication/1_StaticWebHosting/website s3://YOUR_BUCKET_NAME --region YOUR_BUCKET_REGION
After running this command all of our static files should appear in our S3 bucket when we refresh the page showing our bucket contents. If you are having issues syncing the files across buckets using the command line make sure you are logged in as the same IAM user that created the bucket or that the keys/permissions line up.
Finally, we want to make sure that our bucket is publicly accessible to the internet. For this we add a bucket policy as outlined below:
JSON schema for our S3 bucket policy:
My bucket is called wildrydes-082317 and created within us-west-2 (Oregon) so my static website files are publicly accessible here: http://wildrydes-082317.s3-website-us-west-2.amazonaws.com/
In the next step we will configure a Cognito user pool to manage users. This hooks up the functionality for users to create accounts, verify their email addresses and sign in to the Wild Rydes site.
Following the above instructions, the first step is to create a Cognito user pool using the AWS console. Cognito user pools provide out of the box functionality for federated identity providers (such as Google and Facebook login), password recovery and user authorization security in the cloud. You can learn more about user pools here.
When we create our Cognito user pool and create an app client. App clients have permission to call unauthenticated APIs (such as register, login and forgot passowrd). Take note of your Pool Id and the App client id (featured below) as we will insert these values into js/config.js
Head into your S3 bucket, download and modify js/config.js with your appropriate values from Cognito. Reupload the file back to your S3 bucket. We will have to do this one more time to populate the
invokeUrl with a value from API gateway. Populating the
Once we have updated our Cognito object within the config file, head over to the register page at
YOUR_S3_URL/register.html. In my case the full url is:
Sign up and create an account. Use your real email address! Cognito sends a test email with a link to verify your account. When you check your email after creating your account you will have a verification code, such as:
YOUR_S3_URL/verify.html and enter your email address and confirmation code.
Go to signin page and signin with your new account:
You can customize the verification message by navigating to your Cognito User Pool by clicking Message Customizations on the left navigation panel.
It is worth noting here that we could use other authentication services such as Auth0 (they have an awesome developer blog). This is an Amazon provided tutorial though so we are using all AWS functionality.
When we successfully create a user, verify and sign in we will get to this screen:
In this step we’ll implement a Lambda function that will be invoked each time a signed in user requests a unicorn. Lambda functions are the core functionality qualifying apps as Serverless. Lambda functions are a managed service provided by Amazon. We provide the code for the Lambda function and only pay for the time it takes that function to execute. We do not have to deal with provisioning EC2 instances or Elastic Load Balancing (typical operations functions for cloud applications). The primary advantage of this approach is that is is far cheaper than dedicated cloud hosting. It can also allow us to focus more on writing code and less on operations. Serverless and Lambda functions are a new Amazon service and new paradigm for web applications so there will be a learning curve but have the potential to save us massive time and money down the road.
The full steps for setting up the serverless backend are available here.
Before we even get to setting up Lambda functions and a serverless application we are going to create a DynamoDB database. DynamoDB is Amazon’s managed NoSQL database. We are going to use DynamoDB to store information about the ride request when a user requests a Unicorn.
When we create the database note the ARN. It will look something like this:
Amazon Resource Name (ARN) arn:aws:dynamodb:us-west-2:XXXXXXXXXXXX:table/Rides
Now that the database is created we’re going to an IAM role for the Lambda function. Every Lambda function must have an IAM role associated with it. The IAM role defines what AWS services the Lambda function is permitted to interact with. In this case we are going to go with the
AWSLambdaBasicExecutionRole. This basic role covers the functionality we need for the Wild Rydes application – writing logs to Amazon CloudWatch and writing items to a DynamoDB table.
Detailed steps are available here for creating the IAM role.
Now that we have the DynamoDB database created and a role ready to associate with our Lambda function we can create the function itself!
Create a Lambda function called
RequestUnicorn. The Amazon Web Services team provided the Node.js script for the Lambda function here. The full code for our Lambda function is below:
const randomBytes = require('crypto').randomBytes;
Currently we can write Lambda functions in Node.js, Python, Java or C#. The above code is a Node.js function that checks the user is authorized, writes to DynamoDB within the
recordRide function and sends a random Unicorn back to the user. After reviewing the code, paste in the Lambda function and create it, leaving the default
We can also configure a test event to make sure our Lambda function is envoked properly. If you would like to test your Lambda function, paste in the sample event code and verify that the execute succeeds.
We have set everything up for our Lambda function and static website. Now we need to set up API Gateway so that our static website can trigger the Lambda function. Amazon’s API Gateway allows us to create RESTful APIs that expose HTTP endpoints. These endpoints can be invoked from the browser.
The final step is to create an API Gateway that will be our REST API. We could use tools like Swagger or stoplight.io at this point. Since we are only creating one HTTP endpoint we will create it manually.
In order to hook up Cognito to API Gateway and protect our endpoints create a Cognito User pool authorizer:
Select Authorizers. Create -> Cognito user pool.
Now that that is configured we create a new resource method for the
POST /ride endpoint.
More detailed instructions are available here but the gist is that we select the option for Proxy Integration and add the WildRydesLambda function tat we created in the last step. Select method request card and under authorization select our Cognito user pool.
We also have to enable CORS for our endpoint. In the API Gateway console, under Actions and replace default values and select Enable CORS. Everything can be left as the defaults.
Deploy the API Gateway by selecting Actions -> Deploy. This generates an Invoke URL that we must include in js/cofig.js. In my case the value is
https://tfyxh265h2.execute-api.us-west-2.amazonaws.com/prod. This endpoint is what our website requests via AJAX that invokes the Lambda function.
Thanks for reading! If you enjoyed please share/upvote so that more people can hop on the serverless bandwagon and drink the Kool Aid.