Introduction to Amazon API Gateway

Introduction to Amazon API Gateway

Overview

In this lab, you will create a simple FAQ micro-service. The micro-service will return a JSON object containing a random question and answer pair using an API Gateway endpoint that invokes a Lambda function. Here is the architecture pattern for our simple micro-service:

Topics covered

By the end of this lab you will be able to:

• Create Lambda functions
• Create API Gateway endpoints.
• Debug API Gateway and Lambda with CloudWatch.

Microservice Architecture

"The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies" – Fowler and Lewis, 2014

The idea of a microservices architecture is to take a large, complex system and break it down into independent, decoupled, easy to manage and extend services. This enables developers to meet their key design goals like extensibility, extendability, availability and maintainability.

Amazon API Gateway and AWS Lambda provide the perfect combination of web services to effortlessly build, deliver and maintain a suite of microservices that can be the foundation of complex software systems.

In today's lab, you will learn how to develop, deploy and debug a simple microservice that would represent one part of a potentially much larger system. It will consist of two pieces, first is the RESTful API and second is the function that is executed when a user hits the endpoint.

Application Programming Interface (API)

An application programming interface is a set of instructions that defines how developers interface with an application. The idea behind an API is to create a standardized approach to interfacing the various services provided by an application. An API is designed to be used with software development kits (SDKs). A software development kit is a collection of tools that allows developers to easily create downstream applications based on the API.

API-First Strategy

Many software organizations are adopting an API-First strategy, where each service within their stack is first and always released as an API. When designing a service, it is hard to know all of the various applications that may want to utilize the service. For instance, the FAQ service we are building today would work great to seed FAQ pages on an external website. However, it is feasible to think that a cloud education company would also want to ingest FAQ within their training materials, for things like flash cards or training documents. If we simply built a static website, the ingestion process for the education company would not be possible, or be very difficult. By providing an API that can be consumed in a standardized format, we are enabling the development of an ecosystem around the service, and use cases that were not initially considered.

RESTful API

Representational state transfer (REST) refers to architectures that follow six constraints:

1. Separation of concerns via a client-server model.
2. State is stored entirely on the client and the communication between the client and server is stateless.
3. The client will cache data to improve network efficiency.
4. There is a uniform interface (in the form of an API) between the server and client.
5. As complexity is added into the system, layers are introduced. There may be multiple layers of RESTful components.
6. Follows a code-on-demand pattern, where code can be downloaded on the fly (in our case implemented in Lambda) and changed without having to update clients.

In the system we are developing today, we are following a RESTful model. Clients send requests to the back—end lambda functions (server). The logic of service is encapsulated within the Lambda function and we are providing a uniform interface for clients to use.

Best Practices for Building a RESTful API

A key goal of building an API is to help establish an ecosystem of innovation around your set of services. Therefore, it is important to make your API intuitive and easy-to-use. Here is a common naming and method scheme to follow:

GET      /questions           Returns all of the questions.

GET      /questions/17        Returns the question number 17.

POST     /questions           Creates a new question.

PUT      /questions/17        Updates question number 17.

PATCH    /questions/17        Partially updates question number 17.

DELETE   /questions/17        Deletes question number 17.

Note: Notice how to get a specific question, the API endpoint is NOT /question/namebut instead /questions/identifier. This enables the API designer to provide functionality to return groups of questions (could be all questions) with the /questionsendpoint as well as single record responses with the /questions/identifier. For more information, see the additional resources section at the bottom of this lab guide.

A few good examples of RESTful API's to look at are:

• The WhiteHouse - https://github.com/WhiteHouse/api-standards#pragmatic-rest
• Spotify - https://developer.spotify.com/web-api/

Amazon API Gateway and AWS Lambda

A microservice using Amazon API Gateway consists of a defined resource and associated methods (GET, POST, PUT, etc.) in API Gateway as well as the backend target. In today's lab, the backend target will be a Lambda function. However, the backend target could be another HTTP endpoint (a third-party API or listening web server), an AWS service proxy or a mock integration to be used as a placeholder.

Amazon API Gateway Introduction

"Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale."

API Gateway is a managed service provided by AWS that makes creating, deploying and maintaining API's easy. API Gateway includes features to:

• Transform the body and headers of incoming API requests to match backend systems
• Transform the body and headers of the outgoing API responses to match API requirements
• Control API access via Amazon Identity and Access Management
• Create and apply API keys for third-party development
• Enable Amazon CloudWatch integration for API monitoring
• Cache API responses via Amazon CloudFront for faster response times
• Deploy an API to multiple stages, allowing easy differentiation between development, test, production as well as versioning
• Connect custom domains to an API
• Define models to help standardize your API request and response transformations.
• And more, see: https://aws.amazon.com/api-gateway/details/

Amazon API Gateway and AWS Lambda Terminology

Resource

A resource is represented as a URL endpoint and path. For example, api.mysite.com/questions. You can associate HTTP methods with resources and define different backend targets for each method. In a microservices architecture, a resource would represent a single microservice within your system.  

Method

In API Gateway, a method is identified by the combination of a resource path and an HTTP verb, such as GET, POST, and DELETE.

Method Request

The method request settings in API gateway store the method's authorization settings and define the URL Query String parameters and HTTP Request Headers that are received from the client.

Integration Request

The integration request settings define the backend target used with the method. It is also where you can define mapping templates, to transform the incoming request to match what the backend target is expecting.

Integration Response

The integration response settings is where the mappings are defined between the response from the backend target and the method response in API Gateway. You can also transform the data that is returned from your backend target to fit what your end users / applications are expecting.

Method Response

The method response settings define the method response types, their headers and content types.

Model

In API Gateway, a model defines the format, also known as the schema or shape, of some data. You create and use models to make it easier to create mapping templates. Because API Gateway is designed to work primarily with JavaScript Object Notation (JSON)-formatted data, API Gateway uses JSON Schema to define the expected schema of the data.

Stage

In API Gateway, a stage defines the path through which an API deployment is accessible. This is commonly used to deviate between versions, as well as development vs production endpoints, etc.

Blueprint

A Lambda blueprint is an example lambda function that can be used as a base to build out new Lambda functions.

Build Your Microservice

Create a New Lambda Function Using Example Code

6. Once you are in the Amazon Web Services Console, click Services > Lambda.
   
7. Click Get Started Now. On the next screen, you will see a variety of different blueprints to select from. These common Lambda functions are great starting places when developing your own Lambda functions. In this lab, we are going to build our own custom lambda function. Click Blank Function.
   
8. Since this Lambda function will trigger via an API Gateway call, on the Configure triggers page, click in the empty box and choose API Gateway from the drop-down list.
   
   
   
9. On the Configure triggers page, you will retain the default settings except for Security. API name represents the name of your API being setup by the Lambda function.
10. Deployment stage represents the stage of your API - e.g., dev, prod, beta, etc.
11. For Security, choose Open from the drop-down. You can ignore the warning once this option is chosen.
12. Click Next.
13. On the Configure function page, ignore the permissions warning. Name your Lambda function (e.g. "lambdafaq") and provide a description of the function. Select the runtime Node.js.
    
14. Copy the below lambda code in a text editor, and copy and paste the lambda function code with "Edit code inline" selected.

console.log('Loading event...');

 

var json = {

  "service": "lambda",

  "reference": "https://aws.amazon.com/lambda/faqs/",

  "questions": [{

    "q": "What is AWS Lambda?",

    "a": "AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service - all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app."

  },{

    "q":"What is the JVM environment Lambda uses for execution of my function?",

    "a":"Lambda provides the Amazon Linux build of openjdk 1.8."

  }

  ]

}

 

exports.handler = function(event, context) {

    var rand = Math.floor(Math.random() * json.questions.length);

    console.log("Quote selected: ", rand);

 

    var response = {

        body: JSON.stringify(json.questions[rand])

    };

    console.log(response);

    context.succeed(response);

};

Your screen should appear as in the following picture:

15. Notice in the code, we defined the function exports.handler. Since we are editing in-line, the file that is created for you is index.js. This means that the function the Lambda service needs to run is at index.handler. Scroll down to the Lambda function handler and role section. You can therefore leave the handler as index.handler.
    
16. For Role, click Choose an existing role.
17. For Existing role, click lambda-basic-execution.
    
18. This role was pre-created for you as part of the lab setup. This role contains the necessary permissions needed by the Lambda function in this lab.
19. Under Advanced Settings, leave the default memory allocation. Since this is a very simplistic lambda function, 128 MB will be sufficient.

Note: Pricing of lambda function execution is dependent on the time it takes to complete the function and the amount of memory allocated to the function.

20. Click Next. Ignore any permissions warnings. Review the function details and click Create Function.
    
    

Congratulations! You have created your API Gateway and Lambda function.

21. You should now see the Lambda function page, with the Triggers tab open.
    
    API Gateway: ycwn10dd4b
    https://ycwn10dd4b.execute-api.us-west-2.amazonaws.com/prod/whelambdafaq
    Deployment stage: prod Method: ANY
    
    CTRL-Click on your new API URL link for a new browser tab, in which you should see one FAQ entry returned.

View logs in Cloudwatch

Test the Lambda Function

23. Let's learn how to test the Lambda function in isolation. Back in the Lambda Service, Click Actions > Configure test event.
    

24. For a simple test, delete the provided keys and values, retaining an empty "{}" to represent an empty JSON object.

25. Click Save and Test.

26. Below the Lambda function, in the Execution Result section, it should say succeeded with the output of the function. The output shows the faq entry wrapped inside a body parameter.
27. Below the Execution result, you will see two columns. The Summary will display the total time it took to run the lambda function and the amount of resources consumed. The Log output column contains all of the logging information. In this section, you will see any console logging as well as any error messages.

28. In the Log output section, click the blue Click here link to view the logs in the CloudWatch Logs console. Here you will see a list of lambda function executions. If you have invoked the lambda function more than once, then many results will show up here. In production, you would not have the ability to click "test" and view the output. Therefore, if you are debugging, you will need to view the lambda logs through this portal.

29. Click on one of the log streams. From here, you will be able to see the same event data that was displayed in the Lambda console.

Congratulations! You have successfully tested and debugged your Lambda function!

Troubleshooting

Here are a few common errors and how to remedy them.

• When you are testing your URL, you see the following message: {"message":"Missing Authentication Token"}. This means either a resource and associated method has not been setup, or the security on the API endpoint has been set to AWS IAM and the credentials have not been included with the API call. Here are some steps to remedy this:

• Ensure that the security drop down is set to Open when creating your API endpoint.
• Ensure that the URL path is complete and points to your resource (/dev/faq/).
• The lambda function code is located on the lab webpage in qwikLABS. Once you start the lab, there is a tab titled "Lambda Function". Click on that tab and you will see the link to the faqLambda.js file.

Conclusion

Congratulations! You have completed this lab and have successfully created a microservice with API Gateway and Lambda. You now know how to:

• Create an AWS Lambda Function
• Create an API Gateway Endpoint for the Lambda Function
• Tested and monitored an API Gateway Endpoint and Lambda function with logs and CloudWatch.

Next Steps

This lab was intended to give you an introduction into microservices architecture patterns, why they are important and how to setup a basic open API.

Now that you have completed the lab, take it a step further! Here are some additional things you could do to extend the functionality of your microservice in your own AWS account:

• Control access to your endpoint by integrating AWS IAM.
• Move the JSON object with the questions into a data store like DynamoDB.
• Implement a URL parameter that will return a specific question.
• Implement an endpoint that returns all questions.
• Implement a new method that adds a question into the list.
• Checkout serverless development frameworks like Apex (https://github.com/apex/apex) or Serverless (https://github.com/serverless/serverless).

Additional Resources

• Best Practices for a Pragmatic RESTful API http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api
• Amazon API Gateway Documentation http://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html
• Amazon Lambda Documentation http://docs.aws.amazon.com/lambda/latest/dg/welcome.html
• Microservice Architecture patterns and best practices http://microservices.io/index.html
• Representational State Transfer https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm