Guide: How to debug Lambda functions locally (with Postgres)

Introduction

This guide will cover all is needed to debug Lambda functions locally. This is really useful for finding and troubleshooting issues that you might run into in the cloud and not rely on multiple logs inside the code, which can become tedious really soon. For this we will be using AWS Serverless Application Model (SAM), Docker(-compose), vscode and postgres.

Requirements

• AWS Account
• AWS CLI
• Docker
• SAM CLI
  • Windows
  • Linux/MAC+

brew tap aws/tap
brew install aws-sam-cli

• Python

After everything is installed, you should check if you’re able to get versions of all of the above:

Step 1:

Configure your aws credentials, running…

aws configure

The following example shows sample values.

Step 2

After you finished clonning the repository you want to connect to, let’s create a local database with docker(-compose). This needs to be done if the password for aws’ rds is dynamically generated (probably for security measures), in which case local sam will not be able to connect to it. You can also use this if you want to debug your lambda on a local DB.

So! First, create a docker-compose.yml file that includes:

version: "3.8"
services:
  backup-cvt-dev-portal:
    image: postgres:10.5-alpine
    container_name: backup-cvt-dev-portal
    environment:
      - POSTGRES_PASSWORD=XXXXXXXX
    ports:
      - "5400:5400"
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql

Then, create an init.sql file that contains the scheme of the database you want to create. You can get that from the tf-config repository. You can populate the database there or afterwards.

CREATE TABLE IF NOT EXISTS event_data
(
    event_id...

Finally, in the location of the files we just created, run

docker-compose up -d

To verify everything installed correctly, run:

docker exec -it sam-postgres-db psql -U postgres

and then try running a query.

Step 3

Dependencies! You need to manually include dependencies for external libraries.

Psycopg2, sqlalchemy and any other external library needs to be downloaded and extracted at the same level as your Lambda folder (this is configured from the template.yaml file).

You should be seeing something like this.

(Don’t forget to include those folders to the .gitignore file)

Step 4

The next step is to edit the template.yaml file:

If the project already contains Layers, they need to be replaced for all lambdas:

Layers:
  Ref: pythonLayers

with its literal array form:

Layers: [
        {
            Arn: "arn:aws:lambda:us-east-2:577426152022:layer:sqlalchemy-1-3-17-custom-psycopg2:2",
            CodeSize: 169
        },
        {
            Arn: "arn:aws:lambda:us-east-2:577426152022:layer:aws-sdk-xray:1",
            CodeSize: 169
        },
        {
            Arn: "arn:aws:lambda:us-east-2:577426152022:layer:aws-sdk-python-boto3-1-12-49:1",
            CodeSize: 169
        },
        {
            Arn: "arn:aws:lambda:us-east-2:577426152022:layer:requests:1",
            CodeSize: 169
        },
        {
            Arn: "arn:aws:lambda:us-east-2:577426152022:layer:space-trace-common:5",
            CodeSize: 169
        }
    ]

To debug Lambda functions, remember to match the Runtime declared with the one you have installed. For example:

Runtime: python3.7 -> Runtime: python3.8

Edit the connection data to the local database:

host="localhost",
port="5432",
database="postgres",
user="postgres",
password="XXXXXXX",

Step 5

Generate an events folder with the events you want to test out. You can generate events at the aws console.

Finally, to invoke the lambda locally, run

sam local invoke LambdaIWantToInvoke --event events/event.json --docker-network host

For example, running

sam local invoke LambdaUnflagAssetInstance --event events/event-unflagAsset.json --docker-network host

Mounts and executes the lambda:

Conclusion

Success! We can see the lambda is not only invoked locally, but also informs us the resources/time it took to execute, which is really useful information for monitoring its performance and to do some cost estimation. Also, if the template.yaml file is correctly configured, the local lambda should be able to invoke other existing lambdas in the cloud, access s3 buckets, sqs, etc, simulating a real execution (which is really convenient).

Why you should augment your team with Folder IT

Outsourcing or Augmenting your AI Team with Folder IT professionals is a cost effective solution that does not sacrifice on quality nor communication effectiveness. Our teams are qualified for working with all the latest technologies and for joining you right away.

Request a quote now for outsourcing your project or staff augmentation services to Argentina.