How to create a Python Lambda function to connect to a AWS Aurora Serverless database using the ‘Data API’?
First of all it’s important to know that the ‘Data API’ that enables the connection with Aurora Serverless is still in beta and only available in N. Virginia (us-east-1). Thus, all your resources must reside in this region.
Firstly, you create your Aurora Serverless database:
Select “Serverless” and specify the username and password to connect:
Select your “Capacity Settings” and “Network Settings”:
After your database is created you can enable “Data API”. Select your database and go to “Modify” and under “Security Group” check the “Data API” box:
Now, we can connect to the database via API or “Query Editor”. I’ll focus on the former. Follow the link to know more about the latter.
Boto3 was updated with the “RDSDataService” to enable the use of the “Data API”. 3 arguments are required: dbClusterOrInstanceArn, sqlStatements, and awsSecretStoreArn. The first is the cluster that we created, the second is what we want to execute (i.e.: select * from table), and the third is the ARN of the connection string. For this you need to use the “AWS Secrets Manager” to store your user/password.
Ok, so far so good. Let’s jump into the Lambda function.
Create a new function using “Python 3.7” and select your role that have to have the “AmazonRDSDataFullAccess" policy attached or similar as stated here in the documentation.
Now that you have everything, just call the API using the Boto3 documentation and fine, right? Not quite. Despite it’s enabled in N. Virginia if you try to run it directly you will get the following error:
Lambda doesn't recognize the new “rds-data” service. Therefore, we need to create a package (.zip) with the latest AWS CLI version and upload it to our Lambda function.
I created mine using a Linux 2 AMI:
# install Python and Pip
sudo yum install python37
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
sudo python get-pip.py
# creating the directory
#install the latest AWS CLI
pip install awscli — target .
#create your function
#zip the function
zip -r9 ../function.zip .
Now you can upload to S3 or to your machine and then to Lambda. I chose to upload to S3 first and from there I updated my Lambda function:
Don’t forget to increase your Lambda timeout (I’m using 1 minute). And just execute it.
Getting the pre-requisites right and packaging your function with the latest AWS CLI is key to be able to use Lambda to connect to Aurora Serverless.
Lambda functions can be found in GitHub.