Serverless NodeJS with Neon Serverless Postgres and AWS Lambda

Photo by Growtika on Unsplash

In this post we will learn to create an Express and Node app running on AWS Lambda. We will use Neon Serverless Postgres and the famous Serverless Framework.

This project have been created learning from the awesome YouTube video by freeCodeCamp. The link for the same is here.

First go to your AWS account and search for lamda.

Next click on Create a function button and check the NodeJS versions available on Lamdha. We will be using version 20 in this tutorial. So, upddate the version accordingly.

We will be mostly using the serverless framework in this project. The link for the same is here. As per the documentation we will add serverless globally by the command npm i -g serverless.

Now, in the terminal run serverless and then select AWS — Node.js — Express API

Next, give the project a name which in our case is serverless-api-nodejs. Also, Register and deploy should be selected as No.

A folder will be created by the name serverless-api-nodejs. So, cd to it and open it in VS Code.

In VS Code click on File and then Save Workspace As… option.

A pop-up will come where just click on the Save button.

Also, initiate git in the folder by running git init command.

We will be using the serverless Postgres in our project. So, add the same using the npm package for the same. Here, we are adding two dependencies in the project.

The command of neonctl auth will open a verification link on our default browser. Here, verify with github.

Once the authentication is done sucessfully, we will get the below screen.

In the neonctl auth in the terminal, also we will get a success message.

We will also add Drizzle ORM to use our Postgres later.

Next, from the terminal we will add the serverless plugin to use serverless in offline development mode. Use the command mentioned in the terminal.

Now, run the serverless offline command and it will open our server on http://localhost:3000/

Now, instead of running this command we will add the script of dev in package.json file. After that we will run the npm run dev command.

Now, we will move our index.js file inside a src folder. Alo, in the serverless.yml file we will update this path and also NodeJS version to 20.

Now, for the environment variables we will use the special dotenv plugin by installing it from the terminal.

Also, in the .gitignore file add the .env* to have them not uploaded in github.

Now, we will create an .env file in the root directory and add some variables in it.

Also, in the .env.dev file in the root irectory put the below content.

Back in the serverless.yml file we will add line to use the .env files. Also, in the environment will have DEBUG and DATABASE_URL been used.

From the index.js file we will send the DEBUG and DATABASE_URL from the / endpoint. After adding this run the npm run dev command again.

Now, in http://localhost:3000/ we will get the DEBUG and DATABASE_URL data also.

Now, we will go to https://neon.tech/ and login to it.

After login click on Create project button.

Once created we will get the connection string for the same.

Now, add the connection string to the .env.dev file.

Now, in the SQL Editor in neon.tech we will run the given command to add some random rows.

We can also create a new branch in neon.tech by going to Branches and then clicking on Create branch.

Give the new branch the name of dev and click on Create new branch button.

Now, go to Projects and then click on the three dots and then the Settings.

In the next page click on the Delete tab and then Delete project button to delete this project.

We will again use the Neon CLI to get the project list by giving the below commands.

Now, we will create the project from command line by giving the create command and then check the connection string also.

We can also create branches from command line and get a list of it.

Now, we will give the main branch connection string in the .env file. And the dev branch connection string in the .env.dev file.

Now, in the index.js file we will connect to the database and do a select command.

Back in http://localhost:3000/ we will see the results with current time.

Next, we will also add the cache in the index.js file.

Now, we will deploy to prod by creating an .env.prod file. And then add a script to deploy the project. Here, we have given the region as the one which we got from neonctl projects list command.

Now, go to aws and click on IAM user.

After that click on User Groups and then Create group.

In the next window give the user group name, which in our case is serverless-framework. And then click on Create Group button.

Next, click on Users and then Create user button.

Next give the user a name which is serverless-api-nodejs in our case. And then click on the Next button.

Next, add the user to a group which is serverless-framework in out case.

Finally, click the Create User button.

Now, we have a new user serverless-api-nodejs

Now, go to the Security credentials and click on Create access key button.

In the next page click on Other and then the Next button.

Now, in the final window click on the Create access key button.

The access key is created which we need to copy along with the secret.

Now, in the .env file add both of them.

Now, create a reference folder and add the policy.json file in it. The data for the same can be taken from this link.

We are not going to put this file in github. So, add it in .gitignore file.

Now, go to the AWS dashboard and get the Account ID.

Now, go back to the policy.json file and search for :AWS_ID: and replace all of it with :<YOUR_AWS_ACCOUNT_ID>:

Now click on Policies and then Create policy in AWS.

Now, copy the policy.json file content to the Policy editor.

After saving it, give the policy a name which is serverless-framework-permissions in our case.

Now, go to User groups and then click on Permissions. Now click on the dropdown and then Attach policies.

In the next screen we will be shown a number of policies. So, search for serverless and we will find or policy. Click on the checkbox to select the policy and then Attach policies.

We have also added a remove script in the package.json file.

Now, run the npm run deploy but it was failed because Lamda was unable to configure environment variables.

To fix this we need to exclude some files by adding them in serverless.yml file.

Now, npm run deploy will work and we will get a random deployment link.

Now, going to the url will show the result from our API endpoint of NodeJS.

Now going to AWS S3 will show a bucket been automatically been created. But this will only start if we hit the serverless endpoint and charged accordingly.

This complete our tutorial. The code for the same can be found in this github repository.