SlashDB On AWS Lambda – Feature

Go serverless with SlashDB and AWS Lambda, and see how you can achieve these advantages:

  • Cost Efficiencies – when you don’t have to manage infrastructure you can componentize parts of your  entire application and only scale what you need to scale
  • Reduced Maintenance – patches, server size adjustments, and load balancing are a thing of the past.  Serverless platforms can take the burden of this off your plate.
  • Faster development – easily harness existing backend services and consider your SlashDB-created APIs mixed and integrated with third party APIs as the backbones to quickly build and deploy apps

This post takes a full dive into building Python-based AWS Lambda functions.  We’ll show you how SlashDB becomes your simple database access component wrapped within simple Python-based logic; a way to totally submerge the RDBMS data integration headache.  Right … no need for imports and the mental gymnastics of MySQL Connector, SQLite, PostgreSQL etc. No dealing with cursors, connections, ‘execute’ statements, pre-composing data lists, fetchone(), fetchall() and so on!! Did I forget to commit(), to close()?  Well here we go and don’t hesitate to follow along yourself with your own SlashDB implementation.

For this post material, you’ll need to be established on Amazon Web Services.  And if you’re not currently a SlashDB subscriber, you can try this out using the free trial.  It’s easier than ever through the Amazon Web Services Marketplace. Click here to start your free trial and give it a try now on AWS

Starting off, you need to have Python installed on your local machine. Then you’ll need to package up components of “Requests” needed by the HTTP library for Python.  See more about this at http://docs.python-requests.org/en/master/. We’ll need “Requests” to send the HTTP requests for SlashDB to do its magic.

Getting Started with Lambda

For this project, create a Python file called gamingLeaderboard.py.  Our project will be a Leaderboard updater.  You see them all the time and they look something like this:

Let’s tell the story with the presentation of this flowchart for what we’d like to implement and display with this Leaderboard.  As input to the Lambda function, we’ll take the user’s score for the game that they just played.
Flowchart of the program logic

The Python Function Code

Now we can code this up within our gamingLeaderboard.py file:

import json

import requests
#def lambda_handler(thisUsersScreenname ,herGamesScore):
def lambda_handler(event ,context):
    
    # Have we seen this user before?
    thisUsersScreenname = event['screen_name']
    herGamesScore = event['score']
    #   Empty "data" dict that may be used as request body for
    #   insertion of screen name and score into the Leaderboard table.
    data = {}
    #   Fill in the entries for the screen name and her score
    data["ScreenName"] = thisUsersScreenname
    data["Score"] = int(herGamesScore)
    data = json.dumps(data)
    
    SlashdbURL = "http://ec2-99-147-60-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName/"+ str(thisUsersScreenname)+".json"
    print(""+str(SlashdbURL))
    response = requests.get(SlashdbURL)
    thisUserData = response.json()
    print(type(thisUserData))
    print(thisUserData)
    # check the response to determine if the user is found (200) or not (404)   
    # Print the status code of the response.
    print(response.status_code)
    if response.status_code == 404:
        congratsorSorryorWelcomeMessage = "Welcome as a first-time gamer!"
        # place this first-time user into the Leaderboard table along with this score.
        ##
        print("Entering the ScreenName: " + str(thisUsersScreenname) + " and Score: " + str(herGamesScore) + " into the Leaderboard table.") 
        response = requests.post("http://ec2-99-147-60-85.compute-1.amazonaws.com/db/Gaming/Leaderboard.json?apikey=dzn8k7hj2sdgddlvymfmefh1k2ddjl05", data = data)
        print("response is: " + str(response))
        ##
        ##
    else:
        if response.status_code == 200:
            # Check the response for the user's personal best score (last highest score).
      
            personalBest =  thisUserData["Score"]
             
            if int(herGamesScore) > int(personalBest):
                congratsorSorryorWelcomeMessage = "Congratulations, you have a personal best score."
                # Update the user's score with this new personal best
                # Using SlashDB's SQL Pass-Thru.   Only the HTTP PUT method is valid for an update.
                # Write back this new personal best value to the Leaderboard table
                #
                response = requests.put("http://ec2-99-147-60-85.compute-1.amazonaws.com/query/UpdateGamersBestScore/Score/" + str(herGamesScore) + "/ScreenName/"+ str(thisUsersScreenname) +".json?apikey=dzn8k7hj2sdgddlvymfmefh1k2ddjl05")
                print("response is: " + str(response)) 
            else:
                congratsorSorryorWelcomeMessage = "Sorry, not as good as your personal best."
              
                
            
    
    # Now we'll request the "Top 5" scores on our Leaderboard.
    SlashdbURL = "http://ec2-99-147-60-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName,Score.json?sort=-Score&limit=5 "
    print("the next SlashdbURL is " + str(SlashdbURL)) 
    response = requests.get(SlashdbURL)
    print("response is: " + str(response))
    topScoresData = response.json()
    
    inTop5Message = "You need to work harder to get into the Top 5 Leader Board category."
    for topScorers in topScoresData:
	  
	    # Let's see if we are in the Top 5 now
	    if thisUsersScreenname == topScorers["ScreenName"]:
	       print("Yay!!! There you are in the top 5 scores")
	       inTop5Message = "Congratulations, you're in the Top 5 lead scores."

 

    # Return for display, the Top 5 ScreenNames and their scores

    print("Returning: congratsorSorryorWelcomeMessage : " + str(congratsorSorryorWelcomeMessage))
    print("               and inTop5Message is :      " + str(inTop5Message))
    print("               and topScoresData is :      " + str(topScoresData))
    
    return str(congratsorSorryorWelcomeMessage), str(inTop5Message), topScoresData
	       
	       

Data Definition Language For Our Leaderboard

Now, we’ll create the DDL for our database/table. With your SlashDB installation, set up the RDS instance in the same AWS  zone/region that you selected when setting up your the SlashDB installation’s EC2 instance – more at https://docs.slashdb.com/user-guide/getting-slashdb.html . From the SlashDB console, select Configure > Databases. 

         Above, an example of type: mysql with the Configuration Permissions and database access security settings.

And now the data definition language for the simple Leaderboard table:

Create Database Gaming;
Use Gaming;
CREATE TABLE Leaderboard
(
ScreenName VARCHAR(255),
Score INT,
CONSTRAINT Leaderboard_pk PRIMARY KEY (ScreenName)
 
);

And now let’s seed the database with game scores.

Insert into Leaderboard values ('Banksy',970987);
Insert into Leaderboard values ('JimSteph',201987);
Insert into Leaderboard values ('BongoBoy',96788);
Insert into Leaderboard values ('PhillySteak3',888965);
Insert into Leaderboard values ('PastaEater',302905);
Insert into Leaderboard values ('BusyGirl',212069);
Insert into Leaderboard values ('ChicFlic5',56438);
Insert into Leaderboard values ('Eagle_One',67544);
Insert into Leaderboard values ('TransAM_kid',9899);

The database will be called “Gaming” and contain the single table: “Leaderboard”.

Thinking in Terms of the Function and Not the Server – Lambda

Now let’s construct the Lambda function:

  Within your AWS Console, navigate to the Lambda service within the “Compute” category.

  Click the “Create function” button:

AWS Lambda Create Function 

Make the selections: “Author from scratch”, Name (using “gamingLeaderboard”), Runtime (using Python 3.6), Role    (“Choose an existing role”), and finally select that role (“service-role/lambda-basic-rule”).

  Click the “Create function” button.

  The next screen is the Configuration UI.  Within Function Code, change the “Handler” to  gamingLeaderboard.lambda_handler.

  Click the “Save” button found on the upper right of the UI.

  Back in the Function Code section of this Configuration page, the “Code entry type” is defaulted to “Edit code inline”.  Since we created the .py file on our local machine (see above), we upload that file in a compressed file (.zip). To do that:

  Create a project directory that contains the gamingLeaderboard.py file.

  On your local machine, issue:

$ python3.6 -m pip install requests -t /path/to/project-directory

Now you may compress the .py file along with the directories needed for “Requests” that were created as a result of the above “install”.

In the Lambda Configuration screen > “Code entry type”, select the drop down’s “Upload a .ZIP file”

Choose the .zip file.  It should be about 1 Mb in size. Click the “Save” button again.

We’re almost ready to run the Lambda function in a Test mode, but first, you will need to include two parameters for our function: Screen name and that user’s score.  They are added via the Test event:

AWS Lambda Test Event Parameters

In the drop down, “Select a test event…”, select “Configure test events”.

Select: Create new test event, Event template (using “Hello World”), and Event name (using “LeadboardInputs”).

Set up a key1 and key2 default mappings to the new user (screen_name) and their most recent score (score):

Configuring Test Event in AWS Lambda

Click the “Create” button on the lower right of the “Configure test event” dialog.

Running the Lambda Function in a Test Mode Prior to Creating the API Gateway

You are ready to click the “Test” button to run the Lambda function.

The function is run using the configured test event input parameters.  A successful execution result will look like this:

Executing AWS Lambda functionw with test event

 

Upon the successful run of the Lambda function, you can review the console within the “log output” subarea of the Details section of the Configuration screen:

START RequestId: 3a7c4e53-4380-11e8-8939-5317dd47e6a3 Version: $LATEST
http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName/FooBarUser.json
<class ‘dict’>
{‘http_code’: 404, ‘description’: ‘The resource could not be found.’}
404
Entering the ScreenName: FooBarUser and Score: 49783 into the Leaderboard table.
response is: <Response [201]>
the next SlashdbURL is http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName,Score.json?sort=-Score&limit=5
response is: <Response [200]>
Returning: congratsorSorryorWelcomeMessage : Welcome as a first-time gamer!
              and inTop5Message is :      You need to work harder to get into the Top 5 Leader Board category.
              and topScoresData is :      [{‘Score’: 970987, ‘ScreenName’: ‘Banksy’}, {‘Score’: 888965, ‘ScreenName’: ‘PhillySteak3’}, {‘Score’: 302905, ‘ScreenName’: ‘PastaEater’}, {‘Score’: 212069, ‘ScreenName’: ‘BusyGirl’}, {‘Score’: 201987, ‘ScreenName’: ‘JimSteph’}]
END RequestId: 3a7c4e53-4380-11e8-8939-5317dd47e6a3
REPORT RequestId: 3a7c4e53-4380-11e8-8939-5317dd47e6a3 Duration: 264.57 ms Billed Duration: 300 ms Memory Size: 128 MB Max Memory Used: 30 MB

And in this iteration of the game play for this user, let’s just up FooBarUser’s score to 900000: Configure test event for LeaderBoardInputs:

{
  "score": "900000",
  "screen_name": "FooBarUser"
}

 

…logging this output:

START RequestId: 0b06e82d-43df-11e8-a0a3-890cda12958a Version: $LATEST
http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName/FooBarUser.json
<class ‘dict’>
{‘ScreenName’: ‘FooBarUser’, ‘Score’: 49784, ‘__href’: ‘/db/Gaming/Leaderboard/ScreenName/FooBarUser.json’}
200
the next SlashdbURL is http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName,Score.json?sort=-Score&limit=5
response is: <Response [200]>
Yay!!! There you are in the top 5 scores
Returning: congratsorSorryorWelcomeMessage : Congratulations, you have a personal best score.
              and inTop5Message is :      Congratulations, you’re in the Top 5 lead scores.
              and topScoresData is :      [{‘Score’: 970987, ‘ScreenName’: ‘Banksy’}, {‘Score’: 900000, ‘ScreenName’: ‘FooBarUser’}, {‘Score’: 888965, ‘ScreenName’: ‘PhillySteak3’}, {‘Score’: 888920, ‘ScreenName’: ‘Xyzzy’}, {‘Score’: 302905, ‘ScreenName’: ‘PastaEater’}]
END RequestId: 0b06e82d-43df-11e8-a0a3-890cda12958a
REPORT RequestId: 0b06e82d-43df-11e8-a0a3-890cda12958a Duration: 76.77 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 31 MB

Varying the Test Input Parameters

Within the “Saved Test Events”,  experiment with different screen names and scores to present, for example, this Log output for the “Leaderboardinputs” event:

{
  "score": "888920",
  "screen_name": "Xyzzy"
}
START RequestId: e7e3faa2-4381-11e8-a59a-ed4077053967 Version: $LATEST
http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName/Xyzzy.json
<class ‘dict’>
{‘http_code’: 404, ‘description’: ‘The resource could not be found.’}
404
Entering the ScreenName: Xyzzy and Score: 888920 into the Leaderboard table.
response is: <Response [201]>
the next SlashdbURL is http://ec2-99-147-99-85.compute-1.amazonaws.com/db/Gaming/Leaderboard/ScreenName,Score.json?sort=-Score&limit=5
response is: <Response [200]>
Yay!!! There you are in the top 5 scores
Returning: congratsorSorryorWelcomeMessage : Welcome as a first-time gamer!
              and inTop5Message is :      Congratulations, you’re in the Top 5 lead scores.
              and topScoresData is :      [{‘Score’: 970987, ‘ScreenName’: ‘Banksy’}, {‘Score’: 888965, ‘ScreenName’: ‘PhillySteak3’}, {‘Score’: 888920, ‘ScreenName’: ‘Xyzzy’}, {‘Score’: 302905, ‘ScreenName’: ‘PastaEater’}, {‘Score’: 212069, ‘ScreenName’: ‘BusyGirl’}]
END RequestId: e7e3faa2-4381-11e8-a59a-ed4077053967
REPORT RequestId: e7e3faa2-4381-11e8-a59a-ed4077053967 Duration: 68.80 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 31 MB

SlashDB: RestFul or Custom Defined – Data Discovery and SQL-Passthru

We saw a number of SlashDB capabilities:

  1. Data Discovery – SlashDB automatically builds a REST-ful API out of connected databases. You can simply browse the data by following hyperlinks between related data resources (records).  The first request in our Python code exhibits this feature and returns a response code of 404 (not found) or 200 (found) to let us know if the gamer is in our database.
  2. Data Discovery – the second request uses the HTTP POST method to insert the new user and her score into the Leaderboard
  3. SQL-Passthru – In the third request we leverage the feature of SlashDB which enables you to execute custom defined queries. More on this at: https://docs.slashdb.com/user-guide/queries.html .  In this example, we built a Query Definition with the SQL statement:

 

    UPDATE Leaderboard SET Score = :Score 
    WHERE ScreenName = :ScreenName

In order to authorize a user to write via SQL Pass-thru the user must be configured in the SlashDB system. In our example the user is MyAPP.  In advance of creating the Lambda function, this user is created under Configure > Users:

 

 

As seen in the screenshot below, the user executing the query (e.g. MyAPP) must be listed in the “Execute” field on the query definition screen. This is done under Configure -> Queries > “edit” button on your query.

 

 

       4. Data Discovery – request the “Top 5” scores on our Leaderboard with the HTTP GET method:

                 http://<your_host>/db/Gaming/Leaderboard/ScreenName,Score.json?sort=-Score&limit=5

When executed from your browser substituting “html” for “json”:

More about sorting with SlashDB can be found at: https://docs.slashdb.com/user-guide/data-discoverymduseful-query-parameters.html#sort

 

Conclusion

With this Lambda function, the client-side developer can now pass the user’s screen name and score and have three concepts returned in one API call requiring no prior knowledge as to the structure of the database, where that database resides or how to manipulate the database (e.g. ORDER BY, Joins to create) for a returned list of values.  SlashDB truly turns any data source into accessible XML, JSON, HTML, or even, as we saw above through a wrapped Lambda function, simple character strings.