Objectlab LLC - Party App

Server

In order to capture the activity being driven by the IOS App as well as drive the D3.js visualizations, we needed a server. The server provided both a network data storage service and general use-case ‘controller’ coordinating the following functions:

User Registration
User Image Upload
Bar Score Update
iBeacon claiming
Update Leaderboards

Since it was also a web server, it would serve up the images, HTML, javascript, and CSS files for the real-time activity displays.

Components

The following describes the software components that were used to build the server. A more detailed description and installation instructions can be found here: https://github.com/objectlab/ObjectlabPartyServer

Hosting

Since I had previously used Google App Engine for app server integration, I briefly considered utilizing it for this application. There are some advantages to using GAE, namely that hosting, scaling, and database services are built-in. For specific reasons (stated below), however, I chose to deploy on Amazon Web Services. I quickly spun up a medium-sized i Ubuntu instance on AWS. Depending on your needs and capabilities, this is often the least painful way to go and the tools that Amazon gives you are really nice, in my opinion.

Database

I’ve used MySQL for other ‘quick-and-dirty’ development projects in the past, so I settled on that. It’s also relatively simple to install on Ubuntu Linux.

Language

Although I use Java almost every day, Python is my go-to. Nothing beats the expressiveness of the language, it’s rich ecosystem, or its speed of deployment. Java is a great language, and the JVM is a battle-tested, mature technology. However, this is both a blessing and a curse as it stumbles under its own weight at times. Simple things should be simple, but they’re not always with this language. This is particularly true when building when building web applications.

App Server

Most of what the server had to do was handle service requests from the IOS app and D3.js visualizations. Given the time constraints, I needed a way to create this “API” quickly. Given that we had a few streams of development going concurrently, and multiple developers, I felt that we needed more control of the HTTP service tier and OS-level filesystem access.

Since this is a ‘micro’-style web application, I decided on a ‘micro framework’. If you’re using Python, then one of the easiest and complete micro frameworks is Bottle.

The Bottle website describes it as ‘fast, simple, and lightweight’ and it is. It supports request-to-function call routing, page templates, and ships with its own web server. Here’s an example web application from the Bottle website:


from bottle import route, run, template

@route('/hello/<name>')
def index(name):
    return template('<b>Hello {{name}}</b>!', name=name)

run(host='localhost', port=8080)

That’s it. Since it’s based on Python, it’s easy to install other components as well. For our app, we utilized three external Python libraries:

MySQLdb: for mysql access from Bottle web service methods

SimpleJSON: for serializing data into JSON service responses

Beaker: for session management

Bottle is somewhat limiting in that it’s really designed for a single page deployment. That is, the server is entirely composed of a single python script which contains all of the web service implementation code. It’s great for a small ‘micro’ application a la Google App Engine. However, it’s not great for large website development, in my opinion. There are better frameworks for that, namely Pyramid/Pylons, Django, or Flask.

Schema

Given the compressed implementation timeframe, we decided to go with a very simple de-normalized table structure. Essentially, the database existed of a list of users, a link to their current profile image, and a list of counters for their bar score, and each ibeacon they found. Frankly, not the most extensible design.

However, we knew the UUIDs for each beacon, where they were (geographically), and what beacon number in the database it corresponded to, so we fixed that into the service protocol. This made inserts, updates fast and services themselves easy to write. Obviously a better approach would be to normalize the relationship between user, location, and device (so we leave that for an exercise for the reader).

Here is the schema for the table:


CREATE TABLE USER (
  USER_ID int(11) NOT NULL AUTO_INCREMENT,
  NAME varchar(255) NOT NULL DEFAULT '',
  IMG_REF varchar(255) DEFAULT NULL,
  BAR_SCORE int(11) NOT NULL DEFAULT '0',
  BEACON_1 int(11) NOT NULL DEFAULT '0',
  BEACON_2 int(11) NOT NULL DEFAULT '0',
  BEACON_3 int(11) NOT NULL DEFAULT '0',
  BEACON_4 int(11) NOT NULL DEFAULT '0',
  BEACON_5 int(11) NOT NULL DEFAULT '0',
  BEACON_6 int(11) NOT NULL DEFAULT '0',
  BEACON_7 int(11) NOT NULL DEFAULT '0',
  BEACON_8 int(11) NOT NULL DEFAULT '0',
  FOUND_EGGS int(11) NOT NULL DEFAULT '0',
  LAST_BAR_DETECTION timestamp NULL DEFAULT NULL,
  LAST_BEACON_CLAIM timestamp NULL DEFAULT NULL,
  DEVICE_ID varchar(255) DEFAULT '',
  UNIQUE KEY `USER_ID` (`USER_ID`)
);

Protocol

Since JSON is easily constructed and parsed in Cocoa, Javascript, and Python, we elected to go with a simple JSON-based protocol to move data between the clients (IOS App and D3 visualization) and the server.

The protocol itself was implemented as a REST-style HTTP definition which, again, is easily implemented in all of the endpoints. Here is an example API call to register a user:


POST http://partyserver/register 
{
	“user” : “cmollis”, 
	“device” : “14B4A219-AD7A-46C2-9773-7880ADD23876”
}

Response:
{
	“responseCd” : 0,  //zero if error 
	“responseMsg” : “user created”, //or error message
	“userId” : 23  //id of the new user
}

Here we allow the user (on the IOS App) to choose their own name which is associated with a device ID on the handset. This device ID is obtained via a call to


NSUUID *device = [[UIDevice currentDevice] identifierForVendor];

The server ensures that a username is unique per device (note however that if the user deletes their app and re-installs it, they will get a new device ID).

Here is the service handler for user registration:


@route('/register', method='POST')
def register_user():
        responseObj = {}
        responseObj = {}
        dataObj = {}

        # JSON input
        device = request.json["device"]
        name = request.json["name"]

        # regular POST input
        #device = request.forms.get('device')
        #name = request.forms.get('name')

        try:
                con = mdb.connect(serverName, userName, password, database)
                cur = con.cursor(mdb.cursors.DictCursor)

                checkUser = "select USER_ID from USER where device_id = %s"
                cur.execute(checkUser, (device) )

                row = cur.fetchone()

                if row:
                        responseObj["responseCd"] = 1
                        responseObj["responseMsg"] = "user already exists"
                        responseObj["userId"] = (row["USER_ID"])
                else:
                        #create this user 

                        createUser = "insert into USER(NAME, DEVICE_ID) values (%s, %s)"
                        cur.execute(createUser, (name, device) )

                        rows = cur.fetchall()

                        responseObj["responseCd"] = 0
                        responseObj["responseMsg"] = "user created"
                        responseObj["userId"] = (con.insert_id())

        except mdb.Error, e:
                con.rollback()
                responseObj["responseCd"] = -1
                errMsg = "error occurred %d %s", (e.args[0], e.args[1] )
                responseObj["responseMsg"] = errMsg

        finally:
                if con:
                        con.commit()
                        con.close()

        response.headers['Content-Type'] = 'application/json'
        return json.dumps(responseObj)

Since the client sets the content type = ‘application/json’, Bottle automatically parses the JSON POST body into Python dictionary and attaches it to the request.

The database is queried for the user/device combination and, if not already created, creates the user. A dictionary object is built with the response elements and serialized as JSON back to the caller.

Deployment

Although the HTTP server that ships with Bottle is fine for development, it wouldn’t be scaleable enough for a ‘live’ deployment, nor would it be able to serve up images or any other static file. Because it wasn’t clear what type of load we would have (since there was no time to actually test it), I settled on a combination of NGINX and uWSGI.

uWSGI

uWSGI is an application service ‘container’ written in C for performance, for applications that conform to the WSGI (Web Service Gateway Interface) standard. Since Bottle itself is a WSGI server, it can be deployed into any container that supports the standard, like uWSGI. It has many configuration options, but the following describes how we deployed it:


uwsgi --http :9080 --wsgi-file /home/cmollis/partyrockin/partyrockin.py --master --processes 20 --threads 4

Here we specify that we will pre-fork twenty instances of our partyrockin server on port 9080 with 4 connection threads per process.

NGINX web server

NGINX served two functions within our app: (1) it was deployed as a reverse-proxy to the uWSGI server; and (2) it efficiently served up the static images, HTML, Javascript files.

Regarding (1), I like this approach because it’s very simple to add other app machines and processes if the application begins to slow down.

Source Code Available at: https://github.com/objectlab/ObjectlabPartyServer

Server App