Speeding Up Pages in App Engine - Part 3 [django specific]

In past blogs, we've looked at two techniques for speeding up pages. These ideas boil down to:

• Efficient retrieval of information from the datastore.
• Reducing the number of entities in the datastore

The next step is to look at the actual python code serving the requests and speeding it up.

Some background:

MindWell uses Django to serve the requests.  For those unfamiliar, Django is a great project that uses the MVC (Model View Controller) architecture.  In Django you create templates containing the html of the page.  Normally, these templates are great because they offer code reuse and performance is not a problem.  However, on one page I found a performance bottle-neck, so I resolved the issue by skipping the template system altogether.  Sometimes it is good to be different:

Image Source

In MindWell there is a calendar that shows appointment dates and to the user similar to Google Calender.  Mindwell uses FullCalendar which has the ability to take JSON objects and render them on the calendar.  In MindWell there might be 100-300 appointments in one stream of JSON objects.  

Here is one such JSON object:

[

{

"title":"ClientLastName, ClientFirstName",

"start":"2011-01-11T11:00:00",

"end":"2011-01-11T11:45:00",

"allDay":false,

"url":"/Mindwell/2011/01/11/11/00/436/calendar_dos/",

"note":"",

"className":"scheduleClient",

"description":"Session Type:None
Session Result:Scheduled
DSM Code:None
Type of Payment:None
Amount Due:None
Amount Paid:None
Note:None
DOS Date and Time:2011-01-11 11:00:00
DOS Repeat:No
Repeat End Date:None
"

},

...

When I profiled the performance of the template rendering the JSON data, I found it was too slow.  Instead of rendering the objects using a template, I rendered them directly in python.  

Below is the code that renders the feed.  Normally in Django, we return the rendering of a template. Here, however, we create an HttpResponse and text directly, then set the mimetype to json.

#click this link for the code itself
def calendar_feed(request):

    if CheckUserNotAllowed():

        return CheckUserNotAllowed()

    start = request.GET.get('start', None)

    end = request.GET.get('end', None)

    if not start:

        return HttpResponse('')

    if not end:

        return HttpResponse('')

    cal_feed = CalendarFeed(request, 
        datetime.datetime.fromtimestamp(int(start)),
        datetime.datetime.fromtimestamp(int(end)))

    return HttpResponse(

        cal_feed.GetFeed(), mimetype='application/json')


The code that turns each object into a JSON object is shown below.  Note that I still call escape to prevent various Javascript and other attacks.  I want to use the <br/> tag in my JSON object so I put the <br/> tag back in.

#click this link for the code itself
def escape_json(text):
  """ 
  escape html using django (replace single slash with double for
  javascript and finally put back in our<br/>
  """
    return escape(text).replace('\\', 
        '\\\\').replace('&lt;br/&gt;', '<br/>')
        
def Javascript_Object_DOS( dos):
    """ 
    Converts a DOS to JSON.
    """
        return """
{
"title":"%s",
"start":"%s",
"end":"%s",
"allDay":false,
"url":"%s",
"note":"%s",
"className":"%s",
"description":"%s"
},
            """ %(
                escape_json(unicode(dos)), 
                dos.get_starttime().isoformat('T'),
                dos.get_endtime().isoformat('T'), 
                dos.get_calendar_absolute_url(),
                escape_json(dos.get_note()), 
                dos.get_class_name(), 
                escape_json(dos.get_hover_tip())
            )

By replacing the Django template system, I was able to reduce the rendering time of the JSON data by ~30%.  The rendering times in some of the bigger streams were reduced from ~650ms to ~400ms.

The bottom line is this.  While Django's template system is good, there is some overhead associated with using Django's templates. In some cases, it may be worth skipping the templates for your slower or high traffic pages.