Example WSGI server using celery extension

The simple WSGI example provides a concise example of how to use bugzscout, but it is not suitable for production use. In a production environment, the bugzscout reporting should not make an HTTP request to a FogBugz server during another request. It is much more performant to do so asynchronously.

Celery is an asynchronous task queue that supports several backend implementations, like AMQP. The bugzscout package comes with a Celery extension that has a flexible configuration.

The celery extension is the recommended way to submit errors in production environments. This example describes how to setup and use the celery extension in the context of a WSGI server application.

Prerequisite

The celery package needs to be installed in the current environment before this will work. Celery is not declared as a dependency, since it can be bulky and it is not needed to use the core bugzscout functionality.

pip install celery

Celery setup

The rest of this example will assume the code is being added to a module called, myapp.py. The module first needs to import the celery extension and set a global celery variable.

import bugzscout.ext.celery_app
celery = bugzscout.ext.celery_app.celery

Setting a direct reference to the celery instance in bugzscout will be important when consuming the messages in the celery worker.

Next, create a WSGI app. This example provides a slightly more interesting app, which returns different responses based on the HTTP request method.

def app(environ, start_response):
    """Simple WSGI application. Returns 200 OK response with 'Hellow world!' in
    the body for GET requests. Returns 405 Method Not Allowed for all other
    methods.

    Returns 500 Internal Server Error if an exception is thrown. The response
    body will not include the error or any information about it. The error and
    its stack trace will be reported to FogBugz via BugzScout, though.

    :param environ: WSGI environ
    :param start_response: function that accepts status string and headers
    """
    try:
        if environ['REQUEST_METHOD'] == 'GET':
            start_response('200 OK', [('content-type', 'text/html')])
            return ['Hellow world!']
        else:
            start_response(
                '405 Method Not Allowed', [('content-type', 'text/html')])
            return ['']
    except Exception as ex:
        # Call start_response with exception info.
        start_response(
            '500 Internal Server Error',
            [('content-type', 'text/html')],
            sys.exc_info())

        # Record the error to FogBugz and this will return the body for the
        # error response.
        return _handle_exc(ex)

The _handle_exc function, invoked when an exception is caught, will call the celery extension. That will publish a task to be consumed by a celery worker. By returning [''], the response body will be of length zero.

def _handle_exc(exception):
    """Record exception with stack trace to FogBugz via BugzScout,
    asynchronously. Returns an empty string.

    Note that this will not be reported to FogBugz until a celery worker
    processes this task.

    :param exception: uncaught exception thrown in app
    """
    # Set the description to a familiar string with the exception
    # message. Add the stack trace to extra.
    bugzscout.ext.celery_app.submit_error.delay(
        'http://fogbugz/scoutSubmit.asp',
        'error-user',
        'MyAppProject',
        'Errors',
        'An error occurred in MyApp: {0}'.format(exception.message),
        extra=traceback.extract_tb(*sys.exc_info()))

    # Return an empty body.
    return ['']

Finally, add a main block to run the Paste httpserver.

if __name__ == '__main__':
    import paste.httpserver
    paste.httpserver.serve(app, host='0.0.0.0', port='5000')

The full myapp.py module is below.

Overview of WSGI app

When an error is thrown while processing a request, this app will:

1. Catch the error.
2. Call start_response with exception info, which is a WSGI convention for returning error responses.
3. Call _handle_exc.
4. _handle_exc will publish a new task with the error message and stack trace. The description and extra are the same as the simple WSGI example. The call to bugzscout.ext.celery_app.submit_error.delay will do different things depending on how celery is configured. If an AMQP broker is used, the call will publish a message to an exchange. Later, a celery worker will consume that message.
5. Finally, _handle_exc returns [''] so the response body will be empty. Since this is designed to work in production environments, exposing internal stack traces is not be ideal.

At this point, each error is creating a new celery task. There needs to be a celery worker present to consume those tasks.

Configuring celery

The celery instance can be configured as described in the celery docs. For example, it can be configured to use a rabbitmq-server as a broker with:

celery.conf.update(
    BROKER_URL='amqp://user:pass@rabbitmq-server:5672//'
)

Celery worker setup

Since a celery variable is set in the module, the celery executable can use it to start a worker. For example:

celery --app=myapp worker --loglevel=DEBUG

Running celery in this way is suitable for development. For production environments, see the Running the worker as a daemon section of the celery docs.

Full myapp.py source

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""A simple WSGI server that reports errors to FogBugz via BugzScout
asynchronously.
"""

from __future__ import print_function, unicode_literals

import bugzscout.ext.celery_app
import sys
import traceback

# Set celery here, so this module can be designated as the app when running the
# celery work.
celery = bugzscout.ext.celery_app.celery


def _handle_exc(exception):
    """Record exception with stack trace to FogBugz via BugzScout,
    asynchronously. Returns an empty string.

    Note that this will not be reported to FogBugz until a celery worker
    processes this task.

    :param exception: uncaught exception thrown in app
    """
    # Set the description to a familiar string with the exception
    # message. Add the stack trace to extra.
    bugzscout.ext.celery_app.submit_error.delay(
        'http://fogbugz/scoutSubmit.asp',
        'error-user',
        'MyAppProject',
        'Errors',
        'An error occurred in MyApp: {0}'.format(exception.message),
        extra=traceback.extract_tb(*sys.exc_info()))

    # Return an empty body.
    return ['']


def app(environ, start_response):
    """Simple WSGI application. Returns 200 OK response with 'Hellow world!' in
    the body for GET requests. Returns 405 Method Not Allowed for all other
    methods.

    Returns 500 Internal Server Error if an exception is thrown. The response
    body will not include the error or any information about it. The error and
    its stack trace will be reported to FogBugz via BugzScout, though.

    :param environ: WSGI environ
    :param start_response: function that accepts status string and headers
    """
    try:
        if environ['REQUEST_METHOD'] == 'GET':
            start_response('200 OK', [('content-type', 'text/html')])
            return ['Hellow world!']
        else:
            start_response(
                '405 Method Not Allowed', [('content-type', 'text/html')])
            return ['']
    except Exception as ex:
        # Call start_response with exception info.
        start_response(
            '500 Internal Server Error',
            [('content-type', 'text/html')],
            sys.exc_info())

        # Record the error to FogBugz and this will return the body for the
        # error response.
        return _handle_exc(ex)


if __name__ == '__main__':
    import paste.httpserver
    paste.httpserver.serve(app, host='0.0.0.0', port='5000')