Quick test Flask, Flask-Restplus and Flask-Marshmallow

I needed to quickly test a few things today using Flask, Flask-Restplus and Flask-Marshmallow. I created the following boilerplate code that can easily be modified for further quick tests.

from flask import Flask, Blueprint, url_for, jsonify
from flask_restplus import Api, Resource
from flask_marshmallow import Marshmallow, base_fields
from marshmallow import post_dump

# Setup Flask app

url_prefix = '/api/v4'
doc = '/apidoc/'
app = Flask(__name__)
app.config['JSON_SORT_KEYS'] = False
ma = Marshmallow()
blueprint = Blueprint('api', __name__, url_prefix=url_prefix)
api = Api(blueprint, doc=doc, version='3.0')

# Models

class Bunch(dict):
    def __init__(self, *args, **kwargs):
        super(Bunch, self).__init__(*args, **kwargs)
        self.__dict__ = self

class Author(Bunch):
    pass

def mock_author():
    author = Author(id=123, name='Fred Douglass')
    return author

def mock_author_list():
    a1 = Author(id=1, name="Alice")
    a2 = Author(id=2, name="Bob")
    a3 = Author(id=3, name="Carol")
    return [a1, a2, a3]

# Schemas

class AuthorSchema(ma.Schema):
    id = base_fields.Int(dump_only=True)

    absolute_url = ma.AbsoluteURLFor('api.author', id='<id>')

    links = ma.Hyperlinks({
        'self': ma.URLFor('api.author', id='<id>'),
        'collection': ma.URLFor('api.authors')
    })

    @post_dump(pass_many=True)
    def wrap(self, data, many):
        key = 'authors' if many else 'author'
        return {
            key: data
        }

    class Meta:
        fields = ('id', 'name', 'links', 'absolute_url')
        ordered = True

# Setup operations

ns = api.namespace('authors', description='desc')
ep_1 = 'authors'
ep_2 = 'author'

@ns.route('/', endpoint=ep_1)
class AuthorsCollection(Resource):
    def get(self):
        print " ---- GET - result=" + url_for('api.' + ep_1)
        s = AuthorSchema(many=True)
        result = s.dump(mock_author_list())
        return jsonify(result.data)

    def post(self):
        print " ---- POST - result=" + url_for('api.' + ep_1)
        return None, 201

@ns.route('/<int:id>', endpoint=ep_2)
class AuthorsDetail(Resource):
    def get(self, id):
        print " ---- GET - result=" + url_for('api.' + ep_2, id=id)
        print " ---- GET - result=" + url_for('api.' + ep_1)
        s = AuthorSchema()
        result = s.dump(mock_author())
        return jsonify(result.data)

    def put(self, id):
        print " ---- PUT - result=" + url_for('api.' + ep_2, id=id)
        print " ---- PUT - result=" + url_for('api.' + ep_1)
        return None, 204

    def delete(self, id):
        print " ---- DELETE - result=" + url_for('api.' + ep_2, id=id)
        print " ---- DELETE - result=" + url_for('api.' + ep_1)
        return None, 204

app.register_blueprint(blueprint)

if __name__ == '__main__':
    print '>>>>> Starting server at http://localhost:8080{url_prefix}{doc}'.\
        format(url_prefix=url_prefix, doc=doc)
    app.run(port=8080, debug=True)

Marshmallow not respecting ordered=True

I had a little fight with Marshmallow and the ordered=True in class Meta. When ordered=True is set the fields should be outputed in the order specified in the property fields of the class Meta.

Example:

class AuthorSchema(ma.Schema):
    id = base_fields.Int(dump_only=True)

    absolute_url = ma.AbsoluteURLFor('api.author', id='<id>')

    links = ma.Hyperlinks({
        'self': ma.URLFor('api.author', id='<id>'),
        'collection': ma.URLFor('api.authors')
    })

    @post_dump(pass_many=True)
    def wrap(self, data, many):
        key = 'authors' if many else 'author'
        return {
            key: data
        }

    class Meta:
        fields = (
            'id',
            'name',
            'links',
            'absolute_url',
        )
        ordered = True

This was the output.

{
  "author": {
    "absolute_url": "http://localhost:8080/api/v4/authors/123",
    "id": 123,
    "links": {
      "collection": "/api/v4/authors/",
      "self": "/api/v4/authors/123"
    },
    "name": "Fred Douglass"
  }
}

After some Googeling I found out that the culprit was Flask.jsonify. It orders output alphabetically by default. The following setting switches this off.

app.config['JSON_SORT_KEYS'] = False

After this the output was in the specified order.

{
  "author": {
    "id": 123,
    "name": "Fred Douglass",
    "links": {
      "self": "/api/v4/authors/123",
      "collection": "/api/v4/authors/"
    },
    "absolute_url": "http://localhost:8080/api/v4/authors/123"
  }
}

Hope this saves someone else time.

Changing location of virtual environments breaks virtualenvwrapper-win

Today I wanted to move the location of my virtual environments to Dropbox. Makes life easier if I ever have to change computers. I moved the directory that contains the virtual environments and set the system variable WORKON_HOME to the new location. Then I tried it out. Activating the virtual environment with command workon worked just fine. Then I wanted to switch to the virtual environment with the command cdvirtualenv and it failed with an error saying that the location did not exist.

Puzzled I checked the system variables with the command set (I’m running Windows 10) and I noticed the system variable VIRTUAL_ENV that was pointing to the old location. I checked my variable settings under “System” → “Advanced system settings” → “Environment Variables” and could not find it there. Which meant it was being set somewhere when activating the virtual environment. After searching all the virtualenvwrapper-win scripts I could not find “SET VIRTUAL ENV=”. Then I noticed that the “workon.bat” file called “activate.bat” located in the script directory of the virtual environment. Here I found the “SET VIRTUAL ENV=” which contained a hard coded location, which was the old WORKON_HOME location as the virtual environment already existed.

To fix the problem I had to change this line of each “activate.bat” file manually. After that it worked like a dream. Problem solved!

By the way this isn’t a virtualenvwrapper-win problem, but a virtualenv problem.

UPDATE 4-1-2017

During a little break laying in the bath I came up with a much quicker and easier approach to solving the problem… Creating a symbolic link from the old location to the new location.

Just fire up a console with administrator rights and execute the following command.

mklink /d <old directory location> <new directory location>

Hmmmmm, that was quick 🙂

More Flask debugging tricks

I was busy debugging a rest services I’m building in Flask and Flask-Restplus. I needed a way to log the request and response. I found some code that can be used as a wrapper of the WSGI application that does just this.

Create a file called dubug.py that contains the following code.

import pprint

class RequestLoggingWrapper(object):
    def __init__(self, app):
        self._app = app

    def __call__(self, environ, resp):
        errorlog = environ['wsgi.errors']
        pprint.pprint(('REQUEST', environ), stream=errorlog)

        def log_response(status, headers, *args):
            pprint.pprint(('RESPONSE', status, headers), stream=errorlog)
            return resp(status, headers, *args)

        return self._app(environ, log_response)

This wrapper works independently from Flask and wraps the WSGI application. It shows exactly what request is going in and what response is going out.

When running Flask with the built-in server you can use it as follows.

from debug import RequestLoggingWrapper

if __name__ == '__main__':
    app.wsgi_app = RequestLoggingWrapper(app.wsgi_app)
    app.run()

The output goes to the wgi.error stream. For the built Flask server it is printed to stderr.

Flask request debugging

I needed to debug the Flask request. After googeling around for a while I ran into a cool trick on Stack Overflow using the pprint module. The pprint module provides a capability to “pretty-print” arbitrary Python data structures in a form which can be used as input to the interpreter.

import pprint
str = pprint.pformat(request.environ, depth=5)

This same trick can be used with all the Flask variables.

  • request.args: the key/value pairs in the URL query string
  • request.form: the key/value pairs in the body, as sent by a HTML POST form
  • request.files: the files in the body, which Flask keeps separate from form
  • request.values: combined args and form, preferring args if keys overlap

Cheatsheet virtualenvwrapper

This is a cheatsheet for virtualwrapper.

Install virtualenvwrapper

pip install virtualenv

pip install virtualenvwrapper-win

Set environment variable

Add an environment variable WORKON_HOME to specify the path to store environments. By default, this is %USERPROFILE%\Envs. I have set it to the following.

WORKON_HOME=%PYTHON27%\env

Main commands

mkvirtualenv <name>

lsvirtualenv

workon <name>

cdvirtualenv

deactivate

add2virtualenv <full or relative path>

setprojectdir <full or relative path>

cdproject

cdsitepackages

lssitepackages

 

Cleanup the default Python environment

I have been messing around with Python these last few weeks. Playing with Python means installing a lot of packages. I did this without using virtualenv, so my default Python environment had a lot of packages installed globally.

I now have a couple of projects that I want to work on and I what each project to have its own clean environment. That means using virtualenv. When running virtualenv without with the option --no-site-packages all the packages that are installed globally are included in the virtualenv.

I wanted to remove all the global packages to have a clean default environment. I did this by running the following command.

Windows

pip freeze > remove.txt && pip uninstall -y -r remove.txt && del remove.txt

Linux

pip freeze > remove.txt && pip uninstall -y -r remove.txt && rm remove.txt

After cleaning up the default envrionment by removing all the global packages don’t forget to install virtualenv en virtualenvwrapper-win by running the following command.

pip install virtualenvwrapper-win

For quick instructions of the main commands please check out virtualenvwrapper-win Github page.