Here is my use case. A few days ago I've wrote an application to mirror my Flickr accounts. The pics are downloaded on file system and the metadata are stored in CouchDB. Now I use the Flickr interface to upload and tags pics but I got my own data on my own server. That's always cool.

Well done but now it can be useful to have a small web app to see the pics. Right ? That's what I've do. Since I love jQuery and CouchDB is full json compliant I don't want to write a complex app with tones of python code. So the idea is to have a small wsgi app to only serve static javascripts/html/css files and a proxy app to serve json by proxying CouchDB.

For now I'm using WSGIProxy. The code is very simple and look like this:

from wsgiproxy.exactproxy import proxy_exact_request
from webob import Request

class Proxy(object):
    def __init__(self, db=None, **kwargs):
        self.db = db
    def __call__(self, environ, start_response):
        req = Request(environ)
        if req.method == 'GET':
            req.server_name = '127.0.0.1'
            req.server_port = 5984
            req.script_name = ''
            req.path_info = '/%s%s' % (self.db, req.path_info)
            resp = req.get_response(proxy_exact_request)
            resp.content_type = 'text/jasacsript'
        else:
            resp = exc.HTTPForbidden()
        return resp(environ, start_response)


def make_app(global_conf, **local_conf):
    conf = global_conf.copy()
    conf.update(local_conf)
    return Proxy(**conf)

That's cool and simple and ok for small files. But if you want to handle large request WSGIProxy will raise a MemoryError. This is no longer fun. I've already sent a patch but don't want to bother the Paste team with that.

Then I've try to use restkit and thought that it can be the best library to wrote a proxy app. restkit manage a pool of http connection and handle large file request in a clean way.

The result is a small contribution. A set of WSGI applications included in the wsgi_proxy extention.

Here is a simple Paste config file to show own to use the proxies:

[server:main]
use = egg:Paste#http
port = 4969

[app:main]
use = egg:Paste#urlmap
/couchdb = couchdb
/ = proxy

[app:couchdb]
use = egg:restkit#host_proxy
uri = http://localhost:5984/mydb

[app:proxy]
use = egg:restkit#host_proxy
uri = http://benoitc.github.com/restkit/
max_connections=50
allowed_methods = get head post

No code needed. Cheers.

You can also use the Proxy class to proxify clients request and transform the response on the fly:

from webob import Request
from restkit.ext.wsgi_proxy import Proxy

proxy = Proxy()

def application(environ, start_response):
    req = Request(environ)
    req.environ['SERVER_NAME'] = 'example.com'
    req.environ['SERVER_PORT'] = '80'
    # do stuff
    ...
    resp = req.get_response(proxy)

    # do stuff ...
    ...

    return resp(environ, start_response)

I've also tried to replace WSGIProxy in deliverance with an ugly monkey patch:

from restkit.ext.wsgi_proxy import Proxy
from wsgiproxy import exactproxy
print 'Patching exactproxy with restkit proxy'
exactproxy.proxy_exact_request = Proxy(max_connections=4, allowed_methods=['GET', 'HEAD', 'POST']).__call__

It work perfect.

Most (good) web frameworks have a way to avoid Cross Site Request Forgery but in the Ajax world it's not so easy.

Here is my solution. First generate a secret key (sha1 of the current date or whatever) store it in user's session and show an hidden field in the requested webpage:

<input type="hidden" id="_req" name="_req" value="your secret key" />

Here is a snippet for Pylons:

def secure_field(with_id=False):
    value = request.environ.get('_req', None)
    if value is None:
        # generate a key if not already done for the request
        value = sha.new('%s-%s' % (datetime.now(), random.random())).hexdigest()
        session['_req'] = value
        request.environ['_req'] = value
        log.info('setting secure key to %r', value)
    if with_id:
        return hidden('_req', id='_req', value=value)
    return hidden('_req', value=value)

Notice that the field can be render multiple time with the same key (but only one with an id because of XHTML). The key is generated per request. This allow to have multiple _req fields in the same page so you can also add it to non-Ajax forms (and secure them too).

Then you need to POST this key on each Ajax request. Here is a wrapper for jQuery's post method:

post: function(url, data, callback, dataType) {
    // wrap $.post to add _req field
    if (!dataType) dataType = 'html';
    if (typeof(data) == typeof('')) {
        // $(form).serialize() return a string
        data += '&_req='+$('#_req').val();
    } else {
        data['_req'] = $('#_req').val();
    }
    $.post(url, data, callback, dataType);
}

The _req key is added to each POST request.

Last thing. You need to check that the key stored in user's session is also in the POST data. Here two Pylons decorators to avoid illegal requests:

@decorator
def secure_post(func, *args, **kwargs):
    """return html"""
    if request.method == 'POST':
        _req = session.get('_req', None)
        if _req is not None and _req == request.POST.get('_req'):
            del request.POST['_req']
            data = func(*args, **kwargs)
            return data
    if request.environ.get('paste.testing') is True:
        return func(*args, **kwargs)
    return _('Forbidden')

@decorator
def secure_json(func, *args, **kwargs):
    """return json"""
    if request.method == 'POST':
        _req = session.get('_req', None)
        if _req is not None and _req == request.POST.get('_req'):
            del request.POST['_req']
            data = func(*args, **kwargs)
    if request.environ.get('paste.testing') is True:
        data = func(*args, **kwargs)
    else:
        data = dict(error=_('Forbidden'))
    response.content_type = 'application/json'
    return json.dumps(data)

That's it. Now you are sure that all Ajax requests came from the user's web page. Cheers.

Of course the key is valid for more than one Ajax request. You may regenerate it for each main html page. May be this can be improved to change the key for each request including Ajax's but... It's already secure. Right ?

Voilà, il fallait s'y attendre. Je suis remonté contre les artistes. Tous cela à cause de l'HADOPI. Comment c'est arrivé me direz vous ? C'est simple...

Je jouais les insomniaques un soir en regardant M6 et les plus ou moins bon clips qu'il y passe en début de nuit. Et là... ET LA !!! Je tombe sur l'un des derniers clips de Thomas Dutronc. A priori, je n'ai pas grand chose contre lui et j'ai toujours apprécié les chansons de papa. Mais remettons les choses dans leur contexte.

Jacques Dutronc est un artiste reconnu qui a fait sa fortune pendant l'age d'or du vinil et du CD. Les fameuses galettes au oeufs d'or. Je trouvais déjà donc un peu abusé que le fils soit un fervent défenseur du droit d'auteur lors de la bataille HADOPI. En effet, entre les héritages de papa et de maman, je doute que ce jeune homme ai besoin de travailler un jour (A moins qu'il ne se soit fait couillonner par Bernard Madof ce qui expliquerait d'ailleurs le récent retour sur scène de papa, bref). Mais voilà...

Revenons sur ce fameux clip. Il s'agissait de la chanson du petit Thomas: NASDAQ. (Évidement, je ne peux pas intégrer ce clip dans mon billet à cause des droits d'auteurs). Mais, monsieur Dutronc a t'il réalisé qu'il profite de tout ce qu'il critique dans sa chanson ? Peut-on m'assurer que la famille Dutronc n'a aucun compte dans un paradis fiscale ? Qu'elle paye tous ses impôts ? Qu'elle ne profite pas de quelques compagnies de courtage pour accroître sa fortune sans bouger le petit doigt ? Vous m'excuserez, mais j'ai comme un léger doute... Après tout, cette chanson est peut-être plus un résumé de la vie de la dynastie Dutronc plutôt qu'une critique... D'ailleurs, c'est une pratique courante. Dans le reportage suivant, à 11'30", on nous explique comment certains artistes utilisent les paradis fiscaux pour économiser des millions d'impôts:

Certes, ce n'est pas une majorité. Beaucoup d'artistes lutent pour vivre. Voir survivre... Mais la vrai question est comment permettre à tout les artistes de vivre. Je n'ai pas vraiment de réponses, mais il en existe. En tout cas, un bon premier pas serait de bâillonner Thomas Dutronc, pour ne pas pourrir la réflexion. Ensuite pourrions nous abandonner La bataille HADOPI. Puis voir les alternatives que certains propose.

Tout cela pour conclure que ce n'est pas demain la veille que j'irais voir Thomas en concert ou achèterai un de ces disques. C'est dommage car étant donné que le fils est un parfait copié-collé du père ce qu'il faisait ne me déplaisait pas, avant... Merci Thomas pour m'avoir ouvert les yeux sur cette hypocrisie. Et merci HADOPI !

J'ai juste envie de conserver ces deux vidéos dans un coin. L'une comme l'autre sont impressionante et les tricks d'une originalité surprenante.

FormAlchemy 1.3 is released. From the website:

FormAlchemy eliminates boilerplate by autogenerating HTML input fields from a
given model. FormAlchemy will try to figure out what kind of HTML code should
be returned by introspecting the model's properties and generate ready-to-use
HTML code that will fit the developer's application.

Why I choose FormAlchemy instead of another form library ? Everybody knows that explicit is better than implicit. So most of form libraries use a schema to define how widgets are render. FormAlchemy avoid that by using the schema of the data model (SQLAlchemy mappers at the origin). So it generate forms implicitly using your explicit data model.

Another reason is that FormAlchemy is independent. This mean that you can use it in Pylons, Django, repoze.bfg, bobo and (put your favorite framework here).

Thats cool. But we can do more. And that's what we tried to do. SQLAlchemy is not the only library to define data model. So let use the others !

>>> from formalchemy.ext import couchdb
>>> from couchdbkit import schema
>>> class Pet(couchdb.Document):
...     name = schema.StringProperty()

>>> fs = FieldSet(Pet)
>>> fs.bind(Pet())
>>> print fs.render()

>>> from zope import interface
>>> from zope import schema

>>> class IPet(interface.Interface):
...     name = schema.TextLine(u'name')

>>> class Pet(object):
...     interface.implements(IPet)

>>> from formalchemy.ext.zope import FieldSet
>>> fs = FieldSet(IPet)
>>> fs = fs.bind(Pet())
>>> print fs.render()

Shabti

Shabti is a set of pylons templates initiated by Graham Higgins. There is now a FormAlchemy template in Shabti to quick initialize a pylons project with FormAlchemy and fa.jquery. The template also initialize a CRUD interface in the admin controller for you.

Here is some screenshots of the admin UI using fa.jquery:

User listing:

User edit form:

Notice that at this time Shabti require pylons-dev.