Archive for the ‘Framework’ Category

Converting Pylons 1.0 Repoze.who/Repoze.what to Pyramid 1.0a1 Authentication backed with MySQL

Thursday, November 11th, 2010

For one of our projects we used Pylons 1.0 with Repoze.who/Repoze.what, though, we only used groups without permissions. In our application, a member could access one of three controllers based on their membership in a group. There are a number of methods that can be used to set up authentication. One method is to create a thin table with the Primary Key, Username and Password and use an association table to add any additional information. This has the ability to be flexible when you need to add a number of fields that shouldn’t be contained within the AuthUser table. Another method is to use the AuthUser table to hold the related information. This example uses the latter method.

The first changes made are to __init__.py to add the policies and Forbidden view. This view allows us to prompt a user for a username/password when they visit a page that is protected by the ACLs in Pyramid.

auth.py contains our authentication model with the Permission models removed. We put the login/logout views, forbidden view along with the groupfinder and the RootFactory in login.py to consolidate authentication into an auth.py and login.py file for installations in other applications. Last but not least is a simple template to present the user with a login page.

For our RootFactory, we’ve defined three groups, client, manager and admin. Within our __init__.py (or our views if we use add_handler), we can restrict permissions using Pyramid’s authentication.

In our __init__.py, we can use the permission= to specify membership in a group.

config.add_route('admin2', '/admin/', view='project.admin.index', permission='admin', view_renderer='admin_index.jinja2')

If we have used add_handler:

config.add_route('admin2', '/admin/', view='project.admin.index', view_renderer='admin_index.jinja2')

We can use the permissions ACL in the @action decorator:

@action(renderer='admin_index.jinja2', permission='admin')

To access the userid in your views:

from pyramid.security import authenticated_userid

userid = authenticated_userid(request)

At this point, we’ve migrated a Repoze.who/Repoze.what authentication scheme that only used Group membership as its criteria and we have an SQL backed authentication system under Pyramid.

Most of the guidance for this came from:

* http://docs.pylonshq.com/pyramid/dev/tutorials/wiki2/authorization.html

Modifications to __init__.py:

Added to the import section:

from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy

from project.login import forbidden_view, groupfinder
from pyramid.exceptions import Forbidden

Added to the Configurator:

    authn_policy = AuthTktAuthenticationPolicy(
        'sosecret', callback=groupfinder)
    authz_policy = ACLAuthorizationPolicy()
    config = Configurator(settings=settings,
                          root_factory='project.login.RootFactory',
                          authentication_policy=authn_policy,
                          authorization_policy=authz_policy)

Added within the begin()/end() block of the Configurator

    config.add_view(forbidden_view, context=Forbidden)
    config.add_route('login', '/login',
                     view='project.login.login',
                     view_renderer='project:templates/login.pt')
    config.add_route('logout', '/logout',
                     view='project.login.logout',)

auth.py:

import transaction

from sqlalchemy import create_engine
from sqlalchemy import Column
from sqlalchemy import Integer
from sqlalchemy import Unicode

from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.declarative import declarative_base

from sqlalchemy.orm import scoped_session
from sqlalchemy.orm import sessionmaker

from zope.sqlalchemy import ZopeTransactionExtension

DBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
Base = declarative_base()

from sqlalchemy import *
from sqlalchemy.databases import mysql
from sqlalchemy.orm import relation, backref, synonym
from sqlalchemy.orm.exc import NoResultFound

import os
from hashlib import sha1
from datetime import datetime

user_group_table = Table('auth_user_groups', Base.metadata,
    Column('user_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_users.id', onupdate='CASCADE', ondelete='CASCADE')),
    Column('group_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_groups.id', onupdate='CASCADE', ondelete='CASCADE'))
)

class AuthGroup(Base):
    __tablename__ = 'auth_groups'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    name = Column(Unicode(80), unique=True, nullable=False)
    created = Column(mysql.DATE())

    users = relation('AuthUser', secondary=user_group_table, backref='auth_groups')

    def __repr__(self):
        return '' % self.name

    def __unicode__(self):
        return self.name

class AuthUser(Base):
    __tablename__ = 'auth_users'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    username = Column(Unicode(80), nullable=False)
    _password = Column('password', Unicode(80), nullable=False)
    email = Column(Unicode(80), nullable=False)
    contact = Column(Unicode(80), nullable=False)
    company = Column(Unicode(80), nullable=False)

    groups = relation('AuthGroup', secondary=user_group_table, backref='auth_users')

    @property
    def permissions(self):
        perms = set()
        for g in self.groups:
            perms = perms | set(g.permissions)
        return perms

    def _set_password(self, password):
        hashed_password = password

        if isinstance(password, unicode):
            password_8bit = password.encode('UTF-8')
        else:
            password_8bit = password

        salt = sha1()
        salt.update(os.urandom(60))
        hash = sha1()
        hash.update(password_8bit + salt.hexdigest())
        hashed_password = salt.hexdigest() + hash.hexdigest()

        if not isinstance(hashed_password, unicode):
            hashed_password = hashed_password.decode('UTF-8')
        self._password = hashed_password

    def _get_password(self):
        return self._password

    password = synonym('_password', descriptor=property(_get_password, _set_password))

    def validate_password(self, password):
        hashed_pass = sha1()
        hashed_pass.update(password + self.password[:40])
        return self.password[40:] == hashed_pass.hexdigest()

    def __repr__(self):
        return '' % (self.id, self.username, self.email)

    def __unicode__(self):
        return self.username

login.py:

from pyramid.httpexceptions import HTTPFound
from pyramid.security import remember
from pyramid.security import forget
from pyramid.security import Allow
from pyramid.security import Everyone
from pyramid.url import route_url
from pyramid.renderers import render_to_response

from project.auth import AuthUser
from project.models import DBSession

def login(request):
    dbsession = DBSession()
    login_url = route_url('login', request)
    referrer = request.url
    if referrer == login_url:
        referrer = '/' # never use the login form itself as came_from
    came_from = request.params.get('came_from', referrer)
    message = ''
    login = ''
    password = ''
    if 'form.submitted' in request.params:
        login = request.params['login']
        password = request.params['password']
        auth = dbsession.query(AuthUser).filter(AuthUser.username==login).first()
        if auth.validate_password(password):
            headers = remember(request, login)
            """
                  We use the Primary Key as our identifier once someone has authenticated rather than the
                  username.  You can change what is returned as the userid by altering what is passed to
                  remember.
            """
            #headers = remember(request, auth.id)
            return HTTPFound(location = came_from,
                             headers = headers)
        message = 'Failed login'

    return dict(
        message = message,
        url = request.application_url + '/login',
        came_from = came_from,
        login = login,
        password = password,
        )
    
def logout(request):
    headers = forget(request)
    return HTTPFound(location = route_url('root', request),
                     headers = headers)
    
def forbidden_view(request):
    login_url = route_url('login', request)
    referrer = request.url
    if referrer == login_url:
        referrer = '/' # never use the login form itself as came_from
    came_from = request.params.get('came_from', referrer)
    return render_to_response('templates/login.pt', dict(
               message = '',
               url = request.application_url + '/login',
               came_from = came_from,
               login = '',
               password = '',
           ), request=request)

def groupfinder(userid, request):
    dbsession = DBSession()
    auth = dbsession.query(AuthUser).filter(AuthUser.id==userid).first()
    if auth:
        return [('group:%s' % group.name) for group in auth.groups]

class RootFactory(object):
    __acl__ = [ (Allow, 'group:client', 'client'),
                (Allow, 'group:manager', 'manager'),
                (Allow, 'group:admin', 'admin') ]
    def __init__(self, request):
        self.__dict__.update(request.matchdict)

templates/login.pt:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html
    xmlns="http://www.w3.org/1999/xhtml"
    xmlns:tal="http://xml.zope.org/namespaces/tal">

<head>
  <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>
  <title>Authentication Test</title>
  <link rel="stylesheet" type="text/css"
        href="${request.application_url}/static/style.css" />
</head>

<body>

<h1>Log In</h1>

<div tal:replace="message"/>

<div class="main_content">
  <form action="${url}" method="post">
    <input type="hidden" name="came_from" value="${came_from}"/>
    <input type="text" name="login" value="${login}"/>
    <br/>
    <input type="password" name="password" value="${password}"/>
    <br/>
    <input type="submit" name="form.submitted" value="Log In"/>
  </form>
</div>  

</body>
</html>

Using Pyramid with Deform to Edit/Create records in SQLAlchemy backed database

Monday, November 8th, 2010

While working with Pyramid it was worth taking a good look at deform. During my brief look before, the one thing that appeared to be missing was a method to quickly integrate with SQLAlchemy backed data. For a handful of forms, manually creating the appstruct to be passed to the Form wouldn’t be difficult, but, for some more significant applications, it could be quite cumbersome. The following is a one page application that allows you to create a new user, or, edit an existing user by specifying the userid in the URL. The two pieces that do the real work are record_to_appstruct and merge_session_with_post. record_to_appstruct takes the class returned from SQLAlchemy and converts it to an appstruct that deform likes. Once we get the validated data, we merge the record with the post items using merge_session_with_post, merge the record and present a screen with the post data.

This is more a proof of concept, but, works well enough that we were able to convert quite a few forms and work more closely with Deform and Pyramid as we do more development.

__init.py__:


from pyramid.configuration import Configurator
from paste.deploy.converters import asbool

from defo.models import initialize_sql

def app(global_config, **settings):
    """ This function returns a WSGI application.
    
    It is usually called by the PasteDeploy framework during 
    ``paster serve``.
    """
    db_string = settings.get('db_string')
    if db_string is None:
        raise ValueError("No 'db_string' value in application configuration.")
    db_echo = settings.get('db_echo', 'false')
    initialize_sql(db_string, asbool(db_echo))
    config = Configurator(settings=settings)
    config.begin()
    config.add_static_view('static', 'defo:static')
    config.add_route('home', '/', view='defo.views.edit',
                     view_renderer='test.mako')
    config.add_route('homeid', '/:id', view='defo.views.edit',
                     view_renderer='test.mako')
    config.end()
    return config.make_wsgi_app()

views.py:

from defo.models import DBSession
from defo.models import MyModel

from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()

from sqlalchemy import *
from sqlalchemy.databases import mysql
from sqlalchemy.orm import relation, backref, synonym
from sqlalchemy.orm.exc import NoResultFound

import deform
import colander
from webhelpers import constants

class UserSchema(colander.Schema):
    username = colander.SchemaNode(colander.String())
    contact = colander.SchemaNode(colander.String())
    email = colander.SchemaNode(colander.String())
    company = colander.SchemaNode(colander.String())
    addr1 = colander.SchemaNode(colander.String())
    addr2 = colander.SchemaNode(colander.String(), missing=u'',)
    city = colander.SchemaNode(colander.String())
    state = colander.SchemaNode(colander.String())
    zip = colander.SchemaNode(colander.String())
    country = colander.SchemaNode(
        colander.String(),
        widget = deform.widget.SelectWidget(values=constants.country_codes()),
    )
    phone = colander.SchemaNode(colander.String())

class AuthUser(Base):
    __tablename__ = 'auth_users'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    username = Column(Unicode(80), nullable=False)
    email = Column(Unicode(80), nullable=False)
    contact = Column(Unicode(80), nullable=False)
    company = Column(Unicode(80), nullable=False)
    addr1 = Column(Unicode(80), nullable=False)
    addr2 = Column(Unicode(80))
    city = Column(Unicode(80), nullable=False)
    state = Column(Unicode(80), nullable=False)
    zip = Column(Unicode(80), nullable=False)
    country = Column(Unicode(80), nullable=False)
    phone = Column(Unicode(80), nullable=False)

def record_to_appstruct(self):
    return dict([(k, self.__dict__[k]) for k in sorted(self.__dict__) if '_sa_' != k[:4]])

def merge_session_with_post(session, post):
    for key,value in post:
        setattr(session, key, value)
    return session

def edit(request):
    dbsession = DBSession()
    if request.matchdict.has_key('id'):
        record = dbsession.query(AuthUser). \
                 filter_by(id=request.matchdict['id']).first()
    else:
        record = AuthUser()

    schema = UserSchema()
    form = deform.Form(schema, buttons=('submit',))
    if request.POST:
        try:
            appstruct = form.validate(request.POST.items())
        except deform.ValidationFailure, e:
            return {'form':e.render()}
        record = merge_session_with_post(record, request.POST.items())
        dbsession.merge(record)
        dbsession.flush()
        return {'formdata':appstruct}
    else:
        appstruct = record_to_appstruct(record)
    return {'form':form.render(appstruct=appstruct)}

test.mako:

<html>
<head>
  <title>
    Deform Demo Site
  </title>
  <!-- Meta Tags -->
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <!-- CSS -->
  <link rel="stylesheet" href="/static/css/form.css" type="text/css" />
  <link rel="stylesheet" href="/static/css/theme.css" type="text/css" />
  <!-- JavaScript -->
  <script type="text/javascript"
          src="/static/scripts/jquery-1.4.2.min.js"></script>
  <script type="text/javascript"
          src="/static/scripts/deform.js"></script>
</head>
<body>
this is a test template in mako
<p>
% if form:
  ${form|n}
% else:
We got our form data: ${formdata}
% endif
<script type="text/javascript">
   deform.load()
</script>
</body>
</html>

Pylons 1.0 to Pyramid 1.0a1

Sunday, November 7th, 2010

On Nov 4/Nov 5 a rumor that was later substantiated regarding Pylons surfaced. While the initial message was regarding TurboGears which is based on Pylons, the rumor was that Pylons was being rolled into another framework. Ben Bangert issued a post that gave some of the reasoning behind the merger of repoze.bfg and Pylons.

Since we’ve been working on an application for the last few weeks that isn’t in production, it seemed like an ideal test case. While the documentation for Pyramid is superb, transitioning from Pylons to Pyramid still has a few rough edges as the terminology that Pylons developers are used to were changed.

One of the first issues is the way that Pylons handles routes and converting to Pyramid. In Pylons, default route entries are set in config/routing.py:

    map.connect('/{controller}/{action}')
    map.connect('/{controller}/{action}/{id}')

To emulate that behavior in Pyramid, modify __init__.py:

    config.add_handler('client', '/client/:action', handler=Client)
    config.add_handler('clientid', '/client/:action/:id', handler=Client)

Pyramid doesn’t scan controllers, so, if you have multiple controllers, you’ll need to specify each. Also, make sure that you use a unique name (client|clientid) for each handler that you’ve added to avoid any 404s. Rather than the old controller structure you had, your code is now considerably cleaner and looks like:

from pyramid.response import Response
from pyramid.view import action

class Client(object):
    def __init__(self, request):
        self.request = request
        self.dbsession = DBSession()

    @action(renderer='client_index.jinja2')
    def index(self):
        return {'views':5, 'clicks':1}

The decorator signifies the template that you want to use and variables that you want to pass to the template are returned. This ends most of the tmpl_context. or c. clutter that was present in controllers and templates.

More documentation on handlers is available here.

An early version of the pyramid templates does not contain the weberror helper that was available in Pylons. The Pylons paster templates that are included in Pyramid do have the helper. A discussion with Chris McDonough should result in the changes being made to the Pyramid templates as well.

Another area that needs attention is Flash messages. Currently they are not supported and webhelpers.flash appears to implement things in a manner that won’t work with Pyramid. Through the subscribers method in Pyramid, it looks to be somewhat trivial to implement. The existing Pylons template in Pyramid does contain passthroughs of the c./tmpl_context. globals and helper modules that were available in Pylons. Modifying that slightly should allow Flash messages to be easily enabled. If you are going to transition, it would make more sense to use one of the Pylons templates than to migrate straight to the Pyramid templates.

For reference, the file:
pyramid-1.0a1-py2.6.egg/pyramid/paster_templates/pylons_sqla/+package+/subscribers.py_tmpl
mentions the existing global handling of the special objects for which Flash messages can probably be reimplemented.

SQLAlchemy is supported, but, there is an additional extension loaded which appears to do an autocommit on SQL queries. A brief readthrough mentions that you need to use s.join() to join your two database queries to be handled. It appears that you can join two db handles on separate databases which makes this a bit more powerful than using normal transactions as you could ensure a record was written to mongoDB and MySQL. I need to spend a little more time reading through this.

pagination appears to depend on routes, and even with routes installed, an error is thrown with thread._local requiring a mapper which is probably not going to work with Pyramid and will require some rewriting. This appears to be the same issue (thread._local) with Flash messages and it was mentioned that both items were relatively high priority and easy fixes.

forms – By default, formish is installed. While their site was down, it was stated that formish was in no way an endorsement, it was just included as it was part of bfg and is not a dependency in Pyramid*. Some preliminary work with FormAlchemy showed that it should work without too much difficulty, but, I decided to give Deform a try. Since I had already looked at Deform in the past for Pylons, and the screencast demonstration was done in Pylons, I was somewhat familiar with the methods. Converting over to Deform was a matter of reworking a few schemas. I’ve had some difficulties getting Deform to work with output from SQLAlchemy. Basically, Deform works very well with ZODB which is a schemaless database. Using it to edit rows returned from SQLAlchemy requires one to manually iterate through the returned row to create an appstruct to hand to Deform. For a number of simple forms this probably wouldn’t be difficult. As our project has a number of GridSets, converting over to Deform would have been considerably more difficult. To get FormAlchemy to work, we required the following changes:

form.py:

from mako.template import Template

from formalchemy import config as fa_config
from formalchemy import templates
from formalchemy import validators
from formalchemy import fields
from formalchemy import forms
from formalchemy import tables
from formalchemy.ext.fsblob import FileFieldRenderer
from formalchemy.ext.fsblob import ImageFieldRenderer

fa_config.encoding = 'utf-8'

class TemplateEngine(templates.TemplateEngine):
    def render(self, name, **kwargs):
        return Template(filename='/var/www/pyr/atg/atg/templates/forms/%s.mako' % name).render(**kwargs)

fa_config.engine = TemplateEngine()

class FieldSet(forms.FieldSet):
    pass

class Grid(tables.Grid):
    pass

our file using formalchemy:

from pyramid.httpexceptions import HTTPRedirection

from formalchemy import validators
from formalchemy.fields import Field
from atg.form import FieldSet

dbsession = DBSession()

User = FieldSet(auth.AuthUser, session=dbsession)
User.configure(
    include = [
        User.contact,
        User.email,
        User.company,
        User.addr1,
        User.addr2,
        User.city,
        User.state,
        User.zip,
        User.phone,
    ],
    options=[User.email.set(validate=validators.email)]
)

    @action(renderer='client_account.jinja2')
    def account(self):
        record = self.dbsession.query(auth.AuthUser).filter(auth.AuthUser.id==self.uid).first()
        fs = User.bind(record, data=self.request.POST or None)
        if self.request.POST and fs.validate():   
            fs.sync()
            self.dbsession.merge(record)
            self.dbsession.flush()
            HTTPRedirection(location='/client/account')
        return {'fs':fs}

model from auth.py:

class AuthUser(Base):
    __tablename__ = 'auth_users'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement
=True)
    username = Column(Unicode(80), nullable=False)
    _password = Column('password', Unicode(80), nullable=False)
    email = Column(Unicode(80), nullable=False)
    contact = Column(Unicode(80), nullable=False)
    company = Column(Unicode(80), nullable=False)
    addr1 = Column(Unicode(80), nullable=False)
    addr2 = Column(Unicode(80))
    city = Column(Unicode(80), nullable=False)
    state = Column(Unicode(80), nullable=False)
    zip = Column(Unicode(80), nullable=False)
    phone = Column(Unicode(80), nullable=False)

client_account.jinja2:

<form method="post">
{{ fs.render()|safe }}
<input type="submit" value="save">
</form>

There is a minor problem with validation with FormEncode that we’re still working with dealing with validation on a form that has been bound.

webhelpers – Most webhelpers appear to work fine. It was refreshing to see that many of the webhelpers.constants work fine without having to swap the order of the tuples with Deform/Formish. Currently, Flash and Paginate are broken as mentioned above, but, those will be fixed relatively quickly.

Authentication is built in. While the permissions system is quite well thought out, getting it to work in a basic fashion required quite a bit of tweaking. Basically:

__init__.py:

from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy

    authn_policy = AuthTktAuthenticationPolicy(
        'sosecret', callback=groupfinder)
    authz_policy = ACLAuthorizationPolicy()
    config = Configurator(settings=settings,
                          root_factory='atg.models.RootFactory',
                          authentication_policy=authn_policy,
                          authorization_policy=authz_policy)

    config.add_route('login', '/login',
                     view='atg.login.login',
                     view_renderer='atg:templates/login.pt')
    config.add_route('admin2', '/admin/', view='atg.admin.index', view_permission='edit', 
                     view_renderer='admin_index.jinja2')
    config.add_handler('admin', '/admin/:action', handler=Admin, permission='edit')

login.py:

from pyramid.httpexceptions import HTTPFound
from pyramid.security import remember
from pyramid.security import forget
from pyramid.url import route_url

from atg.security import USERS

def login(request):
    login_url = route_url('login', request)
    referrer = request.url
    if referrer == login_url:
        referrer = '/' # never use the login form itself as came_from
    came_from = request.params.get('came_from', referrer)
    message = ''
    login = ''
    password = ''
    if 'form.submitted' in request.params:
        login = request.params['login']
        password = request.params['password']
        if USERS.get(login) == password:
            headers = remember(request, login)
            return HTTPFound(location = came_from,
                             headers = headers)
        message = 'Failed login'

    return dict(
        message = message,
        url = request.application_url + '/login',
        came_from = came_from,
        login = login,
        password = password,
        )
    
def logout(request):
    headers = forget(request)
    return HTTPFound(location = route_url('view_wiki', request),
                     headers = headers)

security.py:

USERS = {'editor':'editor',
          'viewer':'viewer'}
GROUPS = {'editor':['group:editors']}

def groupfinder(userid, request):
    if userid in USERS:
        return GROUPS.get(userid, [])

templates/login.pt:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html
    xmlns="http://www.w3.org/1999/xhtml"
    xmlns:tal="http://xml.zope.org/namespaces/tal">

<head>
  <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>
  <title>bfg tutorial wiki (based on TurboGears 20-Minute Wiki)</title>
  <link rel="stylesheet" type="text/css"
        href="${request.application_url}/static/style.css" />
</head>

<body>

<h1>Log In</h1>

<div tal:replace="message"/>

<div class="main_content">
  <form action="${url}" method="post">
    <input type="hidden" name="came_from" value="${came_from}"/>
    <input type="text" name="login" value="${login}"/>
    <br/>
    <input type="password" name="password" value="${password}"/>
    <br/>
    <input type="submit" name="form.submitted" value="Log In"/>
  </form>
</div>  

</body>
</html>

I’ve not gotten the 401/403 error to prompt for a login, but, I believe that is something minor that I’m missing. In addition, I need to modify the schema to use an SQL backend.

All in all, transitioning the code took a bit longer than anticipated and I received some great advice and help from Ben Bangert and Chris McDonough on IRC. I tried to keep my questions to a minimum, but, transitioning from Pylons to Pyramid is going to be harder than moving from repoze.bfg to Pyramid.

Overall, I feel pretty good about the move. I had been a TurboGears users since 2.0-beta and moved over to Pylons for most development a few years back. After having spent 16-20 hours working with Pyramid, I believe that Pyramid is a step in the right direction. In addition to moving this app from Pylons to Pyramid, we switched from Mako to Jinja which required some rewrites of the templates. Mako is still used for FormAlchemy and is loaded in our Pyramid installation.

I don’t really see anything that would make me consider using another framework and I think it is a step in the right direction. Quick apachebench tests show some performance improvements which is also a nice benefit. While the software has an Alpha designation, from a stability standpoint, I’d say it performs more like a Release Candidate but the Alpha designation is probably maintained so that API changes can be pushed as necessary.

To the entire Pylons and Repoze.bfg teams, I say Congratulations! I see some great possibilities on the horizon with a larger community behind Pyramid.

* Chris McDonough helped clarify this (tweet)

Repoze.who/Repoze.what with Pylons (step by step)

Friday, October 29th, 2010

After working through quite a bit of the documentation on the pylons site and the repoze site, I didn’t really find a step by step guide to get repoze.who/repoze.what working with Pylons.

Some of the references used:

* http://code.gustavonarea.net/repoze.what-pylons/Manual/Protecting.html
* http://wiki.pylonshq.com/display/pylonscookbook/Authorization+with+repoze.what

Thanks to my nephew Qwait for overloading ActionProtector to intercept 403s and give an additional chance to authenticate.

wget http://pylonshq.com/download/1.0/go-pylons.py
python go-pylons.py --no-site-packages pylons
cd pylons
source bin/activate
easy_install repoze.what-quickstart
easy_install repoze.what-pylons
easy_install mysql-python
paster create -t pylons project
cd project

Because of the association table and the cascade, you must use MySQL or Postgresql rather than SQLite. You also might need to modify development.ini [server:main] host/port.

cd project

config/middleware.py:

    # CUSTOM MIDDLEWARE HERE (filtered by error handling middlewares)
    from project.lib.auth import add_auth
    app = add_auth(app, config)

config/routing.py:

    map.connect('/login', controller='login', action='login')
    map.connect('/login/submit', controller='login', action='login_handler')
    map.connect('/login/continue', controller='login', action='post_login')
    map.connect('/logout/continue', controller='login', action='post_logout')
    map.connect('/logout', controller='login', action='logout_handler')

lib/auth.py: – modified to intercept 403 and provide chance to authenticate

from pylons import response, url
from pylons.controllers.util import redirect

from repoze.what.plugins.quickstart import setup_sql_auth
from repoze.what.plugins import pylonshq

import project.lib.helpers as h

from project.model.meta import Session
from project.model.auth import AuthUser, AuthGroup, AuthPermission

def add_auth(app, config):
   return setup_sql_auth(
       app, AuthUser, AuthGroup, AuthPermission, Session,
       login_handler = '/login/submit',
       logout_handler = '/logout',
       post_login_url = '/login/continue',
       post_logout_url = '/logout/continue',
       cookie_secret = 'my_secret_word',
       translations = {
           'user_name' : 'username',
           'groups' : 'auth_groups',
           'group_name' : 'name',
           'permissions' : 'auth_permissions',
           'permission_name' : 'name'
       }
   )

def redirect_auth_denial(reason):
    if response.status_int == 401:
        message = 'You are not logged in.'
        message_type = 'warning'
    else:
        message = 'You do not have the permissions to access this page.'
        message_type = 'error'

    h.flash(message, message_type)
    redirect(url('/login', came_from=url.current()))

class ActionProtector(pylonshq.ActionProtector):
    default_denial_handler = staticmethod(redirect_auth_denial)

model/auth.py:

from sqlalchemy import *
from sqlalchemy.databases import mysql
from sqlalchemy.orm import relation, backref, synonym
from sqlalchemy.orm.exc import NoResultFound

from project.model.meta import Base

import os
from hashlib import sha1
from datetime import datetime

group_permission_table = Table('auth_group_permissions', Base.metadata,
    Column('group_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_groups.id', onupdate='CASCADE', ondelete='CASCADE')),
    Column('permission_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_permissions.id', onupdate='CASCADE', ondelete='CASCADE'))
)
user_group_table = Table('auth_user_groups', Base.metadata,
    Column('user_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_users.id', onupdate='CASCADE', ondelete='CASCADE')),
    Column('group_id', mysql.BIGINT(20, unsigned=True), ForeignKey('auth_groups.id', onupdate='CASCADE', ondelete='CASCADE'))
)

class AuthGroup(Base):
    __tablename__ = 'auth_groups'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    name = Column(Unicode(80), unique=True, nullable=False)
    created = Column(mysql.DATE())

    users = relation('AuthUser', secondary=user_group_table, backref='auth_groups')

    def __repr__(self):
        return '<group: name=%s>' % self.name

    def __unicode__(self):
        return self.name

class AuthUser(Base):
    __tablename__ = 'auth_users'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    username = Column(Unicode(80), nullable=False)
    _password = Column('password', Unicode(80), nullable=False)
 
    @property
    def permissions(self):
        perms = set()
        for g in self.groups:
            perms = perms | set(g.permissions)
        return perms

    def _set_password(self, password):
        hashed_password = password

        if isinstance(password, unicode):
            password_8bit = password.encode('UTF-8')
        else:
            password_8bit = password

        salt = sha1()
        salt.update(os.urandom(60))
        hash = sha1()
        hash.update(password_8bit + salt.hexdigest())
        hashed_password = salt.hexdigest() + hash.hexdigest()

        if not isinstance(hashed_password, unicode):
            hashed_password = hashed_password.decode('UTF-8')
        self._password = hashed_password

    def _get_password(self):
        return self._password

    password = synonym('_password', descriptor=property(_get_password, _set_password))

    def validate_password(self, password):
        hashed_pass = sha1()
        hashed_pass.update(password + self.password[:40])
        return self.password[40:] == hashed_pass.hexdigest()

    def __repr__(self):
        return '<user: id="%s" username="%s" email="%s">' % (self.id, self.username, self.email)

    def __unicode__(self):
        return self.username

class AuthPermission(Base):
    __tablename__ = 'auth_permissions'

    id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement=True)
    name = Column(Unicode(80), unique=True, nullable=False)
    description = Column(mysql.TEXT())

    groups = relation(AuthGroup, secondary=group_permission_table, backref='auth_permissions')

    def __unicode__(self):
        return self.permission_name

controllers/login.py:

from pylons import request, response, session, tmpl_context, config, url
from pylons.controllers.util import redirect

from project.lib.base import BaseController, render
from project.lib.helpers import flash

class LoginController(BaseController):
    def login(self):
        login_counter = request.environ['repoze.who.logins']
        if login_counter > 0:
            flash('Wrong credentials')
        tmpl_context.login_counter = login_counter
        tmpl_context.came_from = request.params.get('came_from') or url('/')
        return render('login.mako')

    def login_handler(self):
        pass

    def post_login(self):
        identity = request.environ.get('repoze.who.identity')
        came_from = str(request.params.get('came_from', '')) or url('/')
        if not identity:
            login_counter = request.environ['repoze.who.logins'] + 1
            redirect(url('/login', came_from=came_from, __logins=login_counter))
        redirect(came_from)

    def logout_handler(self):
        pass
     
    def post_logout(self):
        redirect('/')

templates/login.mako:

<% messages = h.flash.pop_messages() %>
% if messages:
<div class="flash">
    % for message in messages:
      <p class="${message.category}">${message}
    % endfor
</div>
% endif

  <form action="${h.url('/login/submit', came_from=tmpl_context.came_from, __logins=tmpl_context.login_counter)}" method="POST">
    <label for="login">Username:<input type="text" id="login" name="login" /><br />
    <label for="password">Password:<input type="password" id="password" name="password" />
    <input type="submit" value="Login" />
  </form>

controllers/root.py:

from pylons import request, response, session, tmpl_context, config

#from repoze.what.plugins.pylonshq import ActionProtector, ControllerProtector
from project.lib.auth import ActionProtector
from repoze.what.predicates import is_user, has_permission, in_group

from project.lib.base import BaseController, render

class RootController(BaseController):
    def index(self):
        return render('index.mako')

    @ActionProtector(is_user('test'))
    def user(self):
        return render('loggedin.mako')

    @ActionProtector(is_user('nottest'))
    def notuser(self):
        return render('loggedin.mako')

    @ActionProtector(in_group('admin'))
    def admin(self):
        return render('loggedin.mako')

    @ActionProtector(has_permission('edit'))
    def edit(self):
        return render('loggedin.mako')

lib/helpers.py:

from pylons import url

from webhelpers.pylonslib import Flash as _Flash
flash = _Flash()

websetup.py: – after the Session, Base import:

from project.model.auth import *

Setup/Create the database, start paster

paster setup-app development.ini
paster serve --reload development.ini

createuser.py:

#!/usr/bin/python2.6

from sqlalchemy import create_engine
engine = create_engine('mysql://user:pass@localhost/dbname', echo=True)
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session()

from project.model.auth import *

u = AuthUser()
u.username = u'test'
u.password = u'test'
session.add(u)
g = AuthGroup()
g.name = u'admin'
g.users.append(u)
session.add(g)
p = AuthPermission()
p.name = u'edit'
p.groups.append(g)
session.add(p)
session.commit()

Facebook’s Javascript SDK and a short one page application

Thursday, September 16th, 2010

While discussing a project with a client it occurred to me that perhaps you don’t need to get too complex to do something simple. Since using Facebook almost mandates that your surfer has Javascript enabled, we should be able to write a very simple application that posts to someone’s wall after asking for the ‘publish’ permission. After reading the documentation and looking through a few github repositories, the solution was quite simple.

While this code doesn’t use the non-blocking async javascript method, it is a good example. I’ve seen others that use the inline publish, but, for some reason I couldn’t get it to consistently open a dialog box rather than a popup – which Chrome conveniently blocked. I also ran into an issue that Facebook appears to aggressively cache objects that don’t explicitly set an expire time. While I find that acceptable for the application’s long term goals, it did make debugging a slightly frustrating experience.

To try the application, http://apps.facebook.com/onepageapp/. A direct link to the code is available at http://fbapp.cd34.com/opa/index.txt.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml">
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
        <title>One Page App</title>
    </head>
    <body>
<div id="fb-root"></div>
Thanks for clicking.  This is just a test to make sure that the application
works as expected.
<p>
You can skip posting to the wall if you would like.
<script src="http://connect.facebook.net/en_US/all.js"></script>
<script>
appid = '154516391233318';
name = 'One Page App';
caption = 'A single page application designed to test whether it could be done';
description = 'A quick page that allows you to post to someone\'s wall';
href = 'http://apps.facebook.com/onepageapp/';
user_message_prompt = 'Post a sample to your wall';
message = 'Let\'s do a sample post to the wall.';
action_text = 'Get the code';
action_href = 'http://fbapp.cd34.com/opa/index.txt';

FB.init({appId  : appid, status : true, cookie : true, xfbml  : false });
FB.getLoginStatus(function(response) {
  if (response.session) {
    FB.ui(
      {
        method: 'stream.publish',
        display: 'dialog',
        message: message,
        attachment: {
          name: name,
          caption: caption,
          description: description,
          href: href
        },
        action_links: [
          { text: action_text, href: action_href }
        ],
        user_message_prompt: user_message_prompt
      },
      function(response) {
        self.location.href='/opa/thanks.html';
      }
    );
  } else {
    top.location.href='https://graph.facebook.com/oauth/authorize?client_id='+appid+'&redirect_uri='+href+'&display=page&scope=publish_stream';
  }
});
</script>
</body>
</html>