Pylons 1.0 to Pyramid 1.0a1
On Nov 4/Nov 5 a rumor that was later substantiated regarding Pylons surfaced. While the initial message was regarding TurboGears which is based on Pylons, the rumor was that Pylons was being rolled into another framework. Ben Bangert issued a post that gave some of the reasoning behind the merger of repoze.bfg and Pylons.
Since we’ve been working on an application for the last few weeks that isn’t in production, it seemed like an ideal test case. While the documentation for Pyramid is superb, transitioning from Pylons to Pyramid still has a few rough edges as the terminology that Pylons developers are used to were changed.
One of the first issues is the way that Pylons handles routes and converting to Pyramid. In Pylons, default route entries are set in config/routing.py:
map.connect('/{controller}/{action}')
map.connect('/{controller}/{action}/{id}')
To emulate that behavior in Pyramid, modify __init__.py:
config.add_handler('client', '/client/:action', handler=Client)
config.add_handler('clientid', '/client/:action/:id', handler=Client)
Pyramid doesn’t scan controllers, so, if you have multiple controllers, you’ll need to specify each. Also, make sure that you use a unique name (client|clientid) for each handler that you’ve added to avoid any 404s. Rather than the old controller structure you had, your code is now considerably cleaner and looks like:
from pyramid.response import Response
from pyramid.view import action
class Client(object):
def __init__(self, request):
self.request = request
self.dbsession = DBSession()
@action(renderer='client_index.jinja2')
def index(self):
return {'views':5, 'clicks':1}
The decorator signifies the template that you want to use and variables that you want to pass to the template are returned. This ends most of the tmpl_context. or c. clutter that was present in controllers and templates.
More documentation on handlers is available here.
An early version of the pyramid templates does not contain the weberror helper that was available in Pylons. The Pylons paster templates that are included in Pyramid do have the helper. A discussion with Chris McDonough should result in the changes being made to the Pyramid templates as well.
Another area that needs attention is Flash messages. Currently they are not supported and webhelpers.flash appears to implement things in a manner that won’t work with Pyramid. Through the subscribers method in Pyramid, it looks to be somewhat trivial to implement. The existing Pylons template in Pyramid does contain passthroughs of the c./tmpl_context. globals and helper modules that were available in Pylons. Modifying that slightly should allow Flash messages to be easily enabled. If you are going to transition, it would make more sense to use one of the Pylons templates than to migrate straight to the Pyramid templates.
For reference, the file:
pyramid-1.0a1-py2.6.egg/pyramid/paster_templates/pylons_sqla/+package+/subscribers.py_tmpl
mentions the existing global handling of the special objects for which Flash messages can probably be reimplemented.
SQLAlchemy is supported, but, there is an additional extension loaded which appears to do an autocommit on SQL queries. A brief readthrough mentions that you need to use s.join() to join your two database queries to be handled. It appears that you can join two db handles on separate databases which makes this a bit more powerful than using normal transactions as you could ensure a record was written to mongoDB and MySQL. I need to spend a little more time reading through this.
pagination appears to depend on routes, and even with routes installed, an error is thrown with thread._local requiring a mapper which is probably not going to work with Pyramid and will require some rewriting. This appears to be the same issue (thread._local) with Flash messages and it was mentioned that both items were relatively high priority and easy fixes.
forms – By default, formish is installed. While their site was down, it was stated that formish was in no way an endorsement, it was just included as it was part of bfg and is not a dependency in Pyramid*. Some preliminary work with FormAlchemy showed that it should work without too much difficulty, but, I decided to give Deform a try. Since I had already looked at Deform in the past for Pylons, and the screencast demonstration was done in Pylons, I was somewhat familiar with the methods. Converting over to Deform was a matter of reworking a few schemas. I’ve had some difficulties getting Deform to work with output from SQLAlchemy. Basically, Deform works very well with ZODB which is a schemaless database. Using it to edit rows returned from SQLAlchemy requires one to manually iterate through the returned row to create an appstruct to hand to Deform. For a number of simple forms this probably wouldn’t be difficult. As our project has a number of GridSets, converting over to Deform would have been considerably more difficult. To get FormAlchemy to work, we required the following changes:
form.py:
from mako.template import Template
from formalchemy import config as fa_config
from formalchemy import templates
from formalchemy import validators
from formalchemy import fields
from formalchemy import forms
from formalchemy import tables
from formalchemy.ext.fsblob import FileFieldRenderer
from formalchemy.ext.fsblob import ImageFieldRenderer
fa_config.encoding = 'utf-8'
class TemplateEngine(templates.TemplateEngine):
def render(self, name, **kwargs):
return Template(filename='/var/www/pyr/atg/atg/templates/forms/%s.mako' % name).render(**kwargs)
fa_config.engine = TemplateEngine()
class FieldSet(forms.FieldSet):
pass
class Grid(tables.Grid):
pass
our file using formalchemy:
from pyramid.httpexceptions import HTTPRedirection
from formalchemy import validators
from formalchemy.fields import Field
from atg.form import FieldSet
dbsession = DBSession()
User = FieldSet(auth.AuthUser, session=dbsession)
User.configure(
include = [
User.contact,
User.email,
User.company,
User.addr1,
User.addr2,
User.city,
User.state,
User.zip,
User.phone,
],
options=[User.email.set(validate=validators.email)]
)
@action(renderer='client_account.jinja2')
def account(self):
record = self.dbsession.query(auth.AuthUser).filter(auth.AuthUser.id==self.uid).first()
fs = User.bind(record, data=self.request.POST or None)
if self.request.POST and fs.validate():
fs.sync()
self.dbsession.merge(record)
self.dbsession.flush()
HTTPRedirection(location='/client/account')
return {'fs':fs}
model from auth.py:
class AuthUser(Base):
__tablename__ = 'auth_users'
id = Column(mysql.BIGINT(20, unsigned=True), primary_key=True, autoincrement
=True)
username = Column(Unicode(80), nullable=False)
_password = Column('password', Unicode(80), nullable=False)
email = Column(Unicode(80), nullable=False)
contact = Column(Unicode(80), nullable=False)
company = Column(Unicode(80), nullable=False)
addr1 = Column(Unicode(80), nullable=False)
addr2 = Column(Unicode(80))
city = Column(Unicode(80), nullable=False)
state = Column(Unicode(80), nullable=False)
zip = Column(Unicode(80), nullable=False)
phone = Column(Unicode(80), nullable=False)
client_account.jinja2:
<form method="post">
{{ fs.render()|safe }}
<input type="submit" value="save">
</form>
There is a minor problem with validation with FormEncode that we’re still working with dealing with validation on a form that has been bound.
webhelpers – Most webhelpers appear to work fine. It was refreshing to see that many of the webhelpers.constants work fine without having to swap the order of the tuples with Deform/Formish. Currently, Flash and Paginate are broken as mentioned above, but, those will be fixed relatively quickly.
Authentication is built in. While the permissions system is quite well thought out, getting it to work in a basic fashion required quite a bit of tweaking. Basically:
__init__.py:
from pyramid.authentication import AuthTktAuthenticationPolicy
from pyramid.authorization import ACLAuthorizationPolicy
authn_policy = AuthTktAuthenticationPolicy(
'sosecret', callback=groupfinder)
authz_policy = ACLAuthorizationPolicy()
config = Configurator(settings=settings,
root_factory='atg.models.RootFactory',
authentication_policy=authn_policy,
authorization_policy=authz_policy)
config.add_route('login', '/login',
view='atg.login.login',
view_renderer='atg:templates/login.pt')
config.add_route('admin2', '/admin/', view='atg.admin.index', view_permission='edit',
view_renderer='admin_index.jinja2')
config.add_handler('admin', '/admin/:action', handler=Admin, permission='edit')
login.py:
from pyramid.httpexceptions import HTTPFound
from pyramid.security import remember
from pyramid.security import forget
from pyramid.url import route_url
from atg.security import USERS
def login(request):
login_url = route_url('login', request)
referrer = request.url
if referrer == login_url:
referrer = '/' # never use the login form itself as came_from
came_from = request.params.get('came_from', referrer)
message = ''
login = ''
password = ''
if 'form.submitted' in request.params:
login = request.params['login']
password = request.params['password']
if USERS.get(login) == password:
headers = remember(request, login)
return HTTPFound(location = came_from,
headers = headers)
message = 'Failed login'
return dict(
message = message,
url = request.application_url + '/login',
came_from = came_from,
login = login,
password = password,
)
def logout(request):
headers = forget(request)
return HTTPFound(location = route_url('view_wiki', request),
headers = headers)
security.py:
USERS = {'editor':'editor',
'viewer':'viewer'}
GROUPS = {'editor':['group:editors']}
def groupfinder(userid, request):
if userid in USERS:
return GROUPS.get(userid, [])
templates/login.pt:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html
xmlns="http://www.w3.org/1999/xhtml"
xmlns:tal="http://xml.zope.org/namespaces/tal">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type"/>
<title>bfg tutorial wiki (based on TurboGears 20-Minute Wiki)</title>
<link rel="stylesheet" type="text/css"
href="${request.application_url}/static/style.css" />
</head>
<body>
<h1>Log In</h1>
<div tal:replace="message"/>
<div class="main_content">
<form action="${url}" method="post">
<input type="hidden" name="came_from" value="${came_from}"/>
<input type="text" name="login" value="${login}"/>
<br/>
<input type="password" name="password" value="${password}"/>
<br/>
<input type="submit" name="form.submitted" value="Log In"/>
</form>
</div>
</body>
</html>
I’ve not gotten the 401/403 error to prompt for a login, but, I believe that is something minor that I’m missing. In addition, I need to modify the schema to use an SQL backend.
All in all, transitioning the code took a bit longer than anticipated and I received some great advice and help from Ben Bangert and Chris McDonough on IRC. I tried to keep my questions to a minimum, but, transitioning from Pylons to Pyramid is going to be harder than moving from repoze.bfg to Pyramid.
Overall, I feel pretty good about the move. I had been a TurboGears users since 2.0-beta and moved over to Pylons for most development a few years back. After having spent 16-20 hours working with Pyramid, I believe that Pyramid is a step in the right direction. In addition to moving this app from Pylons to Pyramid, we switched from Mako to Jinja which required some rewrites of the templates. Mako is still used for FormAlchemy and is loaded in our Pyramid installation.
I don’t really see anything that would make me consider using another framework and I think it is a step in the right direction. Quick apachebench tests show some performance improvements which is also a nice benefit. While the software has an Alpha designation, from a stability standpoint, I’d say it performs more like a Release Candidate but the Alpha designation is probably maintained so that API changes can be pushed as necessary.
To the entire Pylons and Repoze.bfg teams, I say Congratulations! I see some great possibilities on the horizon with a larger community behind Pyramid.
* Chris McDonough helped clarify this (tweet)