By Ingeniweb. A Django site.
Juin 24, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» Five 1.5.6 testbrowser and recent version of mechanize

Doing development with Plone 3.1, I encountered a KeyError when I tried to use Five.testbrowser.Browser:

Browser()
KeyError: '_seek'
I eventually found that my system (currently Ubuntu 8.04) had the package "python-mechanize" 0.1.7b, which masks the one provided by Zope 2.10 (0.1.2b). Unfortunatly there has been API changes between 0.1.2b and 0.1.7b... (see classes UserAgent/UserAgentBase)

The solution is to tell buildout to get mechanize egg 0.1.2b: this way it will mask the system library.
[buildout]
versions = versions
eggs=
...
mechanize

[versions]
mechanize = 0.1.2b
Et voila!

Juin 9, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» Buildout, ploneldap and ldap products (2)

In my previous post I described how to get a buildout part to get PloneLDAP with an up-to-date LDAPUserFolder. Things are even simpler now since all involved products have been released as eggs. So just list theses in your buildout along with other eggs:

eggs =
...
Products.LDAPUserFolder
Products.LDAPMultiPlugins
Products.PloneLDAP

Second important thing: LDAPUserFolder 2.9 has been released. If you used 2.9-beta you must upgrade, since an important bug related to the negative cache has been fixed. PloneLDAP documents usage with LDAPUserFolder 2.8, but 2.9 is definitely worth it: The negative cache feature avoids doing too many ldap request, especially useless/unsuccessful ones! Many thanks to Jens Vagelpohl (http://www.dataflake.org/software/ldapuserfolder).

» Quick workflow migration from plone 2.x to plone 3

When migrating a site from plone 2.x to plone 3, I often have to port customized workflows. Back in the plone 2.0 days we hadn't generic setup, and workflows were created with python code (with the help of DCWorkflowDump), or directly created in the ZMI and installed on final site by importing a zexp.

Nowadays we want to use generic setup (GS).

The simplest way I have found is to export the workflow as a zexp, import this zexp into the new site, and then make a GS export. Done! you've got a nice XML definition of your workflow, ready to be included in your product GS profile.

Once I encountered one caveat: if the original workflow contains strings that are not in plain ascii (in titles, etc...), the GS export will fail. You'll have to relabel properly everywhere, and add all those labels (msgids) to your translation files (I think i18ndude does it for you).

Juin 6, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» iw.eggproxy: a proxy for pypi

Today we have released iw.eggproxy. It's a module for apache mod_python. Its purpose is to serve as a pypi proxy.

The first motivation to make it was because we have had to work in a private network with a very very slow internet access: for example updating a plone buildout could take more than one hour (just checking eggs freshness) when it should be no more than a few minutes. This condition also prevent to run some kind of rsync against pypi. So the obvious solution was to proxy the eggs we need, on-demand.

The module is installed as a handler on a Location. When accessing this location, eggproxy will serve an index similar to pypi simple view.

This is done like this:

  • we already have the information in index.html: just let apache serve the file
  • or, we use setuptools to fetch index information, build index.html, and let apache serve the file
This allows eggproxy to serve all available eggs from pypi, without actually having to download the whole pypi + content.

Then easy_install can see a package is available on our server, and tries to fetch information on available eggs:
  • the subdirectory and its index.html already exists: just let apache serve the file
  • or, we use setuptools again to get package information, make the subdirectory (package name) and build index.html

Finally, when trying to fetch an egg we do the same:
  • the file is already present and is served by apache
  • or we use setuptools to get the file on the server
iw.eggproxy also provides an update script: "eggproxy_update". This script refreshes the main index and all proxied eggs, if their index.html is older than the interval specified in the configuration file (24h by default).

We have installed it here: http://release.ingeniweb.com/pypi.python.org-mirror

Known bugs: some packages on pypi don't have eggs, then eggproxy does not respond. This is the case with "reportlab" for example.

Enhancements:
  • indexes aggregation. At ingeniweb we plan to install it on local server and agregate pypi and some private eggs indexes.
  • Standalone/pluggable server. Currently we are bound to apache + mod_python, which may not suit to anyone.

Avril 22, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» Buildout, ploneldap and ldap products

UPDATE: all products are available as eggs. See this post.

PloneLDAP is a great product, but its bundle ships a not so recent version of LDAPUserFolder. LDAPUserFolder and LDAPMultiPlugins download urls are both ending with "/download". That seems to confuse plone.recipe.distros, because it installs just one of them. Furthermore, at plone.org PloneLDAP single product (as opposed to PloneLDAP bundle) download url ends with... a space! (%20).
So we have uploaded theses tarballs on release.ingeniweb.com. Here is a working part for buildout:


[ldap_products]
recipe = plone.recipe.distros
urls =
http://release.ingeniweb.com/third-party-dist/LDAPUserFolder-2.9-beta.tgz
http://release.ingeniweb.com/third-party-dist/LDAPMultiPlugins-1.5.tgz
http://release.ingeniweb.com/third-party-dist/PloneLDAP-1.0.tar.gz

Février 11, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» iw.rejectanonymous: private site with plone 3.0

We have made a small package to provide the functionality described in my previous post. It is named "iw.rejectanonymous".

Quick recipe to use it from an integration product (i.e a product responsible of setting up a plone site for your particular environment/customer/...):

  • Add in configure.zcml:
    <include package="iw.rejectanonymous" />

  • Add python code to activate it for your site. This is probably done in a function called by generic setup, this is often located in setuphandlers.py:
    from zope.interface import alsoProvides
    from iw.rejectanonymous import IPrivateSite

    def setupPortal(portal):
    if not IPrivateSite.providedBy(portal):
    alsoProvides(portal, IPrivateSite)
The second step can be done through the ZMI with the "Interfaces" tab.

Janvier 31, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» Making a private site with plone 3.0

UPDATE: we have made a product for this: iw.rejectanonymous

There is a document at plone.org suggesting to use plone 3.0 builtin "intranet" workflows, however this will not make a site absolutely private, i.e. force user to login before he can view anything. This is the use case for an extranet for example.

In the past we used to put something like tal:define="dummy here/rejectAnonymous" in global_defines.pt, rejectAnonymous was a skin script. Now with the help of events we can do far better, and it will work for any content/object within a plone site. As a consequence we must be careful about what is allowed to be retrievied anonymously, since anonymous should be able to see a themed login page.

The idea has been taken from plone.aftertraverse. An event is sent before traversal, but not immediatly after. The problem is that authentication is performed after traversal. Fortunately the request object accepts to register post traverse hooks, with arbitrary parameters.

The code, zcml part:

<subscriber handler=".hooks.insertRejectAnonymousHook"
for="Products.CMFCore.interfaces.ISiteRoot
zope.app.publication.interfaces.IBeforeTraverseEvent"
/>


and hooks.py:

# -*- coding: utf-8 -*-
from zExceptions import Unauthorized

valid_subparts = set(('login.js', 'spinner.gif',
'portal_css', 'portal_javascripts'))

def rejectAnonymous(portal, request):
mtool = portal.portal_membership
if mtool.isAnonymousUser():
url = request.physicalPathFromURL(request['URL'])
if url and not (url[-1] in ('login_form', 'require_login')
or [path for path in url
if path in valid_subparts]):
raise Unauthorized, "You must be authenticated"


def insertRejectAnonymousHook(portal, event):
"""
"""
event.request.post_traverse(rejectAnonymous, (portal, event.request))


The code checking for allowed path may not be the best, and it could certainly be more clever but for-me-it-worked(tm)

Janvier 9, 2008
Bertrand Mathieu
bertrand
Paste here is about »
» VersionConflict in buildout

Today I tried to update my buildout from plone 3.0.4 to 3.0.5. In the relevant section I just changed:

- recipe = plone.recipe.plone==3.0.4
+ recipe = plone.recipe.plone==3.0.5

and re-run buildout, but I got a "VersionConflict" error:

[...]
Uninstalling plone.
While:
Installing.
Uninstalling plone.
Loading recipe 'plone.recipe.plone==3.0.4'.

An internal error occured due to a bug in either zc.buildout or in a
recipe being used:

VersionConflict:
(plone.recipe.plone 3.0.5 (/home/bmathieu/.buildout/eggs/plone.recipe.plone-3.0.5-py2.4.egg), Requirement.parse('plone.recipe.plone==3.0.4'))

The only way I found to get rid of this is to edit the hidden file named ".installed.cfg", and replace the line:
- recipe = plone.recipe.plone==3.0.4
+ recipe = plone.recipe.plone

Then buildout could finish its job. I don't know if this is clean, but it may help.

Décembre 7, 2007
Bertrand Mathieu
bertrand
Paste here is about »
» Maintenance mode for Apache

This has been tested with Plone behind Apache but should apply to any site with Apache as a front server.
In maintenance mode we want Apache to serve a single static page with its associated ressources, and have any other requests redirected to this page. The static files are:

  • maintenance.html : the main file
  • ressources/ : directory containing CSS, javascript, images...
  • robots.txt (instruct to not index maintenance.html)

To create a page with the same design of the real site the firefox extension Save Complete is of great help.

There is 2 strategies:
  • Doing it dynamically by testing a parameter; in our case it will be the presence of a file
  • With an alternative configuration file, swapped with the normal one at maintenance time
In this example we are running a plone site on the same host, port 8080.

1. Dynamically:

DocumentRoot /static/files

Options FollowSymLinks
AllowOverride None
Order deny,allow
Allow from all
Satisfy all

DirectoryIndex maintenance.html

RewriteEngine on

The next rule tests the presence of the regular file "/some/path/maintenance.txt"; if it exists we set the environment variable "MAINTENANCE" to 1.
RewriteCond /some/path/maintenance.txt -f
RewriteRule ^(.*)$ - [env=MAINTENANCE:1]
In maintenance mode we want the browser to never perform any cache: as soon as we'll return to production mode, the normal site should appear. Important: the headers module must be enabled.

Header set cache-control "max-age=0,must-revalidate,post-check=0,pre-check=0" env=MAINTENANCE
Header set Expires -1 env=MAINTENANCE

Next rule instructs to let pass any request matching maintenance files (maintenance.html + CSS/JS/images ressources), else redirect to the maintenance page.

RewriteCond %{ENV:MAINTENANCE} 1
RewriteCond %{REQUEST_URI} ^/ressources [OR]
RewriteCond %{REQUEST_URI} ^/maintenance.html [OR]
RewriteCond %(REQUEST_URI) ^/robots.txt [OR]
RewriteRule .* - [L]

RewriteCond %{ENV:MAINTENANCE} 1
RewriteCond %{REQUEST_URI} !^/maintenance.html
RewriteRule ^.* /maintenance.html [L,R]

The next and last rewrite rule is for normal mode.

RewriteCond %{ENV:MAINTENANCE} !1
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) http://localhost:8080/VirtualHostBase/http/%{HTTP_HOST}:80/plone/site/VirtualHostRoot/$1 [P]

Now, to put the site in maintenance mode, just do "touch /some/path/maintenance.txt"; delete maintenance.txt to go back in normal mode.
2. Using an alternative configuration file:

This case is much simpler. First create a static configuration:

DocumentRoot /static/files
RewriteEngine on

RewriteCond %{REQUEST_URI} ^/ressources [OR]
RewriteCond %{REQUEST_URI} ^/maintenance.html [OR]
RewriteCond %(REQUEST_URI) ^/robots.txt [OR]
RewriteRule .* - [L]

RewriteCond %{REQUEST_URI} !^/maintenance.html
RewriteRule ^.* /maintenance.html [L,R]

Header set cache-control "max-age=0,must-revalidate,post-check=0,pre-check=0"
Header set Expires -1


On Debian this file must be placed in /etc/apache2/sites-available. If the site configuration file is named "plone", you can name its maintenance counterpart "plone-maint" for example. To switch to maintenance mode:

> sudo a2dissite plone
> sudo a2ensite plone-maint
> sudo /etc/init.d/apache reload

To go back to normal mode just switch "plone" and "plone-maint".

Enhancement not covered here:

While the site is supposed to show a maintenance page, it may be desirable to let the maintainer access the site through apache: this should be done with a rewrite condition/rule placed first (condition based on Ip address for example).