By Ingeniweb. A Django site.
Septembre 23, 2008
» The WSGI era is here


This is probably obvious for the people that uses Repoze or Pylons, or early adopters in the Plone world, but from a Plone or a Zope developer perspective, you could live without it until now.

Now WSGI is everywhere.

I remember when Martijn Faassen brought the idea in 2006, of hooking Grok and Zope 3 into the WSGI. Maybe someone else talked about it before but that was the first time I could picture what WSGI could bring.

Now with the work done by people like:

  • The repoze team, that made it easier to run a Plone-based application in WSGI
  • The Paste Script / Paste Deploy team, that provided a simple way to describe a WSGI chain

And major WSGI middlewares like :

  • repoze.who which allows you to deal with authentication separately
  • Deliverance, which let you theme any application and let this application focus on delivering a content
  • Things like Beaker, which let you use memcached for instance, to store session data and cache arbitrary things

From a CTO point of view, a WSGI environment brings me the ability to think about a web application and build it without having to stick into one framework and try to bend all technologies inside it.

For instance:

  • I can write a Plone application and use Beaker to deal with sessions, without having to wrap Memcached into a custom plone package.
  • I can ask a graphic designer to work on a CSS and a layout without having to do it into Plone. It’s not that Plone design tools are bad, but the learning curve of writing a rule file in Deliverance and apply it to any piece of application makes the designer more productive than becoming a specialist of one skinning tool.
  • If my customer use moinmoin as a Wiki, I can put it into my Plone site transparently by defining a composite section in my Paste configuraton file.

You could do all the mentioned thing without WSGI, just by importing the packages and/or dealing with proxies at Apache level. But that is not the point.

The point is that WSGI brought the idea of making all web frameworks and libraries interact together to build one web application.

It is not the silver bullet of course, but my gut feeling is that this will create some kind of reunification in Python Web development communities: people are starting to look at a wider range of package, beyond the framework they use everyday.

      

Septembre 16, 2008
» Paris Bobun Sprint


I don’t have time to talk in detail about it at this time, but I wanted to say that the Bobun Paris Sprint was a great event.

Together with the sprint we have done a few month before here at AlterWay, this a great momentum for the French Plone community.

Those events will probably be recurrent from now on (the one we organized at AlterWay will be organize every year for sure, and I think the Bobun team wants to do it again as well).

Maybe we will be able at some point in a few years to organize the PloneConf in Paris ?

That would be a blast :D

Septembre 11, 2008
» zc.buildout recipe to build your Sphinx doc


Sphinx is now used by quite a few communities : Python, Django, Pylons, Grok (not sure about the current status), …

No wonder, it’s a blast.

We are now starting to use it to produce customer documentation for our buildout-based applications. Basically, a Sphinx structure is created in the buildout using sphinx-quickstart, and a few tweaks are made so the HTML and PDF outputs have a custom look.

Managing the documentation like the code makes life easier. This is one of the basic rule of agile documentation : separate the content from the layout, so you can provide documentation in any shape (html, pdf) with a single source.

To make things easier, I have released collective.recipe.sphinxbuilder.

This recipe:

  • creates for you a Sphinx-based documentation in your buildout
  • creates a single script in the bin folder to build the documentation with one command
  • provides an extensive set of options to drive Sphinx from buildout

Adding it in your buildout is as simple as :

[buildout]
parts =
  sphinx

[sphinx]
recipe = collective.recipe.sphinxbuilder

I have also customized the look and feel of the output so it uses the Plone logo and a custom css. This is configurable from the buildout configuration file of course. By the way, if someone from the Plone community wants to improve the CSS, please dot it ! (I am not good at this :) )

Notice that if you use LaTex or PDF rendering, you will need to install pdflatex. Furthermore, the recipe script will not work under Windows unless you install a Linux-like environment, since it uses the Makefile provided by Sphinx. I guess MSYS+MinGW should make it work, but I didn’t try.

Août 29, 2008
» Thank you, Plone community !


Wow.

I have been really involved in Plone for about one year now. This community just rocks. I met incredible guys, and did tons of work and experiments with them.

I have asked the community for help to go to the Plone Conf with a fund raising. I raised the funds in 3 days… I am astonished, this is a marvelous gift from you guys. I alsmot cried when I saw all those people donating.

I am so lucky, and so proud to be part of the Plone community and to know all of you. I will work hard to produce the best tutorial I can for the Plone Conf, and continue of course my work in the community tasks I am currently involved in.

I would like to say a big, warm thank to (the order is chronological, from the latest donation to the first one):

  • Andreas Jung
  • Veda Williams
  • David Glick
  • Jeff Kowalczyk
  • Youenn Boussard
  • Christian Klinger
  • Jesse Snyder
  • Alec Mitchell
  • John Habermann
  • Maurits van Rees
  • Jean-François Roche
  • Martin Aspeli
  • Alain Meurant
  • Aleksandr Vladimirskiy
  • Jon Stahl
  • Alexander Limi
  • Stephen McMahon

And a very special thanks to Calvin Hendryx-Parker.

This conference will be a blast for sure, if you can go there, do it !

Now I better find a good idea for those T-Shirts I have promised ! … :)

Août 28, 2008
» Plone development: Nose or zope.testing ?


Python needs a better testing tool, hopefully this will happen some days.

Until then we, Plone developers use zope.testing for the best and the worst.

  • the best because zope.testing provides layers and the Zope/Plone stack provides really nice things to work with them.
  • the worst because sometimes it just gives me some headaches to figure out how to set my test fixtures correctly.
  • the worst because I cannot easy-install zope.testing and just call it from the prompt to run some tests (this should get better sometimes see here)

That said, when you work with a zc.buildout based environment, it is easy to use zope.testing, thanks to zc.recipe.testrunner.

But since the eggification of Zope, we write more and more code that would benefit from a lighter test framework. Nose has this lighter approach, I love it to write simple tests without the Java-like heavy UnitTest framework. I can just write:

def test_something():
    assert 1 == 1

And call nosetests. Test fixtures are easy to set at all levels as well, using the with_setup decorator.

I tried to write some tests that could be launchable from both frameworks, by making some bridges, but I came up to the conclusion that a package should fully use either zope.testing, either Nose.

So now, I use two test scripts in my zc.buildout environments, and decide depending on the package. I have written another recipe for buildout to bind Nose: pbp.recipe.noserunner.

From there, I create two sections in my buildout.cfg file:

[buildout]
parts =
    ...
    nose
    ztest

[nose]
recipe = pbp.recipe.noserunner
eggs =
    egg1
    egg2

[ztest]
recipe = zc.recipe.testrunner
eggs =
    egg3
    egg4

These will generate two test scripts

  • nose to run nosetests with egg1 and egg2 in sys.path
  • ztest to run zope.testing with egg3 and egg4 in sys.path

Here’s my grid of choices when I am coding:

  • zc.buildout recipes : zope.testing (doctests using zc.buildout testing framework)
  • Plone packages: zope.testing
  • Python packages and ‘everything’ else: Nose

Août 27, 2008
» Help me go to the Plone Conference !


I have been asked to be a “champion” last winter at the Plone summit.

My task was :

Improve release procedures for add-ons on plone.org: document a release process, and create release tools for packaging and uploading products from the command line.

Since then, the major steps that I have done for it were :

  • change Python’s .pypirc format so it can be used to interact with PyPI and Plone. DONE
  • provide a package to make the new .pypirc format available under Python 2.4 and 2.5. DONE
  • make PloneSoftwareCenter PyPI-compatible. DONE

The steps left are:

  • continue the help on Plone.org Plone 3 migration, so it becomes reality
  • document the release process, based on PyPI, Plone.org and collective.dist

I am planning to go at the Plone Conference and to give a tutorial on all this work to deliver to the community the releasing process documentation. So these two remaining steps have to happen before the conference. I am also planning to sprint with people there.

My company contributes a lot to the Plone community. For instance we have organized the Paris Plone Sprint and we try to contribute as often as we can into the collective. But it has its limits and my airfare cannot be covered at this time.

This is the first time I am asking something to the community : please, help me go to the Plone conference !

I have created a page at ChipIn, and I am trying to raise the money to go to D.C.

What you get if you help me:

  • You will contribute to plone.org and release procedures enhancements
  • Since I can create t-shirts, I will create a unique Plone’08 DC logo T-Shirt that I will ship to the 5 biggest donators.

EDIT : It seems that Paypal won’t take your browser language settings into account. If you can’t read french you can try this hack I have found to switch to english :

when the paypal page is loaded add “locale.x=en_US&” into the url after “websrc?=”

so it looks like this:
https://www.paypal.com/fr/cgi-bin/webscr?locale.x=en_US&cmd=…

Reload that link and it should switch in english.

EDIT 2: don’t worry if Paypal tells you Amina El Kamel received the payment. This is my girlfriend Paypal account ;) .

Click on the cat :

Août 25, 2008
» Visual profiling with Nose and gprof2dot


Nose comes with a handy option to generate profiling stats.

To profile your code, create a test dedicated to this purpose and run it with the right options:

$ nosetests --with-profile --profile-stats-file stats.pf test_performance

This will run the tests that corresponds to the test_performance name and generate a stats.pf file.

Nose uses hotshot, so if you want to generate a file that can be read directly by the pstats module and all the statistics tools out there, you need to convert it using the hotshot.stats module.

From there, there is plenty of tools that can transform such a file into a visual graph. Most of the time, they use Graphviz to render a graph, by generating a file dot can read. This software is most of the time easy to install through a binary distribution on your system. If you need to compile it… good luck.. ;)

Anyway, from there, I use gprof2dot, which renders a nice graph with meaningful colors.

From the author:


The color of the nodes and edges varies according to the total time % value. In the default temperature-like color-map, functions where most time is spent (hot-spots) are marked as saturated red, and functions where little time is spent are marked as dark blue.








If you want to use it, I have created some console scripts for conveniency, you can install using easy_install :

$ easy_install pbp.scripts

It creates a gprof2dot script you can use, following the author documentation, but also a hotshot2dot script that will convert automatically a statistics file and pass it to gprof2dot:

$ hotshot2dot /path/to/my/hotshot/file

This will print in the output a dot file, you can send to the dot program, using a pipe:

$ hotshot2dot /path/to/my/hotshot/file | dot -Tpng -o output.png

You will get the visual result in output.png.

Août 20, 2008
» Atomisator, a framework to build custom RSS feeds


We are all overwhelmed by the amount of data in our feed readers. While this problem is unavoidable if you keep on adding new feeds in it, they could be automatically filtered and categorized to reduce the flow of data.

I wanted for a long time to try out some custom filters over my feeds to find for example related entries, by trying to understand the meaning of the posts, using tools like NLTK.

So I needed a playground for this, where I could play with feeds.

I think the closest tool for this is to use Yahoo Pipes but as far as I know, the only way to create custom filters is to run a web service and call it from Yahoo Pipes.

Anyways, I started to code a framework (at first it was an example for my latest book) that looks a lot like Yahoo Pipes in its principles. I don’t have any User Interface at this time of course, but a simple plugin-based tool that will let me combine my code snippets with feeds.

It is called Atomisator (see http://atomisator.ziade.org).

The big picture

The big picture

The process is quite simple:

  1. Readers are plugins that know how to read a source and provide entries out of it.
  2. Filters are plugins that know how to remove unwanted entries, or enhance them (change their title, summary, etc.). They can be combined.
  3. the entries are then pushed in a database. This is useful to avoid doublons, and to keep track of past entries.
  4. to create the feed, the entries are read from the database
  5. Enhancers are plugins that will add to entries extra info. Typically info that can’t be stored, like Digg comments if the entry is detected on Digg, or Google related searches, and so on
  6. The feed is then generated.

Right now I am focusing on making it fast, which is not simple because the plugins can play with all entries in the database.

It is in early stage and undertested, but it kinda works. I pushed it at PyPI to see of it meets interest. If it does, I will document the process of writing plugins.

Make sure you have SQlite installed, and give it a try :

$ easy_install atomisator.main
$ atomisator -c atomisator.cfg
$ atomisator

You will have an atomisator.xml feed created. You can add other feeds in atomisator.cfg as well and try them.

Now with this environment, I can start to try out custom algorithms over my feeds.

I’ve been told the name doesn’t sound right in Ehglish, but it does in French so I keep it ;)

Août 12, 2008
» iw.eggproxy, a smart PyPI mirror


Mirroring PyPI becomes a recurrent need for Zope development, because zc.buildout makes a lot of package downloads to build one application.

This is useful when you are working in an intranet with limited web access or when you want to speed up download times. It also makes things safer: if PyPI is down and if developers computers don’t have caches, having a mirror will save your day.

While PyPI has proven its robustness (it is 100% up for months now as far as I can see), having mirrors makes a lot of sense.

We have created a small mirror application here at Ingeniweb, that we use for our buildouts needs. This work was thaught and created by my colleague Bertrand Mathieu.

It is a smart proxy that will download packages at PyPI everytime they have been asked by a buildout or an easy_install client. When the package is downloaded, it is kept in the proxy side for any new requests. This means that after a while, the proxy has its own collection of packages that corresponds to the real needs and will not query PyPI anymore.

This approach avoids having to download and synchronize PyPI with crons, which is a heavy process since PyPI weight several gigas. The caveat of course, is that it won’t be able to get a new package if PyPI is down.

Take a look ! http://pypi.python.org/pypi/iw.eggproxy

By the way there is an interesting sprint coming up on all these topics, in Germany :

http://www.openplans.org/projects/black-forest-sprint/project-home

Août 8, 2008
» A new Python book : “Expert Python Programming”


The Packt team added a page about my new book, which will be out sometimes in September, so I guess it is the right time for me to announce it here !

So here comes “Expert Python Programming”, where I explain how we work with Python every day to create software.

This is my first book in English, and writing in another language was quite challenging ;)

Anyway, this book is intended for developers that already have a background in Python and covers only advanced topics (see the editor details). But as I said, it explains how we develop our applications in Python so topics like continuous integration, documentation, testing, releasing, refactoring, etc. are covered.
Managers will also have a good overview of how a Python project can be run and managed, using modern tools like Distributed Version Control Systems (Mercurial for instance) or Buildbot.

Even if I am working on Zope and Plone these days, I have focused on writing a book that is only about Python, to make it useful to any developer. The fact that my friend Shannon did the technical reviewing helped on this : he doesn’t use Zope too much so he was close to the target readership.

I will take more time when it comes out to announce it in the various mailing lists, and to get into greater details about the content that will be available online besides the book (some Python packages, etc)

I can’t wait to see it out and hold it :D

Juillet 29, 2008
» OSCON report #1: the city of Portland


Covering OSCON when you are a speaker is quite hard. Until my talk was done, I was more in a mood of reviewing my slides and not really thinking about other talks or blogging.

Convention center from the Bridge (the two twin towers)

I will try to write a few reports now that everything is over. This first report is about Portland, the city and is not technical at all.

This is the first non-technical entry in my blog, but let’s this it for once. So if you are looking for technical content, skip this entry :)

Portland is an amazing place. The weather was nice throughout the whole week: sunny and not too hot. Some evenings were a bit chilly but that felt nice to me. The convention center where OSCON took place was located on the other side of the Willamette river, in front of the city center. It is a bit far from the center, but finding places were you can put so many people must be hard I guess.

The top six things I have noticed about Portland:

Local beers are great. You can find many places where they brew their own beers, that are comparable to Belgian beers in quality and taste, like the Lucky Lab (Where Jon, a member of the local Plone user group took us, thanks!).  Some place have samples so you can taste several kinds of beers. I also went to the Oregon Brewers Festival right after OSCON was over, where you could taste beers from many places.

Beer Festival

Oregon Brewers Festival

Parking a car is a pain. I think the best way to travel in Portland is a mix of Tram, Bus and Bike. You can put your bike on the nose of the buses or inside the trams. Smart thing. This would be useful in Paris. We ended up parking our car far from the center and used the tram (called Max, and free in the center). Portland should set up a bike rental system like in Paris (Velib’), I am sure this would rock.

Organics: people here seem to be really concerned about sustainability and organics. I’ve been told there are a lot of organics farm around Portland. Supermarkets have a nice amount of organics stuff as well. This is nice.

Starbucks Coffee owns the streets: this is scary. there’s a Starbucks almost on each block here. Anyway, local coffee shops have better coffees and they provide nice places to chill out, read a book or talk with people. I think this is where you can feel the real Portland way of living.

The food is good : I have to admit, when we, french people, travel, we feel a bit superior to some countries on food matters. We tend to show off about it :). But Portland has great places to eat. If you like donuts, the place you must go to is called Voodoo Doughnut. They are creative !

Voodoo Donuts

Voodoo Doughnut

Big open spaces: I went to Mount Hood where people are skying in… July. I wanted to visit this place because it is where Stanley Kubrick did the outside shots of the famous motel in The Shining. I also had a walk at the Hood river sandbar, where people do kite surfing. One tip: don’t walk there with shorts on, the wind is so strong that you get slapped by the sand. So if you like big open spaces and sports, Portland is the place to be: all those place are one or two hours drive from the city center.

Mount Hood

Mt Hood

End of the aparté ! The next entry will focus on OSCON

Juillet 25, 2008
» My OSCON slides online (zc.buildout)


After a few attempts to make my screencast look nice under Google Video, I decided I would just upload the original ones (.mov and .m4v files) and a PDF export of my presentation.

So everything is available here: http://ziade.org/oscon

Slides: http://ziade.org/oscon/oscon.pdf

The .mov files are streamed automatically in your browser I believe, not the m4v ones.

Thanks to Jim Fulton for the quick feedback.

Juillet 18, 2008
» Going to OSCON


I am leaving tomorrow, heading to Portland, OR, to OSCON. My talk on zc.buildout and Plone will be thurdsay the 24th, and I’ll be there the whole week.

If you are going there and want to meet leave me a note, I am looking forward to meet other geeks there. :D

Juillet 9, 2008
» shutil.copytree small improvement


When you have to work with directories and files, there’s a very common pattern: copying a directory into another, but filtering out a few files and directories.

For instance, if you want to copy a directory that contains source code, you will probably remove .pyc files and .svn directories if you work with Subversion. In that case, shutil.copytree cannot be used, so os.walk is the usual way to go (rough example):

import os
from os.path import join, splitext, split, exists
from shutil import copyfile

def copy_directory(source, target):
    if not os.path.exists(target):
        os.mkdir(target)
    for root, dirs, files in os.walk(source):
        if '.svn' in dirs:
            dirs.remove('.svn')  # don't visit .svn directories           
        for file in files:
            if splitext(file)[-1] in ('.pyc', '.pyo', '.fs'):
                continue
            from_ = join(root, file)           
            to_ = from_.replace(source, target, 1)
            to_directory = split(to_)[0]
            if not exists(to_directory):
                os.makedirs(to_directory)
            copyfile(from_, to_)

This is a lot of boiler-plate code, so I usually create a small function that accepts more arguments to filter out files and directory. But Python should provide this pattern in the standard library.

I have proposed a patch for shutil.copytree, to integrate filtering capability. It has been reviewed and commited in the trunk this week, so we will have it in Python 2.6.  Now copytree comes with an ignore argument that has to be a callable. If given it will be called on each visited directory to decide what is copied and what is not.

There’s a default callable in shutil called ignore_patterns, that can be used to filter out files with glob-style patterns. I have added this example to Python doc:

from shutil import copytree, ignore_patterns
copytree(source, destination, ignore=ignore_patterns('*.pyc', 'tmp*'))

Pretty straight forward ! :D
More info and examples here : http://docs.python.org/dev/library/shutil.html#shutil.copytree

Juillet 2, 2008
» plone.org migration


Plone.org migration to Plone 3 is taking a bit longer than expected, but it should turn into reality soon.

There will be many improvements on the set of packages the website uses (I am thinking in particular about Maurits’s work on POI that will speed up the trackers, but it is just an example), and blobs should be used for the products section (more than 700 projects are registered there).

I worked last week-end on the products section, by finishing collective.psc.mirroring, which will copy all packages that are uploaded at plone.org into a file system directory. This directory will be published directly by Apache so the website will become a new package location for zc.buildout (find-links section) and easy_install calls without invoking the Plone instance.

Now I am focusing on PloneSoftwareCenter (PSC) migration. It is a pretty interesting topic: for every project located in the products folder with releases, I am going to extract its “distutils ids”. These ids are the name set in the setup.py file for each release.

I will then look at PyPI through XML-RPC if the package is also released, using the id. In that case, and if the author email is the same on both side, I will validate that the project on plone.org “owns” the given distutils id. From there PSC will act like PyPI and will reject uploads of packages if the user is not the owner of the project that owns the package id.

Of course there will be errors and some people might feel like their package has been hijacked if they cannot upload their packages. But this should be minor and should be OK after a while. A mail will soon be send to the community to ask people to check that they are synced between PyPI and plone.org.

I am really excited about this work because plone.org will then be compatible with distutils register and upload commands, which means that people will be able to update plone.org products section like they do with PyPI : through a single commande line.

Hey Sidnei, what you thought of several years ago is about to turn true. ;)

Edit: My apologies goes to Maurits, who did the work on POI, not Reinout, his brother ;)

Juin 19, 2008
» Python 2.4: tarfile module is buggy, patch it !


I have encoutered really bad bugs in some Python 2.4 applications (Zope based) using the tarfile module. For instance, the TarFile.getmembers method that returns the files a tar file contains will just fail to return all files …

Hopefully Zope will work under Python 2.5 sometimes. This is a work in progress in GSOC if I recall it correclty, but I am not sure of the current state.

Until then I just stick Python 2.5 module in my packages. It seems to work fine (tests pass :D) after a few changes as it is isolated from the rest of Python.

If you want to do it as well, there are only two changes to make it work under Python 2.4, to introduce 3 new constants in os module and to get rid of a syntax that does not work under 2.4:

$ diff /opt/local/lib/python2.5/tarfile.py tarfile.py
46a47,49
>
> os.SEEK_SET, os.SEEK_CUR, os.SEEK_END = range(3)
>
1070c1073,1076
<         self.name = os.path.abspath(name) if name else None
---
>         if name:
>             self.name = os.path.abspath(name)
>         else:
>             self.name = None

Juin 13, 2008
» collective.buildbot mailing list


Interested in collective.buildbot usage or development ?

Join us in the dedicated google group : http://groups.google.com/group/collectivebuildbo

Reminders:

  • collective.buildbot is a set of zc.buildout recipes and support for declarative configuration for Buildbot.
  • BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes, to set up a continuous integration system.

Juin 10, 2008
» zc.buildout on-going work


Just a quick post about the work going on in zc.buildout.

Work published lately:

  • Malthe and Mustapha have added a nice option to be able to add or substract values in variables that are inherited from an extended cfg file (already available, see the doc here)
  • I have added an allow-hosts option, that behaves like easy_install one, which can be used to restrict some accesses (see here)
  • Sidnei fixed some annoying bugs (missing quotes in some process calls)

On-going work:

  • I am working on a timeout config option, to be able to set the socket timeout (see my previous post on this). Andreas added a command-line option a bit ago as well, but we need to refactor it a bit to have it as a config file option.
  • I work on a python API so the buildout can be driven from the code. This useful for instance to create tests that are not using a os.popen or os.system call to build a buildout: a separated process is hard to debug.
  • We are having some thaughts on having a multiple-index enabled buildout. Still brainstorming.

Juin 8, 2008
» Eggs: releasing procedure and continuous integration


Disclaimer: this post is Subversion-centric

When releasing a setuptools based-package to the world, developers will eventually tag it as a stable version, then upload it to PyPI or to their own website. So typically they create a branch out of the trunk, fix the version, then create a tag for the release.

Now for continuous development releases, where people get for example a daily snapshot from the trunk, having a dev suffix to the egg version makes it possible to distinguish it from a stable release, so ‘my.package-2.4dev’ will not superseed ‘my.package-2.4′. So a given package will need a dev prefix in its trunk, that gets removed in the branch where the next release occurs.

From there, easy_install will be able to distinguish them.

The question is: how should we do this ?

The Zope 3 way

The Zope 3 egg collection has a simple way to manage it, describe here: http://svn.zope.org/*checkout*/Sandbox/philikon/foundation/releasing-software.txt. The packages have a dev tag suffix in the version metadata, that is removed in the subversion branch used to tag and release

This is simple and straight forward. Although, releasing the trunk has to be done manually. The problem is that any release of the trunk will have the same name for a given version (my.package-2.4dev) , and in some case people won’t upgrade their environment because the archive keeps the same name.

The setuptools way

Setuptools comes with a continuous integration feature that allows people to push the dev tag in setup.cfg. Then it adds it automatically to the archive name when it is generated.

see: http://peak.telecommunity.com/DevCenter/setuptools#managing-continuous-releases-using-subversion

Building a release will then consist of using the same process than Zope 3 one, except that the tag is removed from the setup.cfg file this time.

Now if you try to release the trunk using disutils sdist or bdist_egg command you will automatically get a dev suffix and the Subversion revision number sticked to it. This means that each new revision can generate a new version that will have a distinct name:

  • my.package-2.4dev-r1245
  • my.package-2.4dev-r1246

easy_install will grab the latest trunk revision when “my.package-2.4dev” is required, and handle upgrades the right way. This is better that a manual dev tag because when you re-release a new version of the trunk, it will superseeds and therefore upgrades previous revisions.

Another nice feature is to be able to provide to easy_install a subversion link directly, as long as you append the egg full name to the link:

easy_install http://my.svn/my.package/trunk#egg=my.package-dev

collective.releaser

The problem with setup.cfg is that if you forget to remove the dev tag from it when releasing a stable version (you bad boy), you will get the dev-r4565 suffix in the egg name.

collective.releaser takes care of this with a new setuptools command called release, by creating a release branch and removing the dev tag automatically. It upgrades the CHANGES.txt file and version.txt for Plone products.

It also registers and uploads the package to any PyPI-like server. To decide where the package should be sent, it looks at the release-packages variable into the .pypirc, to see if the package name matches it. To release all packages to PyPI a default file would be:

[distutils]
index-servers =
  pypi
[pypi]
username:tarek
password:secret
# regular expression-based variable
release-packages =
  .*

From there, making a release is as simple as:

$ python2.4 setup.py release
running release
This package is version 0.1.1
Do you want to run tests before releasing ? [y/N]: n
Do you want to create the release ? If no, you will just be able to deploy again the current release [y/N]: y
Enter a version [0.1.1]: 0.1.2
Commiting changes...
Creating branches...
...
Running "mregister sdist bdist_egg mupload -r pypi"
running mregister
Using PyPI login from /home/tarek/.pypirc
Registering iw.resourcetraverser to http://pypi.python.org/pypi
...
running mupload
Using PyPI login from /home/tarek/.pypirc
Submitting dist/iw.resourcetraverser-0.1.2.tar.gz to http://pypi.python.org/pypi
Submitting dist/iw.resourcetraverser-0.1.2-py2.4.egg to http://pypi.python.org/pypi
...
0.1.2 released

As described on its PyPI page, it has a plugin system to perform extra steps when a release is made. For instance, it is provided with a mail hook to be able to send mails everytime a release is made.

The tool is pretty new and needs to be smoothed up. For instance, we need to add a few extra controls like making sure long_description is reST-valid. But It works, and is being used by a few fellows in the Plone community already :).

Last, for continuous releases from the trunk, you can set up an alias:

$ python setup.py alias devrelease mregister sdist mupload

You will then be able to upload trunk releases with a single call:

$ python setup.py devrelease

The idea is to provide it as the standard tool to release add-on Products when Plone.org will be upgraded and able to interact with distutils commands. I will promote it through the work I am doing for the PSPS task I am championning (Improve release procedures for add-on products), but it has to live a bit.

Help and opinions welcome !

Mai 22, 2008
» DIY PyCon FR shirts


We made the shirts at home for PyCon FR and that was a LOT of fun, even if we had to work hard to make sure all shirt were ready by the time the event started. We used a silk-screen technique and the output looks really professional. The ink used makes the shirt design last longer than shirts that uses transfer. Although the number of colors is limited.

Have a look at the pictures : http://www.flickr.com/photos/82007723@N00/sets/72157605181699082/

Guess what: some people ordered us some t-shirts for some other events, so some more fun ahead ! :D