By Ingeniweb. A Django site.
Juillet 13, 2009
» Get your egg version form the metadata.xml file


A quick tip useful to keep the coherence between the setup.py file of your egg and the profiles/default/metadata.xml file. The simpliest way to do this is to compute version value in setup.py from the metadata.xml. In the setup.py file remove the version = ‘x.y.z’ line by the following code:

from xml.dom import minidom

metadata_file = os.path.join(os.path.dirname(__file__),
                             'collective', 'bobo',
                             'profiles', 'default', 'metadata.xml')
metadata = minidom.parse(metadata_file)
version = metadata.getElementsByTagName("version")[0].childNodes[0].nodeValue
version = str(version).strip()

You can check it works with the command:

python setup.py --version

» Manage monkey patches and performance improvement in Plone 3


Few days ago Jean-Michel François proposed a useful patch for PlonePAS that can be applied for all Plone release until 3.2. Plone 3.3 will embed this patch.

How can we add this patch in a traceable way for an not so old Plone 3.1 or 3.2 ?

First, we can use the new release  of Products.PlonePAS that should be compatible with our Plone installation. The second option is to add a monkey patch in the policy product of our site. One more monkey patch…

Some projects have so many monkey patches that it is difficult to know where is the code that run your site. Martin Aspeli did a tool to handle monkey patches in an elegant way for Zope 2 and Zope 3: collective.monkeypatcher. It allows you to plug  your monkey patches with a simple ZCML directive. Later Gilles Lenfant added a control panel for Zope 2 to be able to have a visual way to follow patches with collective.monkeypatcherpanel.

How does it works?

In your buildout.cfg add :

eggs +=
   collective.monkeypatcher
   collective.monkeypatcherpanel
zcml +=
   collective.monkeypatcher
   collective.monkeypatcherpanel

To create patches add a ‘patches.py‘ file in your egg (if you have more than 2 or 3 patches you should create a directory). Our performance patch looks like this:

import copy

def enumerateUsers( self
                  , id=None
                  , login=None
                  , exact_match=False
                  , **kw
                  ):

    """ See IUserEnumerationPlugin.
    """
    plugin_id = self.getId()

    criteria=copy.copy(kw)
    if id is not None:
        criteria["id"]=id
    if login is not None:
        criteria["login"]=login

    if not kw and id:
        data = self._storage.get(id, None)
        if data is None:
            user_info = []
        else:
            user_info=[ { 'id' : self.prefix + id,
                     'login' : id,
                     'title' : data.get('fullname', id),
                     'description' : data.get('fullname', id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } ]
    else:
        users=[ (user,data) for (user,data) in self._storage.items()
                    if self.testMemberData(data, criteria, exact_match)]

         user_info=[ { 'id' : self.prefix + user_id,
                     'login' : user_id,
                     'title' : data.get('fullname', user_id),
                     'description' : data.get('fullname', user_id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } for (user_id, data) in users ]

        return tuple(user_info)

In the configure.zcml of your policy product add an include:

<include file="patches.zcml" />

The file patches.zcml will contain following code:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:monkey="http://namespaces.plone.org/monkey"
    i18n_domain="collective.monkeypatcher">

    <include package="collective.monkeypatcher" file="meta.zcml" />

    <monkey:patch
       original="enumerateUsers"
       replacement=".patches.enumerateUsers"
       docstringWarning="false"
    />

</configure>

Run your buildout, start your site and the patch is applied. You can go in the Zope Control Panel to see how many patches are already compatible with this tool.

Juillet 7, 2009
» Le libre en question


Aujourd’hui un client nous met le couteau sous la gorge d’une manière assez peu banale. Il y a quelques années il a choisi de prendre une plateforme libre pour développer ses outils internes et une partie des services qu’il propose à ses clients.

Son choix a été de prendre Plone pour les raisons suivantes :

  • pas de coût de licence
  • un développement via l’interface web de Zope quasiment sans avoir à écrire de code
  • une documentation abondante et une communauté réactive
  • présence en France d’acteurs majeurs du développement international de la plateforme
  • une licence libre (la GPL) qui lui permet de penser que la plateforme ne deviendra jamais propriétaire

Le projet a eu des hauts et des bas. Il y a eu des formations externes, de la maintenance externe, du développement interne et externe. Globalement le projet a dépassé les espérances et est passé en mode industriel il y a un an. Le client possède toutes les sources et la connaissance de son applicatif. Il est maître de son destin. Avec le recul il est satisfait de cette plateforme et de son coût.

Ayant acquis une maitrise certaine de la gestion de contenu dans son corps de métier, ce client aimerai développer un module métier qu’il utilisera pour lui dans un premier temps, mais qu’il compte aussi revendre à ses concurrents à moyen terme. Pour cela ce module sera vedu comme une technologie propriétaire. Cela soulève une série de questions politiques et juridiques en interne (attention, certains de ces arguments sont partiellement erronés ) :

  1. si un site web utilise du code GPL pour fournir un service à des clients, un client peut demander à avoir accès au code source de l’application puisqu’il paie pour ce service
  2. si nous fournissons des sources sous GPL notre code source ne peut être commercialisé à des tiers sans qu’ils soient obligés de reverser ce code dans le domaine public
  3. si nous utilisons une extension sous GPL tout le code source devient GPL

Sur le fond je comprends la démarche de la vente d’un savoir faire que l’on veut protéger. Chez ce client tout ces constats ont amené a une réponse : pour faire du propriétaire il faut se baser sur une technologie propriétaire qui permet de faire ce que l’on veut une fois achetée.

Que faire ? Faut-il abandonner tous les développements en Plone  ? Il est tout à fait possible de réaliser un tel module propriétaire avec Zope et de l’intégrer avec d’autres systèmes sans que cela pose de problèmes de licence. Par contre, le Core de Plone 2 et de Plone 3 sont sous licence GPLv2 ce qui provoque un certain malaise. En anticipant de tels questionnements, la fondation Plone a décidé que le Core de Plone 4 serait sous licence BSD et  qu’elle accepterait que les modules d’extensions soient sous licence compatibles LPGL et GPL. Elle accepte aussi que des plateformes sous licence propriétaire soit développé à partir de Plone.

Dans notre cas précis ce questionnement est orienté : il faut vendre notre savoir faire pour générer des royalties et améliorer la rentabilité. Pour cela le choix d’une technologie libre est risquée et il faut adopter en masse une technologie propriétaire quitte à remettre en cause les développements en cours. Les coûts à justifier sont tels qu’il n’est pas possible de tolérer qu’un autre technologie puisse damer le pion à la technologie propriétaire à court terme car le choix de la technologie propriétaire est celui fait à moyen terme.

Il nous est proposé de fournir un canva pour fabriquer ce module dans la technologie propriétaire qui soit intégrable à Plone. Et dans tous les cas Plone va être abandoné. Autant nous mettre la corde au cou nous-même et attendre le bourreau.

Juin 29, 2009
» collective.releaser missing releases and upload bug


Since few weeks we have problems to use collective.releaser when we use release-packages: packages are not uploaded to our private pypi or are published in pypi when they should be private and public packages are not published at all.
collective.releaser seems be a victim of this bug as the 0.6.2 release was never published on pypi even if the tag was done 2 months ago.

Today ready this:
collective/releaser/commands.py:L107

Line 110, when the expression is matching something the loop continues, then in the line 117 an empty list is in found sometimes.

If you add these two lines after the line 112 you can again upload your releases in cheeseshops:

if founded != []:
    break

Juin 3, 2009
» Heads up: new plone.recipe.pound release


Since few weeks plone.recipe.pound 0.5.4 was broken with an output like this:

An internal error occured due to a bug in either zc.buildout or in a recipe being used:
Traceback (most recent call last):
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1509, in main
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 473, in install
File "/tmp/tmpVcq-b1/zc.buildout-1.2.1-py2.4.egg/zc/buildout/buildout.py", line 1091, in _call
File "/home/encolpe/.virtualenvs/internal/preprod/plone.recipe.pound/plone/recipe/pound/build.py", line 74, in install
installed = CMMIRecipe.install(self)
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 155, in install
system("make install")
File "/home/encolpe/.virtualenvs/internal/preprod/downloads/dist/zc.recipe.cmmi-1.2.0/src/zc/recipe/cmmi/__init__.py", line 27, in system
raise SystemError("Failed", c)
SystemError: ('Failed', 'make install')

This recipe directly unherit from zc.recipe.cmmi and act as a wrapper around the CMMIRecipe.install method. Most arguments used are transmitted to the recipe are self attributes or keys in self.options. The version 1.2.0 of this recipe publish the May, 18 moved the way that extra arguments are transmitted:

  • zc.recipe.cmmi 1.1.x:  self.options['extra_options']
  • zc.recipe.cmmi 1.2.0: self.extra_options (an empty string by default)

There’s no regression tests, neither an entry in the release history. Youenn did a new release of plone.recipe.pound to fix this new behavior.

Morality:

  • if you are using zc.recipe.cmmi 1.1.x you can still use plone.recipe.pound 0.5.4
  • if you are using zc.recipe.cmmi 1.2.0 or above use plone.recipe.pound 0.5.5
  • other recipes using zc.recipe.cmmi may be broken

Mai 7, 2009
» Install a maintenance page without restarting apache


These days we are working with fabric to remove some hazard in maintenance.
Recently someone gzip a Data.fs file in production and our customer loose 2 days of work. We choose to use fabric (http://www.nongnu.org/fab/) to limit command line working to the strict minimum.

Now, almost all procedures were added in a fabric factory, from the installation of a development environment to the upgrade of the production server. In this last part, we search a solution to put a maintenance page without have to gain root privileges to restart the apache used in frontend. For static or CGI sites you can use a .htaccess file to override the global rules. For a stopped Zope server it doesn’t help much.

The goal is to use RewriteCond and RewriteRule in a such way that a static HTML file would be displayed when it exists. We can call it ‘maintenance.html‘ and rename it ‘maintenance.html-disabled‘ to let the site be displayed normally.

We start with a virtual host generated by iw.recipe.squid:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

First we need to declare a document root and to bypass all security rules. We create a new path in the root of our buildout folder called ‘maintenances_pages’.

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

These declaration should satisfy all paranoid BOFH.
Now create a file called maintenance.html within the folder. All code (javascript, CSS and images) must be embeded, this is a limitation of this approach.
Well, how to detect that a file exist on the filesystem from apache. You can do this with the flag ‘-f’ from the rewrite module:

    RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f

Then redirect all pages to our maintenance page:

   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

We also need to disable the standard rewrite rules. The easier way to do it is to use ‘!-f‘.

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

These rules will not be applied anymore when our maintenance file will be name ‘maintenance.html’. Our goal is reached.

The whole virtualhost file look like this:

<VirtualHost *:80>
   ServerName www.gosseyn.fr

   RewriteEngine On
   RewriteLog /my/path/to/squid/parts/squid/log/rewrite_www.gosseyn.fr.log
   RewriteLogLevel 0

   CustomLog /my/path/to/squid//parts/squid/log/access_www.gosseyn.fr.log common
   ErrorLog /my/path/to/squid//parts/squid/log/error_www.gosseyn.fr.log

   DocumentRoot /my/path/to/buildout/prod/maintenance_pages

   <Directory "/my/path/to/buildout/prod/maintenance_pages">
       Options None
       AllowOverride None
       Order allow,deny
       allow from all

       <LimitExcept GET>
           Order allow,deny
           allow from all
       </LimitExcept>
   </Directory>

   ## our maintenance page
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html -f
   RewriteCond %{REQUEST_URI} !/maintenance.html
   RewriteRule $ /maintenance.html [R=302,L]

   ## common rules for squid rewrite rules
   RewriteRule ^(.*)$ - [E=BACKEND_LOCATION:127.0.0.1]
   RewriteRule ^(.*)$ - [E=BACKEND_PORT:8081]
   RewriteRule ^(.*)$ - [E=BACKEND_PATH:site]

   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)/$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]
   RewriteCond /my/path/to/buildout/prod/maintenance_pages/maintenance.html !-f
   RewriteRule  ^/(.*)$ http://127.0.0.1:3128/%{ENV:BACKEND_LOCATION}/%{ENV:BACKEND_PORT}/http/%{SERVER_NAME}/80/%{ENV:BACKEND_PATH}/__original_url__/$1 [L,P]

   ## specific rules base on cookie

</VirtualHost>

It will be difficult to include this in a recipe as we modify an existing virtualhost in a very specific way.

As usual, all feedbacks are appreciated.

Mars 16, 2009
» How to extend you Plone 3.2 buildout


Plone 3.2 and next comes with a hardcoded versions.cfg for each Plone release. For the current release:
http://dist.plone.org/release/3.2.2/versions.cfg

The Plone Upgrade Guide will show you how to transform a basic buildout created with paster:
http://plone.org/documentation/manual/upgrade-guide/version/upgrading-from-3-x-to-3.2

This solution will work until you already defined a ‘[versions]‘ section in your main buildout file. What we need is to merge the two ‘[versions]‘ sections.

For example, you current ‘buildout.cfg‘ looks like this:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

versions = versions

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

If you add extends = http://dist.plone.org/release/3.2.2/versions.cfg it will be overloaded by the local section displayed above and will be simply ignored by ‘bin/buildout‘. If you run an update now all plone bundle eggs will be updated to the last published eggs (Plone 3.3b1, etc).
The solution consist to use the ‘extends’ directive to merge sections. For that we need to put your local ‘[versions]‘ section in a separate file.
dev-versions.cfg‘ will contain:

[versions]
zope.testing=3.5.1
zope.interface=3.4.1
Products.errornumber=1.2
archetypes.schematuning=1.1
Products.CacheSetup=1.2

The ‘buildout.cfg‘ will contain:

[buildout]
...
eggs =
	archetypes.schematuning
	Products.CacheSetup
	Products.errornumber
	collective.recipe.omelette
	collective.workflowed
	Products.DCWorkflowGraph
	Products.PrintingMailHost
	plone.reload
	Products.PDBDebugMode
	Products.DocFinderTab

extends = dev-versions.cfg http://dist.plone.org/release/3.2.2/versions.cfg

versions = versions

We can update now and verify our eggs versions:

  • Plone 3.2.2
  • plone.app.locales 3.2.0
  • Products.CMFPlacefulWorkflow 1.4.0
  • Products.CacheSetup 1.2

It is a certified Plone 3.2.2 installation with our specifics packages.

Mars 5, 2009
» ‘Practical Plone 3′ review


I finished to read the long waited (read updates here :) ) ‘Practical Plone 3′ book. It reprensents a big amount of very good work.

If you want to begin in Plone or if you want a webmaster guide for your brand new Plone site the two first parts are designed for you. Each step of your needs are describe in a very good teaching way.

Parts are growing in difficulties. The first will show you how to install and what in installed in a default Plone site. The second part will learn you how to become the owner of your site by creating content and configuring it. The third part is for those who wanted to learn how to use addon products for Plone or want to customize Plone site: zc.buildout, PloneFormGen and ArchGenXML are presented here. The last part will show you some general needs around Plone: put Plone in production behind Apache or IIS with speed optimizations, and LDAP/Active Directory integration.

Even if the target of this book is beginners it was a very interresting reading for an old developper like me. If you don’t use it for yourself, you will use it to give answers to your end-users. I can only recommand everyone to buy it. Nice work guys.

Février 24, 2009
» Don’t forget to write tests for your patches


As all new major releases Plone 3.2 is heavy tested by early integrators that want the new version. They report bugs and sometimes patches. Hopefully most of them are using ’svn diff’ or ‘diff -u’ to prepare those patchesinstead of saying: “Replace the line XX by this and add that line here”…

But the one thing they never do it is to add a test to demonstrate their bugs. Then one of the maintainer have to take time to write a test around what (s)he think to be the bug. Often, the patch is just applied, Plone is growing with another untested code and the life continue.

Please help maintainers: if you are able to write a patch you should be able to write the associated test.

Février 22, 2009
» Which mind mapper tool are you using ?


Since one year now I am using mind mapping tools to manage knowledge around private and professional projects. I tried only Open Source (but not always free) software :

All are very intuitive and propose import/export for various format but only FreeMind have its Plone addon. It is a flash viewer for .mm files.
None of them propose collaborative edition.

I didn’t find any python program that implement some mind mapping features in pypi neither on search engines.

In January we generalized the use of mind mapping tools and saw that a lot of us had started to use them since a while without communicate around this need. All these softwares can import map from mindmanager but they cannot import or export map from one of them. The work to merge all our maps to one format will be hard.

Which mind mapping tool are you using ?

Février 19, 2009
» Fix PIL 1.1.6 installation from sources under debian


Today I found a bug in kupu integration in Plone. I prepared a patch and I tried to reinstall plonenext environment. But this time it fails to install on the PIL egg.

The PIL egg is very dependant to your architecure. It depends of libjpeg, tkinter (tcl/tk), and some other libraries. I’m working with a Debian unstable with 3 versions of tcl/tk for various programs: 8.3, 8.4 and 8.5.

The errors was the following:

getting distribution for ‘PIL’.
libImaging/Effects.c:210: warning: ‘perlin_init’ defined but not used
libImaging/Geometry.c:236: warning: ‘quadratic_transform’ defined but not used
libImaging/Quant.c:311: warning: ‘test_sorted’ defined but not used
libImaging/Quant.c:676: warning: ‘checkContained’ defined but not used
libImaging/QuantHash.c:136: warning: ‘_hashtable_test’ defined but not used
_imagingtk.c:20:16: error: tk.h: Aucun fichier ou répertoire de ce type
_imagingtk.c:23: error: expected ‘)’ before ‘*’ token
_imagingtk.c:31: error: expected specifier-qualifier-list before ‘Tcl_Interp’
_imagingtk.c: In function ‘_tkinit’:
_imagingtk.c:37: error: ‘Tcl_Interp’ undeclared (first use in this function)
_imagingtk.c:37: error: (Each undeclared identifier is reported only once
_imagingtk.c:37: error: for each function it appears in.)
_imagingtk.c:37: error: ‘interp’ undeclared (first use in this function)
_imagingtk.c:45: error: expected expression before ‘)’ token
_imagingtk.c:51: error: ‘TkappObject’ has no member named ‘interp’
_imagingtk.c:55: warning: implicit declaration of function ‘TkImaging_Init’
error: Setup script exited with error: command ‘gcc’ failed with exit status 1

All development libraries was installed, but the installer cannot find /usr/include/tk.h. There are /usr/include/tcl8.3/tk.h, /usr/include/tcl8.4/tk.h, /usr/include/tcl8.5/tk.h. Then I wrote the little patch below for the setup.py file:

Index: setup.py
===================================================================
— setup.py    (révision 460)
+++ setup.py    (copie de travail)
@@ -199,6 +199,9 @@
add_directory(library_dirs, “/usr/lib”)
add_directory(include_dirs, “/usr/include”)

+        if os.path.isfile(os.path.join(’/usr/include’, ‘tcl’+TCL_VERSION,’tk.h’)):
+            add_directory(include_dirs, os.path.join(’/usr/include’, ‘tcl’+TCL_VERSION))
+
#
# insert new dirs *before* default libs, to avoid conflicts
# between Python PYD stub libs and real libraries

I hope this patch is generic enough to be merged in the next Imaging release. Waiting this next release, I’m searching for a method to patch an egg before it is compiled by zc.buildout.

Octobre 5, 2008
» Why Plone architecture must change


I proposed a conference on this theme but it was not keeped for PloneConf2008. Here comes a short work on its, and I will hope it create a debate.

Plone is born with a goal: make Zope 2 and CMF simple. Those two are frameworks and allowed you to create a website… a so rustic website. With Plone you only needed to fill one form to have a cool site ready to use and to custom. At that time Plone was CMFPlone, an other implementation of CMFDefault.

Some new products were inserted in the bundle like GroupUserFolder in a such way that they cannot be separated from the base implementation of Plone.

The second step was the Archetypes framework: a new manner to manage object’s attributes as schema. With all informations stored in the schema,  mutators and accessors are generated on boot time and forms are generated from schema’s widgets. It was cool, but not perfect. Then the product ATcontentTypes was created to be the glue to upgrade CMFPlone as Plone: all base types from CMF are now overloaded by a type from ATContentTypes.

So came Zope 3 and Five. Five is the glue that allows Zope 2 developpers to use component architecture from Zope 3 in their code. Plone 2.5 and Plone 3 are using Five.

Another product came from Zope ‘renegades’: PluggableAuthService and its implementation in Plone: PlonePAS. It should replace the old GroupUserFolder but users and groups management templates were never refactored.

Plone 3 introduces new component in plone.app to our greatest happiness.

What part of Plone needs CMF?
Why Plone needs to know about Archetypes storage and CMFEditions strategy?
Why GroupUserFolder is always in the bundle as PlonePAS fit is API?

Now Plone is like the linux kernel: a big monolithic Plone with a lot of modules that create a base Plone 3 site. And so much glue! GroupUserFolder is always here because nobody knows and wants to work on the portal_group replacement.
If you are following Plone4Artist or PloneGov you can see that a part of these projects needs to overload Plone base configuration.

CPS 3, an other CMF-based CMS, was conceived with the thinking that components need to be independant to be reusable and maintainable:

  • CPSSchema depends only on Zope
  • CPSCore depends only on Zope
  • CPSDocument depends only on CPSSchema
  • CPSDefault depends on CPSCore and CPSSchema, and implements the CPS site.

After 2 years it was divided into platforms:

  • CPS Legacy
  • CPS Courier
  • CPS Groupware

In front of that Plone propose at single product with a lot of glue that depends on others products or components that use their own glue and so on…
There’s no plone.core and plone.default and we cannot create a plone.artistsite or a plone.govsite.
Do you think that everyone need an openid or an ldap integration in Plone ?

Plone 4 must be a reimplementation, not only a new glue with new concepts. I don’t want any new functionality in Plone 4, I want modularity and scalability.

Octobre 2, 2008
» La gestion des modules et des thèmes dans Drupal 6


Avant de me lancer dans la conception d’un thème j’ai voulu essayer de voir comment Drupal fonctionnait au niveau de la gestion des extensions. Tout d’abord les thèmes et les modules sont séparés ce qui permet d’intégrer plus facilement un concepteur graphique dans un projet. Les thèmes sont à rajouter dans ‘sites/all/themes‘ et les modules dans ‘sites/all/modules‘.

Je cours chercher un premier module à intégrer dans mon site: FCKeditor. J’utilise déjà cet éditeur en ligne dans Plone, cela me permettra de comparer la difficulté d’installation dans les deux cas.
Pas de chance, ce module est relativement complexe à installer. Une fois l’archive décompressée dans votre arborescence il faut encore aller chercher une version de fckeditor sur son site officiel et la déposer dans le module. J’espère que la dernière version de FCKeditor est compatible avec cette version du module.
Ensuite, pour avoir la possibilité d’utiliser un navigateur de fichier lors de l’édition des pages il faut encore suivre une procédure pour l’activer. Le fichier README est là pour me guider, mais la procédure est pour Drupal 5.2. Je croise les doigts.
Au final, de retour dans l’interface de configuration des modules le menu FCKeditor est bien présent, mais il me demande d’aller configurer les permissions pour savoir si les anonymes et les utilisateurs authentifiés peuvent utiliser ce module… beaucoup de chose à faire pour avoir une installation fonctionnelle par défaut.

Les autres modules présents par défaut dans Drupal 6 présentent un niveau de complexité moins élevés dans le sens où il faut les activer (forum, blog, sondage,…) puis gérer les permissions.

Pour installer FCKeditor dans Plone il faut décompresser l’archive, redémarrer le serveur puis appuyer sur le bouton ‘installer’ dans l’interface de configuration. L’installation est fonctionnelle immédiatement sans même avoir lu le README.

Pour moi c’est un mauvais point pour drupal : mes modules n’intègre aucune procédure d’installation automatique qui les rend fonctionnel
immédiatement pour un débutant. Cela me ramène 4 ans en arrière pour le monde Zope.

Maintenant que je peux créer un peu de contenu je vais pouvoir m’atteler à la création d’un thème.

Septembre 30, 2008
» Installation de Drupal chez Nexen


En tant que salarié du groupe Alterway nous avons eu compte d’essai chez Nexen services pour hébergé un site en PHP/MySQL. En temps que développeur Python nous nous sommes posé la question: « C’est bien gentil, mais à quoi cela va-t-il nous servir ? ». Pourquoi ne pas tester un framework LAMP en reprenant un site fait de bric et de broc pour en faire un site professionnel… pour l’instant le nom du site restera secret.

Après investigation auprès de mes camarades de Nexen c’est sur Drupal que je vais faire mes armes. Cela fait 5 ans que je n’ai pas touché à du PHP, j’ai un brin de nostalgie. La première chose à faire est de trouver de la documentation. Un coup d’œil dur le site de la communauté drupal française et me voilà avec un tutorial un peu court mais suffisant pour commencer.

Viens la question fatidique de la gestion du projet. Je serai seul pour le moment, mais le développeur de la version précédente du site viendra sans doute me donner un coup de main durant la phase de migration. De plus, je voyage beaucoup, un système centralisé n’est pas envisageable.

Il me reste bazaar, darcs, mercurial et git. J’ai déjà testé les trois premiers avec un penchant pour darcs mais c’est un bonne occasion pour tester git qui semble avoir le vent en poupe. un coup de ‘man
gittutorial’ pour démarrer :

Préparation des sources en local

$ tar zxf /home/encolpe/Desktop/drupal-6.4.tar.gz
$ mv drupal-6.4/ drupal-nexen
$ cd drupal-nexen/
drupal-nexen$ git init

Initialized empty Git repository in /…/drupal-nexen/.git/

drupal-nexen$ git config –global user.name “Encolpe Degoute”
drupal-nexen$ git config –global user.email encolpe@gmail.com

drupal-nexen$ git add .
drupal-nexen$ git commit

drupal-nexen$ tar zxf /home/encolpe/Desktop/fr-6.x-1.x-dev.tar.gz

drupal-nexen$ git status

drupal-nexen$ git add */*/translations *.fr.txt
drupal-nexen$ git commit

La prise en main de git semble simple jusque là.
Faire un ‘git add .’ est beaucoup plus explicite que la commande ’svn import’: c’est le dossier courant qui est importé, pas de doute possible.

Il faut ensuite configurer un client FTP pour déposer vos fichiers sur votre compte hébergé. Je choisi le logiciel en ligne de commande ‘ncftp‘ dans l’espoir de pouvoir le coupler avec les commits dans la branche principale de git pour gagner un peu de temps.
Ce couplage devra être asynchrone…

J’ai maintenant un beau site qui me montre sa page d’installation sans me demander de mot de
passe (sic). D’ailleurs il ne semble pas très content. Soit disant il faut que je copie ‘sites/default/default.settings.php‘  dans ‘sites/default/settings.php‘ . Pas de panique.

drupal-nexen$ git mv sites/default/default.settings.php sites/default/settings.php
drupal-nexen$ git commit sites/default

Ensuite vous pouvez modifier le paramètre db_url pour les accès à votre base (notez la triple présence de username) :

$db_url = ‘mysql://username:********@mysql.username.nexenservices.com/username’;

Après un nouveau commit et le transfert du fichier vers votre compte hébergé vous pouvez maintenant terminer l’installation en choisissant la langue de votre site et en laissant faire l’installeur… ou non.

Erreur en installant Drupal

Erreur en installant Drupal

Vu d’ici cette histoire de ‘register_globals‘ à désactiver semble un brin complexe. Est-il possible de configurer ce genre de paramètre avec mon compte Nexen ?

Il faut aller voir sur la page d’accueil du compte http://www.nexenservices.com/webmestres/index.php pour aller chercher la solution. Une fois connecté il faut choisir votre site mutualisé en bas de la colonne de gauche puis cliquer sur ‘Configuration avancée‘ à nouveau en bas de la colonne de gauche.

Il faut configurer la racine du site, soit le dossier ‘/‘ et désactiver ‘register_globals‘, ce qui se fait en deux étapes et en image :

Choisir php et / pour le dossier racine

Choisir php et / pour le dossier racine


Il suffit de laisser les choix par défaut et de valider

Il suffit de laisser les choix par défaut et de valider

Maintenant il faut reprendre l’installation en allant sur la page http://mysite.nexenservices.com/install.php et en recommençant la procédure complète. L’installation se termine bien cette fois.

La prochaine étape sera de créer un thème pour le site. Nom de code: Opération Zèbre.

» De l’ergonomie des formulaires


Aujourd’hui un client m’a demandé comment rajouter un sous domaine chez gandi. Pas de problème, j’en profite pour en créer un aussi pour mon prochain billet.

  1. se connecter
  2. cliquer sur l’onglet « Administration »
  3. cliquer sur le nom de domaine choisi
  4. en bas de la page, dans les « Paramètres techniques », il faut choisir « Gérer les zones »
  5. choisir « Ajouter un enregistrement » en bas de la page
  6. Type: A ou CNAME
  7. Nom: sous domaine à créer
  8. Valeur: adresse IP (type A) ou nom réel de la machine (type CNAME)

La page « Gérer les zones » est mis à jour.

Mais une heure plus tard le nouvel enregistrement à disparu. Tiens, un fantôme dans la machine…
Je recommence la même série de manipulation et je cherche ce que j’ai pu raté.
En fait la constatation finale n’est qu’une étape car un bouton est apparu en haut de la page, à l’opposé des autres boutons. Très bien intégré dans la charte il passe inaperçu au premier coup d’œil. Le bouton dit « Valider les changements ».

Outre le fait que lors de l’ajout du nouvel enregistrement rien ne disait explicitement qu’il était temporaire, le fait que ce bouton apparaisse en haut et soit noyé dans la charte alors que les autres boutons sont en bas de la page est un bon exemple de ce qu’il ne faut pas faire : seule une personne connaissant déjà cette interface va remarquer cette nouvelle zone ou des boutons sont apparus comme par magie. Dans un cas pareil il faudrait avoir un décrochement de la charte graphique pour mettre en avant ces nouveaux boutons et mettre en avant leur importance pour la suite des opérations.

Pour l’instant, nos attendons notre sous-domaine pour tester un nouveau site en conditions réelles.
C’est long d’attendre…

Septembre 25, 2008
» A new Python Web Framework based on Stackless Python


Just a bill to announce the first release of Nagare that use stackless Python.
If you want to test it you can try the Full Demo
If you want to install it you can follow the Quick Start Guide

Septembre 16, 2008
» Trying out ScribeFire


I’m always on the road and offline editing is for me the main missing point of CMS and Blog systems.
Today, I’m switching to ScribeFire Firefix extension to edit and publish this blog. The problem with this approach is that you need one external application to manage the web application you are using…
On Thursday I will be in the Google Developer Day 2008 in Paris to learn more about Gears roadmap. Gears is interreting to me because it allows you to make your web application working offline.

Juillet 4, 2008
» Write your first Zope 2 product


‘The topic is to make understandable what is a Zope product in the python world.

What is a python module?

A Python module is a directory on your filesystem that contain at least a file called __init__.py. This file can be empty, but the name is fixed.

How Python can find such a directory?
The standard path where python search for modules is in the directory site-packages of your Python installation. By default you find it in C:Python2.4Libpython under windows or in /usr/lib/python2.4 un Unix-like systems (Linux, Mac OS, BSD, etc). You can extend this search by adding the PYTHONPATH variable in your environment.

@set PYTHONPATH=C:Zope2.9libpython

or

export PYTHONPATH=/usr/lib/python2.4:/opt/Zope2.9/lib/python

In a Python module every sub-directory that contains a file named __init__.py is considered as a sub-module.
On these points Zope doesn’t differ from Python

What is a Zope Products?

You can divide your Zope installation in three parts: the Zope framework, the Zope Server and the ZODB. A Zope product is an application for the Zope Framework. You can use it alone, without the Zope Server or the ZODB. Actually it’s the failure of Zope to not have any public software based on the Zope Framework without the Zope Server.
Zope 2 is using a specific code to load products to register them as specific applications. You can read it in your Zope installation in the file OFS/Application.py.

During the warm up the Zope Server is looking in the zope.conf file for the ‘products‘ keywork. Each time it is defined this keywork is following by a filesystem path where Zope Products are uncompressed.
This list of filesystem paths is initialized with the two paths:

  1. the lib/python/Products directory of your Zope installation
  2. the Products directory of the Zope instance

The first one contains base application you need to work like ZCatalog, PythonScripts (for ZMI). You should never add a product in there and read the documentation available in these products will help you to understand Zope Framework standards.
The second one is empty and is designed to store cusomer Zope Products.

Implement your first Zope 2 product

To create your product you create a Python module in the Products directory. You should Have something like this:

Products

> MyProduct

> __init__.py

To be recognized as a Zope Product the __init__.py file must contain a function called initialize. This function is mainly declarative. You can read a full sample on the document ” Write a toolbox for a Plone 2.5 product “.
The main goal of the initialize function is to register your new classes into the good drawer in the Zope Application. In the sample we register a new tool object but you will do the same for content types, i18n domains, etc.

» Write a toolbox for a Plone 2.5 product


For years we have had the need to embed a utility in Plone products that allows us to execute private methods in restricted Python mode. You need such a utility when using a ZMI Python script in a Page Template or in an Expression. The last one is a CMFCore class that is used in workflow transition condition or in Archetypes field condition for example. To be able to do this utility you need to create a ‘portal tool’ for your site with some public methods. Here I will show you how to build a very basic utility that I can use in the next blog entry.

What is a tool

In pure Zope 2/CMF, a tool is a unique item with a forced id that inherits from the SimpleItem class. An instance of a tool is always created on the root of your Plone site to be acquirable from everywhere in the site.

The CMFCore product provides a function that allows us to call a tool in an unique way: ‘getToolByName’.
In python modules, external methods, or python scripts, you can use following code:

from Products.CMFCore.utils import getToolByName
mytool = getToolByName(obj or context, 'tool id', default value is not found - None if not specified)

You can use it in Page Template too, using the ‘module’ call:

python:modules['Products.CMFCore.utils'].getToolByName(here, 'tool id')

Implementation

What should a tool module contain

Now back to our implementation.

First you need to have a product to write in. To create a new product please read this howto , or download someone else’s to play with.
In your product create a file called ‘tool.py,’ if there will be only one tool in your product, or ‘myfunctiontool.py’ if there will be more than one. As a best practice, I recommend to always choose the second way.

A simple tool code could be:

from OFS.SimpleItem import SimpleItemfrom Products.CMFCore.utils import UniqueObject

from AccessControl import ClassSecurityInfo

from Globals import InitializeClass

class ProductNameTool(SimpleItem, UniqueObject):

    """ Description of what your tool handle
    """

    id = 'portal_productname'

    meta_type = 'Product Name Tool'

    manage_options = SimpleItem.manage_options
    security = ClassSecurityInfo()

InitializeClass(ProductNameTool)

Description of imported items:

  • SimpleItem is a class coming from Zope that ensures us that we have persistence and base views available in ZMI.
  • UniqueObject is a utility class coming from CMF that ensures us that nobody can rename an instance of it. It’s important for us because our tool must never be renamed by error.
  • ClassSecurityInfo is the class to use in every Zope persistent class in order to define the security of each method.
  • InitializeClass is the helper function that registers our class in the Zope context.

Description of class attributes:

  • The id must be unique on your site root and should start with ‘portal_’ or end with ‘_tool’. If your tool unherit from ZCatolog the id must be suffixed by ‘_catalog’.
  • All classes must have a meta type. You can just extend your class name with spaces.
  • manage_options contains the tabs displayed in the ZMI.
  • security defines the security in your class. All classes must contain such a declaration.

Initialize our tool at Zope startup

Now that Zope knows that you have a tool, what about Plone? Plone requires an extra step in order to register our tool (full details in: “Write your first Zope 2 product“.
We have to edit the __init__.py file of our product and add following lines:
 

from Products.CMFCore.utils import ToolInit

from myfunctiontool import ProductNameTool

tools = (ProductNameTool,)

def initialize( context ):

    ToolInit( 'My Product Tools'
                , tools=tools
                , product_name='ProductName'
                , icon='tool.gif'
                ).initialize( context )

ToolInit is a class that handles tool registration for us. We can register all tools defined in our product in only one pass.
ProductNameTool is our tool class. Here we need to import all tool classes we want to register.

At last, we call the ToolInit constructor and run its ‘initialize‘ method to build our tool factories:

  • the first parameter is the name viewed in the Add dropdown menu in the ZMI
  • tools is a tuple of classes
  • product_name should contain the name of your product
  • icon is the name of the icon displayed in the ZMI (16×16) that must be on the root of your product

We can restart Zope then add the tool in the ZMI. After that, we can add methods to this tool. Most often these methods are public or protected by ‘View’ or ‘Manage portal’ permissions.

Future of Zope 2 tools

In Zope 3 tools are becoming utilities. In Plone 3 we are in limbo, with all old Zope 2 tools registered both as tools and as utilities. In the near future, however, only the utilities will be used. Use Tool for Plone 2.5 development only.

You can read more about utilities in Zope 3 and Plone 3 documentation.

Juin 21, 2008
» Use google document to create the Plone Documentation


Today I was writing a little documentation about Plone 2.5 in reaction to a first draft on workflow management by groups. My goal is not to write a document alone that will replace this draft but to imply its author. A this point a tool was missing until I remember we can use Google Docs application to share our ideas and write this document together.

We have shortly try out Google Docs and our first impressions are positive: each other can write in the same document and modification are reported in live on the other screen. Add a GTalk window or an IRC session to discuss and you have a very powerful tool.

The next step is to finish this document and to copy/paste it in Plone Documentation. I hope to do this tomorrow night.

Hey Limi, when will we embed this in Plone ;^)