By Ingeniweb. A Django site.
Avril 29, 2010
» AllowedContentType in Plone2.5

Hello,

I notice that the method allowedContentTypes cost time when you have a lot of type in plone2.5.I don’t know if plone3 or plone4 are impacted of that. I have 120 types and the time of execution of context.allowedContentTypes is about 0.32sec.

The path of allowedContentTypes is :

allowedContentTypes -> portal_types.listTypeInfo -> for each content type: portal_types.isConstructionAllowed -> portal_types._queryFactoryMethod -> Products.Five.pythonproducts.patch_ProductDispatcher__bobo_traverse__ -> Products.Five.pythonproducts.product_packages

and Products.Five.pythonproducts.product_packages time call is 0,003s

When you have 100 content type only product_packages is responsible of 0,3sec , this method is a performance bottleneck.

So add this patch fix the problem :

from Products.Five import pythonproducts
old_product_packages = pythonproducts.product_packages
pythonproducts.product_packages =  forever.memoize(old_product_packages)

Regards Youenn


Janvier 12, 2010
» DateTime against mx.DateTime

I notice that in a zope application that I have to maintain is slow because of DateTime class.
The profile in this application test give the top time to this class.
So I want to test an other implementation which is name mx.DateTime. The difference is that mx.DateTime is writen in C.
So in a terminal , I install the two eggs via easy install :

./bin/easy_install DateTime
./bin/easy_install egenix-mx-base

And do a little script for testing the two api:

import sys
from mx import DateTime as mxDateTime
from DateTime import DateTime
from datetime import datetime
from time import time

def create_mxdatetime():
    return mxDateTime.now()

def create_zopedatetime():
    return DateTime()

def create_datetime():
    return datetime.now()

def bf(f, i):
    t1 = time()
    for i in xrange(i):
        f()
    t2 = time()
    print "bench for %s is %s" % (str(f), t2 - t1)

bf(create_mxdatetime, int(sys.argv[1]))
bf(create_zopedatetime, int(sys.argv[1]))
bf(create_datetime, int(sys.argv[1]))

This script just create Date in the three implementation : zope, mx and the standard library

And the results:

bash-3.2$ bin/python bench.py 1000
bench for function create_mxdatetime at 0x7db70 is 0.00120091438293
bench for function create_zopedatetime at 0x4215f0 is 0.84446310997
bench for function create_datetime at 0x4214b0 is 0.00220394134521
bash-3.2$ bin/python bench.py 10000
bench for function create_mxdatetime at 0x7db70 is 0.0117778778076
bench for function create_zopedatetime at 0x4215f0 is 8.81699991226
bench for function create_datetime at 0x4214b0 is 0.041069984436
bash-3.2$ bin/python bench.py 100000
bench for function create_mxdatetime at 0x7db70 is 0.11746096611
bench for function create_zopedatetime at 0x4215f0 is 87.8845770359
bench for function create_datetime at 0x4214b0 is 0.222129106522

No comment.. and in memory ?

So

bash-3.2$ bin/easy_install pympler

and now::

>>> from pympler import asizeof
>>> from datetime import datetime
>>> from mx import DateTime as mxDateTime
>>> from DateTime import DateTime
>>> asizeof.asizesof(DateTime() , mxDateTime.now(), datetime.now())
(1760, 56, 32)

Hoaaa !! a zope DateTime is 798 time slower than mxDateTime and it consume 31 more space than mxDateTime

I think ReplacingDateTime could be a good performance issue for zope, no ?

With DateTimeNG (zope DateTime with mx.DateTime) performance is 10 time better than DateTime. Memory consume is the same

From mx.DateTime documentation
Comparing the types to time-module based routines is not really possible,
since the used strategies differ. You can compare them to tuple-based
date/time classes though: DateTime[Delta] are much faster on creation, use
less storage and are faster to convert to the supported other formats than
any equivalent tuple-based implementation written in Python.
Creation of time-module values using time.mktime() is much slower than
doing the same thing with DateTime(). The same holds for the reverse
conversion (using time.localtime()).
The storage size of ticks (floats, which the time module uses) is about 1/3
of the size a DateTime instance uses. This is mainly due to the fact that
DateTime instances cache the broken down values for fast access.
To summarize: DateTime[Delta] are faster, but also use more memory than
traditional time-module based techniques
.


Juillet 13, 2009
» Manage monkey patches and performance improvement in Plone 3


Few days ago Jean-Michel François proposed a useful patch for PlonePAS that can be applied for all Plone release until 3.2. Plone 3.3 will embed this patch.

How can we add this patch in a traceable way for an not so old Plone 3.1 or 3.2 ?

First, we can use the new release  of Products.PlonePAS that should be compatible with our Plone installation. The second option is to add a monkey patch in the policy product of our site. One more monkey patch…

Some projects have so many monkey patches that it is difficult to know where is the code that run your site. Martin Aspeli did a tool to handle monkey patches in an elegant way for Zope 2 and Zope 3: collective.monkeypatcher. It allows you to plug  your monkey patches with a simple ZCML directive. Later Gilles Lenfant added a control panel for Zope 2 to be able to have a visual way to follow patches with collective.monkeypatcherpanel.

How does it works?

In your buildout.cfg add :

eggs +=
   collective.monkeypatcher
   collective.monkeypatcherpanel
zcml +=
   collective.monkeypatcher
   collective.monkeypatcherpanel

To create patches add a ‘patches.py‘ file in your egg (if you have more than 2 or 3 patches you should create a directory). Our performance patch looks like this:

import copy

def enumerateUsers( self
                  , id=None
                  , login=None
                  , exact_match=False
                  , **kw
                  ):

    """ See IUserEnumerationPlugin.
    """
    plugin_id = self.getId()

    criteria=copy.copy(kw)
    if id is not None:
        criteria["id"]=id
    if login is not None:
        criteria["login"]=login

    if not kw and id:
        data = self._storage.get(id, None)
        if data is None:
            user_info = []
        else:
            user_info=[ { 'id' : self.prefix + id,
                     'login' : id,
                     'title' : data.get('fullname', id),
                     'description' : data.get('fullname', id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } ]
    else:
        users=[ (user,data) for (user,data) in self._storage.items()
                    if self.testMemberData(data, criteria, exact_match)]

         user_info=[ { 'id' : self.prefix + user_id,
                     'login' : user_id,
                     'title' : data.get('fullname', user_id),
                     'description' : data.get('fullname', user_id),
                     'email' : data.get('email', ''),
                     'pluginid' : plugin_id } for (user_id, data) in users ]

        return tuple(user_info)

In the configure.zcml of your policy product add an include:

<include file="patches.zcml" />

The file patches.zcml will contain following code:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:monkey="http://namespaces.plone.org/monkey"
    i18n_domain="collective.monkeypatcher">

    <include package="collective.monkeypatcher" file="meta.zcml" />

    <monkey:patch
       original="enumerateUsers"
       replacement=".patches.enumerateUsers"
       docstringWarning="false"
    />

</configure>

Run your buildout, start your site and the patch is applied. You can go in the Zope Control Panel to see how many patches are already compatible with this tool.