By Ingeniweb. A Django site.
Avril 29, 2010
» AllowedContentType in Plone2.5

Hello,

I notice that the method allowedContentTypes cost time when you have a lot of type in plone2.5.I don’t know if plone3 or plone4 are impacted of that. I have 120 types and the time of execution of context.allowedContentTypes is about 0.32sec.

The path of allowedContentTypes is :

allowedContentTypes -> portal_types.listTypeInfo -> for each content type: portal_types.isConstructionAllowed -> portal_types._queryFactoryMethod -> Products.Five.pythonproducts.patch_ProductDispatcher__bobo_traverse__ -> Products.Five.pythonproducts.product_packages

and Products.Five.pythonproducts.product_packages time call is 0,003s

When you have 100 content type only product_packages is responsible of 0,3sec , this method is a performance bottleneck.

So add this patch fix the problem :

from Products.Five import pythonproducts
old_product_packages = pythonproducts.product_packages
pythonproducts.product_packages =  forever.memoize(old_product_packages)

Regards Youenn


Avril 7, 2010
» New Release of iw.fss

I’ve just released a new version of iw.fss (2.8rc2 for plone 3) and FileSystemStorage (2.6.3 the same but for plone2.5).

Those releases change the behaviour of getData method which retrieve data from filesystem.
This change is important in case of big file because now all data are handled by filestream_iterator (in all case). So no memory is consumed when we access directly to the data (if you respect this new API)

For the developper point of view , getData works like OFS.Pdata. So to get all data (don’t do that please) just call str(field.get(instance)), and to get the first block of data call field.get(instance).data and you have a pointer to the next block in calling field.get(instance).next.

So if you are using iw.fss for yours projects please update it, our plone will thank you after that !!

Regards Youenn


Août 31, 2009
» How to configure an custom vary tag for squid

You want to make an authenticated cache with apache/squid-varnish/plone and you don’t
know how do that : it’s possible with the vary tag.

Vary header tell to proxy cache what’s headers is variant for an object for a cache.
For example if you tell to the cache that the variant is Cookie , then for a same url with different cookie value the result of the cache is different.
The Server send to the proxy (in the response) which header is considered for vary by sending Vary: list of request header name

In Cachefu, you can configure that by rule with varyExpression.

In global configuration of cache fu you can also configure an global vary header. By default this configuration is send with rule.portal_cache_settings.getVaryHeader()

You can activate or desactivate vary with the header_set configuration ( vary field).

Vary headers must be present in the request (not response) of the browser in order to be considered to be variant for the proxy cache. So we are limited with the standard header of the protocol http.

But with cookie and apache (apache is in front of squid) we can elaborate strategy to construct a vary tag more efficient.

The second aspect of the cache work is purge content when the content change.

PURGE of Vary objects is still very poorly supported in squid, and you can only purge one variant at a time and need to get the URL cached again before being able to purge another variant. So how to deal with that also ?

First , how build our vary tag ?

The trick is to construct an custom vary tag with apache.
We can to do this with RewriteRule::

RewriteCond %{HTTP_COOKIE} mycookie="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=mycookie:%1]

So in this example mycookie contains the value of cookie_key

You can add a cookie for the language , a cookie for group , a cookie for a permission and so on and then construct your custom vary tag with values of this specifics cookies with mod_headers

RequestHeader append MyVary %{mycookie}e

And then the value of mycookie is considered to be variant..

If you want have a specific vary tag for anonymous you can test the presence of
__ac cookie and send a custom MyVary in this case

RewriteCond %{HTTP_COOKIE} __ac="([^"]+) [NC]
RewriteRule ^(.*)$ - [E=authenticated:1]

RequestHeader append MyVary %{mycookie}e env=authenticated
RequestHeader append MyVary anonymous env=!authenticated

So now with that you can vary cache as you want. Now how to treat the big deal of purge.

The trick is  have an image (or a ajax request or ..) in content that is never in cache. This image is serve by a browser view (in case of zope application) that set a cookie. This cookie value is added to Vary tag. So the Vary tag change if the value of this cookie change and then the content is updated (for all request).

For example we can construct a cookie with the value of the catalog change

catalog_count = pcs.getCatalogCount()
context.REQUEST.RESPONSE.appendHeader('Pragma','no-cache')
context.REQUEST.RESPONSE.appendHeader('Cache-control', 'no-cache')
cookie = context.REQUEST.cookies.get('X_CACHE_CATALOG', 0)

if cookie != str(catalog_count) :
context.REQUEST.RESPONSE.setCookie('X_CACHE_CATALOG',
catalog_count ,
path="/")
return catalog_count

And in apache we add
RewriteCond %{HTTP_COOKIE} X_CACHE_CATALOG=([^"]+) [NC]
RewriteRule ^(.*)$ - [E=X_CACHE_CATALOG:%1]
RequestHeader append MyVary %{mycookie}e:%{X_CACHE_CATALOG}e env=authenticated

And when catalog change, the vary also (in the second request) and the cache is updated. You can elaborate other strategies for purging vary object with this technique.

The last point is to combined Etag and Vary Header in response. IE with a Vary header don’t treat correctly Etag header and If-None-Match is never sending. So in apache remove the tag Vary and then Etag work well for all browser

Header unset Vary