Release products for Plone: some bad and good attitude

Time passed from age when plone.org was the only place where a Plone user was able to find and download Plone products, and also where a Plone product author (starting from now: a releaser) could publish and distribute his own projects using directly the PloneSoftwareCenter installation.

Plone 3.1 arrived, and the Plone products begin to be found as Python eggs. This was great! Also with buildout adoption, this new way to make and release products change our world.

In facts this wasn't a real "new" way to do Python stuff. Python world already use eggs, simply Plone update its status.

Automation make release process faster
So, Plone products became Python eggs? Well... they can now be released also on the cheeseshop. This (informally know as pypi) is the main Python collector for eggs in the world, and more important the process to release additional eggs on the pypi repository is simple and fast... a lot faster that use the TTW procedure from the Products section on plone.org.

You need simply to type:

    python setup.py register sdist upload

...and the current egg will be registered and released. Very simple!

Plone.org as a egg collector
What about Plone.org? Modern Plone advanced users know that you can look for Plone add-ons onto the cheeseshop, but the great part of users community (and new also users that wanna try the CMS) will user still plone.org Products area.

Again using PloneSoftwareCenter, the plone.org became a pypi server for Plone stuff. Fantastic! Amazing! Releasing a product there became as fast as on pypi!

    python setup.py register sdist upload -r plone.org

This will work starting from Python 2.6, but there's a Python module for being able to use this also on Python 2.4 (see collective.dist).

All those great pieces of information are taken from the tutorial "How to upload your package to Plone.org". Every Plone releaser must read this at least one time.

So, where are the bad news?
The main problem there is in the last sentence of previous part. I fear that:

• not all of the Plone releaser know this tutorial, or
  
• they ignore it, or
• they think is not so important.

To be honest, skipping the "release on Plone.org" part don't give problem to developers: the software is still on pypi and so installing it on customer servers is possible. Also, pypi is the first choice for downloading eggs with buildout (plone.org is used only as mirror, in the case that pypi is not responding).

So why bother about continuing to perform Plone.org releases?

Because I'm pretty sure that we need to always think about new Plone users and non-developers users.

Some days ago I was creating material for a Plone training (for Plone administrators, not Plone developers) and I wanted to insert a reference to a specific Plone product (a new portlet). I was not sure of the project name so searched for it.
My "developer attitude" made me go onto pypi.python.org, but then said: "This is not good and simple to understand for non-developer... let use plone.org, I need to teach to those people how performing such tasks".
Epilogue: the portlet product is not registered on Plone.org.

This give me the idea for this article. Plone newcomers will never find this portlet! They only know that there is a Plone website (plone.org) with a Product addons section (plone.org/products). What the hell is "pypi"?

You can test this every day: try to monitor how often a new Plone product release is registered on pypi, simply looking at its homepage... I think that every day no less that 10, maybe 15 new releases are done.

Now try to find how many new release are putted on Plone.org products section... you know the answer.

So: this his an invitation to Plone developers to register they work also on Plone.org, and read the tutorial given above. This is important for the community.

Maybe the tutorial above needs to be highlight someway on the Plone website.

Any other bad attitude?
Taking the release process also on Plone.org as a standard is a great goal, but we can do better.

Let's not imagine that a collective.foo project has been successfully released on pypi and plone.org. PloneSoftwareCenter is a fully ready-to-use pypi server implementation... but it offer a lot of additional features.
Those features are not so simple and quick to be executed as the release process itself, but they are very important for users.

The problem that I'm going to explain has become a little worst now that the Products section on plone.org is changed, becoming a lot cooler with additional features.

After performing a plone.org egg release, the product description data of our collective.foo is not fully completed. This mean that releasers must go to the plone.org project created and edit it.

EDIT: to perform such link actions on plone.org you will have problems with the current TinyMCE version that the site use. You need to disable TinyMCE while you are editing reST data.
To do this, use Firefox with Web developer toolbar and using the "Disable" menu, disable all Javascript. Enable it again only when finished.

Let's explorer what is not managed from Python release process, starting from the "Default" schemata:

• Categories. This is a very important field for users that want to search for Plone products from plone.org. The Products section has a specific drop down, for example to filter only "portlets" or "content type" products. Sadly if we don't compile this our product can't be found when filtering. This is one of the worst error you can do there.
  
• Contact address. This is compiled directly from the Python egg, but I want to highlight also this field because sometimes the e-mail there is only a default paster template created, that came from the ZopeSkel defaults. Sometime you read "plone-developers@lists.sourceforge.net", but this is only a default, not customized.
• Home page. This field is often compiled with the SVN repository URL... not a very big problem, but the good field for SVN is some lines below this. Also this trivial error came from ZopeSkel default.
• Screenshot. For product that give something to the Plone UI I think is a very good attitude to take 3 minutes and make a screenshot. This help users to understand what you have done, and make plone.org a better website... (let me say "less boring").
• All other fields. All other field in this first schemata are less important, but take you time to think if you have something to put there.
  

Now the "Metadata" schemata:

• The only field there that must be noted is the Classifiers field. You can compile this directly from the Python egg, and is also used on the cheeseshop... so take some time to add all classification that match from the classifier list. This make possible to search categories from pypi website. The most important (but don't limit yourself to it) is the "Framework :: Plone" value. Don't forget this.

Skipping the third schemata, let's go to the "Review" ones.

• Self-Certification Checklist. This is a self-certification, so the developer review himself and his work. The only good attitude there is to be honest... and take 5 seconds to fill this data. Dont' be unpolite!
  

Some last thing to do? Create a documentation section if you have something to give to users.

Finished? Well... no... also the Release content on plone.org need to be checked:

• Release Summary. Compile this with a few lines of text, like "bugfix release", "securify fix to previous version", "added some simple features", "a complete rewrite of previous versions", "first public release", ...
• Full Release Description. A complete text of the new features, not a changes list of features (that can be added below).
• Changelog. Changelog (given there or wherever you think is better) is a great information for your product, and I really like this attitude, used widely in Plone and Zope projects.
• Compatibility. Very important, and now a much more, as plone.org Products section filter automatically projects for the "Plone 4" release.
  You developed a great project for Plone 4? Well... if you don't say that this is Plone 4 compatible filling this field, probably a lot of users will not find your product (they need to change the combo box to "Any version").

All the field above, excepted "Compatibility" that is a very important field, can be skipped and ignored if you write a good README.txt file in you egg (see below) that will automatically fill the main text field of the Product content and also the pypi page.
I find a good attitude take some time to compile those fields, but I can forgive you if skip it!

Is this enough?
Going back to the egg structure, the two most important field (also used in the project "Default" schemata are the description and long-description.

First of all sometimes (not very often) eggs are released without a "description" information, or a stupid ones. For finding those projects Zopyx released the nice zopyx.trashfinder. This is used for the project description on plone.org and also on pypi! Write a good (but also short) summary there!

Another bad attitude is not compile at all, or compile with a few futile lines of text the README.txt file, that will fill the "long description" text of your egg.
Take you time to write there a good README and documentation. Give there all the information you have.
What syntax I need to know and write there? Use reST! So you can write text headers, format text and images! Images are important (so why limit yourself to the plone.org screenshot image?).

   .. image:: http://youhost/youpath/youhelpfulimagename.jpg
:alt: A good alternative image

Deprecated README
When you write a good README, take some time to read it from scratch at every release. It's quite common to find Plone products (also well-know products) that say something wrong, just because a sentence wasn't true again. A set of examples:

• "Compatible with Plone 3.0"... but the last release is only for Plone 4
• "Installing without buildout" instructions.
  I've never truly understand such kind of instructions. Buildout is the only official way to install Plone. You can't install Plone with "easy_install Plone" so why teach to install something other with "easy_install collective.myamazingproduct"?
  More bad: maybe the last release has added the "Plone" dependency in the egg... so one time again, easy_install it system wide will give you only headache.
• ...(I'm sure you can find some other example)...

A README with bad instructions can be worst than don't having a guide at all. Such kind of bad documentation bring users to say "Plone is difficult" or "Plone is undocumented".

Last product version must not be the only version available
Final bad attitude.
When a new release is registered on pypi, it automatically hide the previous versions. If you type (as example) http://pypi.python.org/pypi/Plone you are seeing the last version directly, that is (at the time I'm writing this post) Plone 4.1b1.

I had a discussion about this recently on Product-Developers mailing list: as far as the new product release brake compatibility with previous Plone version (I mean: version that are still maintained) the guy that register the release on pypi must also think about don't hide the previous releases. For what I know, there isn't an automated way to do this, you need to login on the cheeseshop.

A good example? http://pypi.python.org/pypi/Products.Poi
As you can see, the new 2.x (only for Plone 4) is not hiding the 1.2.x branch for Plone 3. Also, documentation is clear. If a user read the page of the 2.x release you can read explicitly "Poi required: Plone 4; this version of Poi will not work with Plone 3".

No more to say, now go and check how good are your Plone releases!