Showing posts with label vocabulary. Show all posts
Showing posts with label vocabulary. Show all posts

Monday, April 23, 2012

Form in Plone: a simple approach using collective.wtforms

If you follow the Plone-Developers mailing list, you probably already know about a recent thread called "Rewrite old cpt forms to new technology like z3cform". If not: let simply me say that it's talking about removing old Plone stuff, replacing it with the new-way of doing form in Plone: z3c.form.

Although this is a very interesting discussion (that can also help you understand how things works inside the community) the argument of my article came from a single comment of Nathan Van Gheem, that introduced to me collective.wtforms.
This is a Plone integration for a Python, framework independent, library that generate forms: WTForms.

WTForms in general
Using WTForms in Python seems really easy, as introduced in the Getting started section of the documentation. The concept behind are the same we already know from Zope and Plone libraries:
  • a schema definition
  • a set of field types
  • a set of widgets
As it is a Python only framework we don't find ZCA around us.

Using collective.wtforms
The collective.wtforms package is simple. The Plone integration seems a simple work. It only gives you a base WTFormView class (a Zope 3 view that easily integrated you form in the Plone layout) and a WTFormControlPanelView class (if you ever need a Plone control panel form).

That's it.

Inside the view definition you must then use the basical WTForm features. Let's see an example:
from wtforms import Form
from wtforms import TextField
from wtforms import validators
from collective.wtforms.views import WTFormView

class Form1(Form):
    one = TextField("Field One", [validators.required()])
    two = TextField("Field Two")
    three = TextField("Field Three")

class Form1View(WTFormView):
    formClass = Form1
    buttons = ('Create', _(u'Cancel'))
    #label = _(u'Form 1')

    def submit(self, button):
        if button == 'Create' and self.validate():
            # do fun stuff here
            self.context.value = self.form.one.data
Then you need a zcml registration:
  <browser:page
      name="form1"
      for="*"
      class=".forms.Form1View"
      permission="zope2.View"
  />
You can find the example above, and other discussed later, in the package example.wtforms.
Problems
Of course, as always, is simple making simple things.
I found mainly two problems: the widget layout and internationalization.

WTForms widget layout
The template definition done in collective.wtforms is enough to display a form in the Plone way, however when displaying the "real widget" code we are using the WTForms core features. In that case we sometimes see some strange HTML (I mean: strange for Plone users).

One example: when using RadioField fields, the form radio set in wrapped in a UL/LI HTML structure.

This is not a big problem, just I want to say that this is uncommon in Plone forms.
Obviously WTForms can be extended and supports custom widgets.

A bigger task: I18N
A bigger problem was internationalization. The current alpha version of collective.wtforms (1.0a3) doesn't support internationalization of the UI, however fixing this is simple (you can find my changes in a fork of the original project)

With small changes you can see a fully translated of:
  • form title
  • form general description
  • submit buttons
  • fields label
  • fields description
The main problem is that WTForms doesn't support any internationalization.

Recently they added a new i18n module that helps users to translate the internal label (like: the error message after you didn't provided a required field). However this is not usable out of the box in Plone, because Plone translation mechanism is not the basic Python ones.
I tested it adding an italian translation to WTForms (it was missing in the core, so I also provided it to authors and they quickly integrate it. Man: I really love open source!) and I see no difference.

So what I did is to integrate the native ".pot" translation file into the Plone environment and leave this translations to the Zope Page Template engine... and obviously it worked!

Then: I needed some other simple fixes (like: we can't directly render the WTForms label, but we need to use manually render it using TAL).

Again: the the fork for see some code.

Vocabularies
Translating the vocabulary labels for select, multiselect and radio fields was not so simple. WTForms simply want an iterable argument named choices.

What I was forced to do (better patterns are welcome) is to provide a VocabularyWrapper class where vocabulary labels are translated accessing directly the translation machinery.

Conclusions
I'm sure that we can find also other form library outside Zope (Deform can be another valid choice and also YAFOWIL), however I find the use of WTForms really simple and easy to learn.

Saturday, July 16, 2011

Hidden (great) secrets inside ATVocabularyManager

ATVocabularyManager is a well-know Plone product developed by BlueDynamics that make simple handling vocabulary values used by your contents directly inside Plone.
That's all: the power of the product is all in this first sentence.

Generic Setup Strike Back
In my personal experience I not commonly found projects that need a Plone user able to change vocabularies, but in the information architecture this is quite common (they call this "Controlled Vocabulary"). So I used ATVocabularyManager a couple of times in the past, but I never became habit to rely on it.

Another thing I don't liked was the unexistent Generic Setup integration.

Recently a customer asked us a new project with some new content types, with a lot of field with controlled vocabularies (with many values inside). Also he explicitly ask to be able to handle and change it in the future.

So I looked back to ATVocabularyManager, hoping that during this time something changed.

A New Hope
This time I note immediately that latest releases (1.5 branch for Plone 3.3, and 1.6 for Plone 4) give us something new.

First of all: for the first time I understand that the name prefix "AT" means obviously "Archetypes", but you can think it as "the way of controlling vocabulary is done using some archetypes contents". This mean that you can use it only if your vocabularies are inside archetypes content types? No!
You can also use it to handle whatever ZCML vocabulary you need (portlet? Dexterity?)

The other thing I found is that now we have Generic Setup integration. Great!

The Generic Setup integration
Right now the integration is not fully complete. Seems that export step is not there, but looking at the code I saw that the import step code (the most important!) is available. The product right now suffer only some missing of documentation.

How the import steps works? Instead of creating new vocabulary content types ("Simple Vocabulary", "Sorted Simple Vocabulary", ...), it is based on the "IMS VDEX Vocabulary File" content.

At first glance this can seem the less user friendly way and most obscure content type (and probably this is true) but going back to information architecture this is probably the best choice, because it's based on an XML international standard for handle vocabularies: the IMS VDEX.
Another good news: this format also support i18n (and also Plone)!

What I needed after this is simple: provide a VDEX compatible XML file. How? Let show a complete example.

How to add the GS support
First of all you need to provide a "vocabularies.xml file" to your profile directory.
The format of the file is as follow:

<?xml version="1.0"?>
<object name="portal_vocabularies" meta_type="ATVocabularyManager">
<object name="test.vdex" /> 
...
</object>

So you need to provide a reference to a vocabulary file for every vocabulary you need to import (.vdex of .xml file extensions are valid ones).

Where to put all vocabulary files? You need also to put at the same level a "vocabularies" directory. Inside this you simply need to put all files.

Now I will show the file format of our test.vdex file.

<vdex xmlns="http://www.imsglobal.org/xsd/imsvdex_v1p0"
orderSignificant="true">
  <vocabIdentifier>test-vocab</vocabIdentifier>
  <vocabName>
    <langstring language="en">A test vocabulary</langstring>
    <langstring language="it">Un vocabolario di test</langstring>
  </vocabName>
  <term>
    <termIdentifier>aaa</termIdentifier>
    <caption>
      <langstring language="en">A value</langstring>
      <langstring language="it">Un valore</langstring>
    </caption>
  </term>
  <term>
    <termIdentifier>bbb</termIdentifier>
    <caption>
      <langstring language="en">Another value</langstring>
      <langstring language="it">Un altro valore</langstring>
    </caption>
  </term>
</vdex>

That's all. We created a vocabulary with two entry inside (foo values are "aaa" and "bbb"). As say above, handle this directly from Plone is not very comfortable (simpler vocabulary type are easier to understand) however this is great for Generic Setup install step!

Internationalization note
A little bug on this approach. Seems that even if you plan to not provide an internationalization for you vocabularies (for example: you only need to provide your italian, spanish or something other translation) you still need to provide also the english one, or the vocabulary content inside ATVocabularyManager control panel will not show you the right title of the vocabulary (something like "unnamed vocabulary" instead of "Un vocabolario di test").
But you can use a trick and duplicate your locale specific translation also for english. Also, put english translation first.

Not very comfortable right now
If you still think it, you are right. Maybe that vdex is a well know standard for handle vocabulary, but build a vocabulary with this XML format can be not very simple.

For example, the customer give us a document (a MS Word attachment, obviously) with a set of lists of values. The easy way is to put all this in some CSV files, where columns are "italian translation" and "english translation".
But after that we need to convert this in the vdex format. How?

What I did is too look on the cheeseshop for a library that can convert a CSV in a VDEX file. What I find is vdexcsv!
Two funny thing about it:
  • It was released something like two hour before I performed that search!
  • The company behind this product is again BlueDynamics!
What this product does? You only need to easy_install it then you will be gifted with a new bash command: csv2vdex.

The documentation is clear: you need to provide a CSV file, some parameter, and you'll obtain your vdex file.

For the example above I used a CSV like this:

"key";"english";"italian"
"aaa";"A value";"Un valore"
"bbb";"Another value";"Un altro valore"

This was named test.csv.

Then I called the script in this way:
csv2vdex test-vocab 'A test vocabulary,Un vocabolario di test' test.csv test.vdex --languages en,it --startrow 1

One last step: the generated vdex file is perfect for all but the root node name. As in the example above you need to have the root node called vdex, but the script generate it as vocabulary. However after contacting Jens (the product creator, that also help me to reach this results) he was in agreement to change this in future releases of vdexcsv.
For now, simply rename the node!
EDIT (2011-08-22): the good guys released vdexcsv 1.1 that is now fully standard compliant, so no more needs to manually rename the node!