Plone and microdata: the "event" case

After reading some book and good tutorials about  HTML 5, one of the most promising part I found is the HTML 5 extendibility. In one word: one step towards semantic Web.

Microdata introduction (very very quickly)

I don't want to put there examples of how use microdata with HTML 5: the Web is full of simple and interesting examples.
Let simply describe in words what you can do: you will continue writing your pages normally, for example a page where you put information about a person, then using some additional HTML attributes you can mark some information into the page saying "Ehi there! This DIV is the name of the person, this P is it's address..." and so on. You simple need to play with itemscope, itemtype and itemprop attributes.

What is the useful part of this? Integration!
For example: some search engines understand these new syntax and can index this information in a special way. If the page is about a person information, the SERP page from the search engine can display immediately the name and other data, giving a quick preview.

But not only search engines. Scanning a remote web site for finding information about peoples became something that a machine can do.

Right now the suggested set of standards is the one at schema.org.
You can find there a good set of basic category types and subtypes (Person, Book, Movie, Review, Organization, ...).

Take a look there, is really interesting.

What about Plone?

Plone 4.2 is there! It has been released some days ago!
Looking at the changes of this amazing release you can find that it's now using the HTML doctype (it's still using an XML valid template language, so we are now using what is informally called XHTML 5).

What is changed with this? Nothing... HTML 5 make you able to use a lot of cool stuff, but an HTML 4 (XHTML 1.0) code can be still a valid HTML 5 code. You are not forced to do cool stuff.
Real changes will start now; future release of addons, or new Plone releases, can start using news features. Some features are already there but probably you didn't noticed them (for example: Archetypes already support the new placeholder HTML attribute).

Let's go back to microdata format defined at schema.org. When I first read all possible types defined there, I immediately focused on the Event type, and you can understand why: we have an Event content type in Plone.
How can be simple to spill out from it microdata informations?

Getting microdata for Event in Plone

Let me aswer immediatly at the question above: it's very simple. Plone Event content type contains all needed information for provide microdata, so it's only a matter of content view.

I did some test in a new add-on. This product is embarrassing simple: pushing this new feature inside Plone is really matter replacing the Event view, nothing more is needed.

Problem 1: testing the format

Seems that there aren't a lot of testing framework for you microdata. Google is the provider of the main one: the Rich Snippet Testing Tool, but is not so simple and clear to use.
Note: the Google SERP page is supporting the event entity but note that providing the Event microdata will not ensure you that Google will use it.

Testing a Plone Event content with collective.microdata.event with the RNTT gives me some feedback of page validity, but also a lot of warning about missing information.

Also: Google documentation seems not really updated right now. Their example still use other microdata formats while a warning message explicitly suggest you to use schema.org.

Another tool commonly suggested is a JavaScript based ones: Microdata Tool.

Problem 2: changes at Plone templates

Changes at Plone code are simple: the new event_view template provided can be a simple copy of the one from CMFPlone.
There's only a single change that I did and I didn't like and this is related to the position where to put the itemscope and itemtype attributes.

Starting from Plone 4, the "best" way to create views for Plone content type is the one that use content-core slot of the main template. This change simplyfied content's view because common fields, like title and description, has been moved away from views: we no more need to copy the same code in every template.
However the old-style way used in Plone 3 (filling the main slot) is still available.

What is the problem? The right way there is to to put itemscope and itemtype in the container element that contains all event data, but right now the main event container is outside the event_view.pt template.
So: the new view provided with collective.media.event is gone back to the Plone-3-style, using a template that fill the main slot.

I don't like this. What is missing there is a way of being able to mark a content in some way, and let Plone extract the itemtype from there.
This is probably a simple change to be done to the Plone core (probably getting the itemtype of contents using an adapter can be enough).

Next step(s)

Let's sum what other features are missing to Plone for being able to make contents use microdata.

• mark events also in folders views, collections... (know that, in a page, you can provide more than one event)
• do not forget the new plone.app.event!
• a way to provide other formats from schema.org to other content types

Form in Plone: a simple approach using collective.wtforms

If you follow the Plone-Developers mailing list, you probably already know about a recent thread called "Rewrite old cpt forms to new technology like z3cform". If not: let simply me say that it's talking about removing old Plone stuff, replacing it with the new-way of doing form in Plone: z3c.form.

Although this is a very interesting discussion (that can also help you understand how things works inside the community) the argument of my article came from a single comment of Nathan Van Gheem, that introduced to me collective.wtforms.
This is a Plone integration for a Python, framework independent, library that generate forms: WTForms.

WTForms in general
Using WTForms in Python seems really easy, as introduced in the Getting started section of the documentation. The concept behind are the same we already know from Zope and Plone libraries:

• a schema definition
• a set of field types
• a set of widgets

As it is a Python only framework we don't find ZCA around us.

Using collective.wtforms
The collective.wtforms package is simple. The Plone integration seems a simple work. It only gives you a base WTFormView class (a Zope 3 view that easily integrated you form in the Plone layout) and a WTFormControlPanelView class (if you ever need a Plone control panel form).

That's it.

Inside the view definition you must then use the basical WTForm features. Let's see an example:

from wtforms import Form
from wtforms import TextField
from wtforms import validators
from collective.wtforms.views import WTFormView

class Form1(Form):
    one = TextField("Field One", [validators.required()])
    two = TextField("Field Two")
    three = TextField("Field Three")

class Form1View(WTFormView):
    formClass = Form1
    buttons = ('Create', _(u'Cancel'))
    #label = _(u'Form 1')

    def submit(self, button):
        if button == 'Create' and self.validate():
            # do fun stuff here
            self.context.value = self.form.one.data

Then you need a zcml registration:

  <browser:page
      name="form1"
      for="*"
      class=".forms.Form1View"
      permission="zope2.View"
  />

You can find the example above, and other discussed later, in the package example.wtforms.
Problems
Of course, as always, is simple making simple things.
I found mainly two problems: the widget layout and internationalization.

WTForms widget layout
The template definition done in collective.wtforms is enough to display a form in the Plone way, however when displaying the "real widget" code we are using the WTForms core features. In that case we sometimes see some strange HTML (I mean: strange for Plone users).

One example: when using RadioField fields, the form radio set in wrapped in a UL/LI HTML structure.

This is not a big problem, just I want to say that this is uncommon in Plone forms.
Obviously WTForms can be extended and supports custom widgets.

A bigger task: I18N
A bigger problem was internationalization. The current alpha version of collective.wtforms (1.0a3) doesn't support internationalization of the UI, however fixing this is simple (you can find my changes in a fork of the original project)

With small changes you can see a fully translated of:

• form title
• form general description
• submit buttons
• fields label
• fields description

The main problem is that WTForms doesn't support any internationalization.

Recently they added a new i18n module that helps users to translate the internal label (like: the error message after you didn't provided a required field). However this is not usable out of the box in Plone, because Plone translation mechanism is not the basic Python ones.
I tested it adding an italian translation to WTForms (it was missing in the core, so I also provided it to authors and they quickly integrate it. Man: I really love open source!) and I see no difference.

So what I did is to integrate the native ".pot" translation file into the Plone environment and leave this translations to the Zope Page Template engine... and obviously it worked!

Then: I needed some other simple fixes (like: we can't directly render the WTForms label, but we need to use manually render it using TAL).

Again: the the fork for see some code.

Vocabularies
Translating the vocabulary labels for select, multiselect and radio fields was not so simple. WTForms simply want an iterable argument named choices.

What I was forced to do (better patterns are welcome) is to provide a VocabularyWrapper class where vocabulary labels are translated accessing directly the translation machinery.

Conclusions
I'm sure that we can find also other form library outside Zope (Deform can be another valid choice and also YAFOWIL), however I find the use of WTForms really simple and easy to learn.

Tabindex: handle focus and JavaScript events on every elements

My last reading (still in progress) is a book about JavaScript: "JavaScript Cookbook" (Shelley Powers - O'Reilly). It's full of very interesting chapters (probably it will feed many future article here).

Today I will talk about a JavaScript behavior learned in one of the examples found in the book, related to JavaScript events inside Web pages.

Types of usable event
Using pure JavaScript, not all existing events can be used on every page elements. For example: one of the first things I liked when I started using jQuery year ago was the possibility to use onclick event for whatever DOM page element, also in Internet Explorer (that commonly dosn't allow this event to be fired for all types of element).

The example I found in the book talks about another event: onkeypress.
What kind of elements can react to this kind of event? Commonly only elements able to take the focus (like links, but more common are form elements like textarea of input).
The example shows how using another uncommon JavaScript attribute we can change this behavior, enabling all other page elements. I'm talking of tabindex attribute.

How to use tabindex
As I'm mainly a Plone programmer, when tabindex was removed from all templates of the CMS (don't remember exactly, but I think it was when Plone 3.0 was released) I was happy. Why? Because tabindex is commonly not needed in a well-designed Web page.

The tabindex attribute define the order in which a user navigate the page using the keyboard (the TAB /SHIFT+TAB key) but if you define the order of your DOM elements in a logical manner you will simply not need this attribute. However note that it is not deprecated:

• Some non-common pages can need this attribute to force the navigation first to a page section that isn't at the top (to be honest, I can't find a simple example of this)
• A value of tabindex of "-1" make an element, commonly navigable using keyboard, not accessible anymore.

See this good tabindex reference for more details.

Now the great tabindex magic I learned: when tabindex is used on a DOM element (with a value of 0 or more), it magically gain the power of obtain focus.

Let see some examples.
Please note:

• I tested examples with Firefox, Opera, Chrome and Safari, on MacOS. With older Internet Explorer versions you probably need some fixes in event handling.
• If you are on MacOS, use Chrome because the keyboard navigation with TAB is a mess. See below for more on this.

The first example is a simple HTML page with 2 links, one paragraph, and where onclick and onkeypress on those elements raise simply an alert message (the text inside the node).

<html>
<head>
<script type="text/javascript">
<!--
window.onload=function() {

    var handleAction = function(event) {
        alert(this.innerHTML);
    }

    var p = document.getElementById('foo');
    p.onclick = p.onkeypress = handleAction;

    var links = document.getElementsByTagName('a');
    for (var i=0;i<links.length;i++) {
        links[i].onclick = links[i].onkeypress = handleAction;
    }

}
//-->
</script>
</head>

<body>
    <a href="javascript:;">Link 1</a>
    <p id="foo">
        Test here!
    </p>
    <a href="javascript:;">Link 2</a>
</body>
</html>

First of all the mouse event: you will be able to click on both links and the paragraph (not sure of this in all IE versions as I'm not using jQuery there).

Now let's try the keyboard navigation. Using the TAB you will be able to move only onto link elements. When one element get the focus, clicking a key you will get the same message you get for a mouse click.

If you look at the code you'll see that I'm trying to register the onkeypress event also on the HTML paragraph, but it is ignored.
This is right: a world where every DOM element can get the focus will make keyboard navigation painful. But: how can I do if I really need that a single DOM element (commonly not focusable) could take focus and keyboard events?

This is where tabindex help. If you look at the second example you will see that the only difference is the use of tabindex on the P node.

<html>
<head>
<script type="text/javascript">
<!--
window.onload=function() {

    var handleAction = function(event) {
        alert(this.innerHTML);
    }

    var p = document.getElementById('foo');
    p.onclick = p.onkeypress = handleAction;

    var links = document.getElementsByTagName('a');
    for (var i=0;i<links.length;i++) {
        links[i].onclick = links[i].onkeypress = handleAction;
    }

}
//-->
</script>
</head>

<body>
    <a href="javascript:;">Link 1</a>
    <p id="foo" tabindex="0">
        Test here!
    </p>
    <a href="javascript:;">Link 2</a>
</body>
</html>

However, the behavior difference is big.
You can try again the page with keyboard navigation with all browsers used before (again: I suggest to use Chrome if on MacOS).

Now you are able to give the focus to the paragraph! Also, keyboard event is now raised on the paragraph. You probably get the point...

Tabindex attribute can make focusable whatever element of the page and you are able to use keyboard events on them.

Isn't this great? I think that could be helpful sometimes.

Browsers differences (AKA: MacOS hate TAB)
Let's change the article focus.
Testing example one and two with all defined browsers shown some different behavior.
The tabindex definition says that element with tabindex are the first that take the focus (from the lower value to the upper), keeping the DOM order when values are equals; only then, all other focus capable elements are traversed (still, using the top-down DOM order).

For a reason I never deepen, on MacOS (but workarounds can be found on the Web) only form elements seems able to take focus (links not). Also with Firefox, that on other Linux/Windows environment works like a charm, I can't put the focus on links.
The only tested browser on MacOS that works as expected is Chrome, however also Chrome does something unexpected: if you test the second example you will see that the DOM order are kept (while the tabindexed ones must be the first).

But example 2 works also on other browsers on MacOS: you are able to put the focus using TAB key onto the paragraph.
So you can think "I can try to put a tabindex on every DOM page element I want to be accessible using keyboard to fix this MacOS problem".

If you test example 3 (that put "tabindex=0" also on A elements) you'll that this is false. Still, links are not accessible using keyboard. Strange...

5 minutes with tal:attributes

Let's take a quick read about tal:attributes, an important Template Attribute Language statement of the Zope and Plone development.
About static attributes
You already know that when using tal:attributes you can mix dynamic attributes with static ones:

<h1 class="main"
    tal:attributes="id python:'main-%s' % (25+25)">
Hello World
</h1>

...that produce...

<h1 class="main" id="main-50">
Hello World
</h1>

Obviously, when you use the same attributes in the static and dynamic part, the dynamic ones take precedence:

<h1 class="main"
    id="main-999"
    tal:attributes="id python:'main-%s' % (25+25)">
Hello Worldd
</h1>

...that produce...

<h1 class="main" id="main-50">
Hello World
</h1>

When I learned this behavior, years ago, I started as attitude to remove all static attributes that later would be overridden by dynamic ones (like in the first example). Why? Because in this way is impossible for a developer to do something stupid like don't realize that the attribute is dynamic and not static.

Today I changed my idea because when you begin to look at someone else code, you will like documentation.

Sometimes when you look at a page template made by someone else you don't need (want) to understand the meaning of every single attribute, but a general idea can be enough:

<a href="http://myhost/site/folder/document-to-be-updated"
   tal:attributes="href view/getSomething">
Update me!
</a>

In the example above, knowing that "view/getSomething" return a valid URL can be understood because the value is used for an href attribute, but what kind of URL? Is an internal URL? External ones? An URL to a specific view?
To know this we need to investigate the getSomething code, or test the application.
This can be avoided if the original template designer documented an example URL using a static href URL that document more or less the general meaning, exactly like done above.

Playing with attributes order
Let's look at this TAL code that generate XML:

<foo attb="value2"
     tal:attributes="attc string:this is C;
                     atta string:this is A;" />

... that produce...

<foo attb="value2"
     attc="this is C"
     atta="this is A" />

When mixing static and dynamic attributes, static ones are rendered first keeping the original order, then all dynamic ones again keeping the order.

In the example above I'd like to see three attributes in the right logic order, but two of them are dynamic:

<foo atta="value1"
     attb="value2"
     attc="value3"
     tal:attributes="attc string:this is C;
                     atta string:this is A;" />

... that produce...

<foo atta="this is A"
     attb="value2"
     attc="this is C" />

So with static attributes you can control also order of dynamic ones.

OK, this is not really interesting (commonly who cares of order of attributes inside generated code?) but sometimes you need to controls exactly the generated code. For example, when you wrote functional Python tests, then you check output.
Omissis
Another important thing to learn is omit an attribute dynamically, based on expression. Yes, because with tal:attributes you can also want to control if an attribute must be added to a node or not.
This is very popular when you want to work with an HTML select element or checkbox:

<select tal:define="values python:[1,2,3,4]">
    <option name="foo:list" value="1"
            tal:repeat="val values"
            tal:attributes="value val;
                            selected python:val==2 and True or False" >
        <span tal:replace="val" /></option>
</select>

... that produce ...

<select>
    <option name="foo:list" value="1">
        1</option>
    <option name="foo:list" value="2" selected="selected">
        2</option>
    <option name="foo:list" value="3">
        3</option>
    <option name="foo:list" value="4">
        4</option>
</select>

As you can note, we simply return True or False, without specifying what kind of value we want for the attribute. This is the way for obtaining the standard XHTML value equals to the attribute name.

Another example is for attributes where we want to controls also the attribute value:

<div tal:define="foo python:2">
    <span tal:attributes="class python:foo==1 and 'fooClass' or None">First</span>
    <span tal:attributes="class python:foo==2 and 'fooClass' or None">Second</span>
</div>

... that produce ...

<div>
    <span>First</span>
    <span class="fooClass">Second</span>
</div>

There we are also controlling the class attribute value: we need to specify its value or return None to omit the attribute.
Using static default
The last tip! I learned this one recently, looking at 3rd party code.

Sometimes you put in the static node attribute the very-common value of an attribute, for example a class:

<ul>
    <li class="veryCommonClass1 veryCommonClass2"
        tal:repeat="foo python:[1,2,3]"
        tal:attributes="class python: foo==2 and 'veryCommonClass1 veryCommonClass2 selected' or 'veryCommonClass1 veryCommonClass2'"
        tal:content="foo" />
</ul>

... that produce ...

<ul>
    <li class="veryCommonClass1 veryCommonClass2">1</li>
    <li class="veryCommonClass1 veryCommonClass2 selected">2</li>
    <li class="veryCommonClass1 veryCommonClass2">3</li>
</ul>

In the example above we used a real static HTML attribute for our nodes (and this is good, also for documentation) but it's annoying to be forced to repeat the whole default value also in the tal:attributes expression.

This can be simplified a bit, using default!

<ul>
    <li class="veryCommonClass1 veryCommonClass2"
        tal:repeat="foo python:[1,2,3]"
        tal:attributes="class python: foo==2 and 'veryCommonClass1 veryCommonClass2 selected' or default"
        tal:content="foo" />
</ul>

... that produce ... the same output!

Seems that the variable default can be used in the tal:attributes expression to get the static value we used.
Unluckily default is not a string but a Python object, so we can't use it also for compose the class value when we need to add also the "selected" class: trying in the expression a default + 'selected' will raise an error.

Thats all! TAL can be a language sometimes tricky! Hopefully we will use all Chameleon in the near future and things will be a little cleaner!