Honey I Blew Up My Plone Product!

This article talk about Plone products but I think this is a very general argument that hit every programming language: the lack of modularity in products.
Know that:

• I'm not talking of the Plone core, that is obviously going in the right direction (the plone.* and plone.app.* universe is one of the clearest example) but only of products add-ons.
• I'm not also talking of bad code (no "Spaghetti code"). Maybe your final result is great and will change the world! I don't care.

Your first time
If you are not a genius, probably the first Plone product you did was crappy. Apart of the features in it, its structure was probably something like a single product called "Products.MyProjectXy".
What this mean is: all Plone features you need in a single product (a real monolith!).
Maybe you were already an experienced programmer (so your product was well designed... inside) but the final result was not the best.

After some times, when you started looking at other product's code (or if you bough the Martin's book), you probably quickly learned that splitting the product is good: the Separation of concerns is really important.

What is the level of modularity you need to reach? In how many different eggs/packages you need to split your project? I've no a real response about it... we have a Third Normal Form about software design? I don't remember too much of my lessons about program engineering...

Thinking modular
Let me say: splitting a Plone project in more than one egg is "boring". Sometimes you have the sinister feeling that you are loosing your time (and I also remember times when ZopeSkel wasn't there to help us) and the temptation to ignore this practice. Maybe a Demon of Bad Software is talking at your soul in that moment.

An example: you are in a very productive state, lines and lines of wonderful code and new features are flourishing under you fingers (Daniel Pink call this the "Flow State")... then you develop something you know it's better to be moved away from the module you are developing because you know that this choice, one day, will pay you back. But to do that, you need to stop your "Python Frenzy" for some minutes.
Don't know you, but this sometimes hurts me. However I know this is good and must be done so (bleeding) I move the code away. Not later! Immediately! Why? Because the path to "Products.MyProjectXy" is easy... when you are there, it's too late!

Every time a customer ask you for a new feature:

• think if this feature can be made more "general purpose"...
• ... or if another customer can like it ...
• ... or if someone else already asked for something similar.

If at least one of those answer is true, probably you must develop something "more general".

Too famous Plone features (sometime more that the product itself)
Some concrete examples.

Let me say that sometimes, when you create a Plone product, you don't know how useful to the community it can became in future. Worst, sometimes a feature inside you product can be useful to other...

... is not always easy to "think at future": the list of projects that follow talk about great Plone products I used a lot of times and the saved my time. So probably done by great developers... however inside them I can still found sometimes the problem I'm described above.

The Kupu link administration
We have all moved away from WYSIWYG Editor Kupu, preferring the famous TinyMCE.
However Kupu still have a useful feature inside it (I still remember the first time I see it at one of Plone Conference... woah!): an administrative tool that can transform your links inside documents from "normal" ones to the "Use resolve UID" form, or get back to the normal form.

Unluckily this feature is shipped with the Kupu product (and: using AJAX stuff, it need that Kupu si really installed to work properly).

The Maps location field and widget
The product Maps give us some simple but direct features of integration with Google Maps. Also, it contains a new type of field and widget: the LocationField (that can contains 2 coordinated values) and the LocationWidget (that help filling the LocationField using Google Maps interface).

This new field/widget pair can be very useful, but this feature is inside the Maps project, not given as external dependency. Also you are forced to install the product in your Plone site if you need it (having it only in your buildout is not enough).

Flowplayer metadata extractions
The well know collective.flowplayer is giving simple Plone Video features to Plone. Instead of simply giving you the video player integration, it contains a simple piece of code that extract video metadata from uploaded files using the 3rd party library hachoir.

We have two problem there: if you need the metadata extraction feature you need to add Flowplayer to your buildout (minor issue: this time you don't need to install it) but also you are forced to use hachoir, that is not a perfect library (first of all: it brakes you pdb!).

Other examples?
I'm sure there are other good products that give you additional features you can like to have also outside the products itself.

Conclusion
Keep you products compact but always think about software reuse. Experience done with your own customers and users can help you to know when a piece of technology need to be moved away and maintained alone. Also: small products are simple to be maintained.

So: don't listen at the Demon that say "Enlarge you Egg".

Tabindex: handle focus and JavaScript events on every elements

My last reading (still in progress) is a book about JavaScript: "JavaScript Cookbook" (Shelley Powers - O'Reilly). It's full of very interesting chapters (probably it will feed many future article here).

Today I will talk about a JavaScript behavior learned in one of the examples found in the book, related to JavaScript events inside Web pages.

Types of usable event
Using pure JavaScript, not all existing events can be used on every page elements. For example: one of the first things I liked when I started using jQuery year ago was the possibility to use onclick event for whatever DOM page element, also in Internet Explorer (that commonly dosn't allow this event to be fired for all types of element).

The example I found in the book talks about another event: onkeypress.
What kind of elements can react to this kind of event? Commonly only elements able to take the focus (like links, but more common are form elements like textarea of input).
The example shows how using another uncommon JavaScript attribute we can change this behavior, enabling all other page elements. I'm talking of tabindex attribute.

How to use tabindex
As I'm mainly a Plone programmer, when tabindex was removed from all templates of the CMS (don't remember exactly, but I think it was when Plone 3.0 was released) I was happy. Why? Because tabindex is commonly not needed in a well-designed Web page.

The tabindex attribute define the order in which a user navigate the page using the keyboard (the TAB /SHIFT+TAB key) but if you define the order of your DOM elements in a logical manner you will simply not need this attribute. However note that it is not deprecated:

• Some non-common pages can need this attribute to force the navigation first to a page section that isn't at the top (to be honest, I can't find a simple example of this)
• A value of tabindex of "-1" make an element, commonly navigable using keyboard, not accessible anymore.

See this good tabindex reference for more details.

Now the great tabindex magic I learned: when tabindex is used on a DOM element (with a value of 0 or more), it magically gain the power of obtain focus.

Let see some examples.
Please note:

• I tested examples with Firefox, Opera, Chrome and Safari, on MacOS. With older Internet Explorer versions you probably need some fixes in event handling.
• If you are on MacOS, use Chrome because the keyboard navigation with TAB is a mess. See below for more on this.

The first example is a simple HTML page with 2 links, one paragraph, and where onclick and onkeypress on those elements raise simply an alert message (the text inside the node).

<html>
<head>
<script type="text/javascript">
<!--
window.onload=function() {

    var handleAction = function(event) {
        alert(this.innerHTML);
    }

    var p = document.getElementById('foo');
    p.onclick = p.onkeypress = handleAction;

    var links = document.getElementsByTagName('a');
    for (var i=0;i<links.length;i++) {
        links[i].onclick = links[i].onkeypress = handleAction;
    }

}
//-->
</script>
</head>

<body>
    <a href="javascript:;">Link 1</a>
    <p id="foo">
        Test here!
    </p>
    <a href="javascript:;">Link 2</a>
</body>
</html>

First of all the mouse event: you will be able to click on both links and the paragraph (not sure of this in all IE versions as I'm not using jQuery there).

Now let's try the keyboard navigation. Using the TAB you will be able to move only onto link elements. When one element get the focus, clicking a key you will get the same message you get for a mouse click.

If you look at the code you'll see that I'm trying to register the onkeypress event also on the HTML paragraph, but it is ignored.
This is right: a world where every DOM element can get the focus will make keyboard navigation painful. But: how can I do if I really need that a single DOM element (commonly not focusable) could take focus and keyboard events?

This is where tabindex help. If you look at the second example you will see that the only difference is the use of tabindex on the P node.

<html>
<head>
<script type="text/javascript">
<!--
window.onload=function() {

    var handleAction = function(event) {
        alert(this.innerHTML);
    }

    var p = document.getElementById('foo');
    p.onclick = p.onkeypress = handleAction;

    var links = document.getElementsByTagName('a');
    for (var i=0;i<links.length;i++) {
        links[i].onclick = links[i].onkeypress = handleAction;
    }

}
//-->
</script>
</head>

<body>
    <a href="javascript:;">Link 1</a>
    <p id="foo" tabindex="0">
        Test here!
    </p>
    <a href="javascript:;">Link 2</a>
</body>
</html>

However, the behavior difference is big.
You can try again the page with keyboard navigation with all browsers used before (again: I suggest to use Chrome if on MacOS).

Now you are able to give the focus to the paragraph! Also, keyboard event is now raised on the paragraph. You probably get the point...

Tabindex attribute can make focusable whatever element of the page and you are able to use keyboard events on them.

Isn't this great? I think that could be helpful sometimes.

Browsers differences (AKA: MacOS hate TAB)
Let's change the article focus.
Testing example one and two with all defined browsers shown some different behavior.
The tabindex definition says that element with tabindex are the first that take the focus (from the lower value to the upper), keeping the DOM order when values are equals; only then, all other focus capable elements are traversed (still, using the top-down DOM order).

For a reason I never deepen, on MacOS (but workarounds can be found on the Web) only form elements seems able to take focus (links not). Also with Firefox, that on other Linux/Windows environment works like a charm, I can't put the focus on links.
The only tested browser on MacOS that works as expected is Chrome, however also Chrome does something unexpected: if you test the second example you will see that the DOM order are kept (while the tabindexed ones must be the first).

But example 2 works also on other browsers on MacOS: you are able to put the focus using TAB key onto the paragraph.
So you can think "I can try to put a tabindex on every DOM page element I want to be accessible using keyboard to fix this MacOS problem".

If you test example 3 (that put "tabindex=0" also on A elements) you'll that this is false. Still, links are not accessible using keyboard. Strange...

Related contents: keeping compatibility with Plone 3

Some days before the Christmas break we found a little compatibility problem with collective.portletpage: when used on Plone 4, and a user add a related item to a Portlet Page content, the "Related content" area were doubled, shown twice.
One in the Plone 3 style (a ul/li structure inside a fieldset element) and another in the new Plone 4 style (that now is a dl/dt+dd code).

The problem? Starting with Plone 4, the related content section has been removed from content view of default content types and has been registered as a viewlet.
This change is great if speaking of design of Web pages. Now a user can move the viewlet in the content view.
Before this (in Plone 3) the related contents area was always at the bottom of the content's view template, called explicitly as a macro... that still exists in Plone 4.

Technically speaking, the new Plone 4 viewlet is registered for every content type, but still the Plone 3.3 related contents macro exists, and contains the same old code (by the way... I think that keeping it as an empty macro could be a better choice).
So: the problem I'm describing is probably quite frequent. It strikes every content type product updated to Plone 4 but were the view has not been updated recently. I meditate about this problem two minutes and I found other two products we own that right now suffer the same problems: Smart Link and RedTurtle Video. We already planned to fix them sooner.

Quick fix
If you don't need anymore to keep Plone 3 compatibility, simply remove the old call to the macro from your content's view. So: take a loot to your content types now!

Less quick (but fair) fix
What if you want to keep Plone 4 compatibility? Removing the macro call make your product no more perfectly working on Plone 3.3: explicit related contents area will be removed from the Plone version where the related item viewlet is not present.

For  Plone newcomers this can seem a waste of time, but for us isn't. Even if Plone 4 is quickly growing, we still have a lot of old Plone 3 customers that ask for new features. I personally don't like too much the "branch way" of keeping compatiblity (continue releasing an "old family" product and a "new generation" ones).

So you can find many workaround for this, but the new Plone 4 feature (a viewlet for related items) in my opinion is also a great enhancement.
For this reason I take some time to develop a very simple product: collective.relateditems.
With this you simply need to update your content view to Plone 4, then add this product to your buildout when you are in a Plone 3 environment (see the documentation to know how to make this dependency automatically fulfilled).

The product simply register a related items viewlet exactly how is done for Plone 4, but keeping Plone 3.3 CSS styles.
What this product also does is to register this viewlet only for content's that implements a new provided interface. So you are the one who choose if display related items viewlet or not (thanks to ZCA, there is a very simple way to make Python classes implementing an interface even without touching 3rd party code).

Conclusion
I have no real conclusion, just a suggestion: go and try your old product, adding to it at least one related item!

5 minutes with tal:attributes

Let's take a quick read about tal:attributes, an important Template Attribute Language statement of the Zope and Plone development.
About static attributes
You already know that when using tal:attributes you can mix dynamic attributes with static ones:

<h1 class="main"
    tal:attributes="id python:'main-%s' % (25+25)">
Hello World
</h1>

...that produce...

<h1 class="main" id="main-50">
Hello World
</h1>

Obviously, when you use the same attributes in the static and dynamic part, the dynamic ones take precedence:

<h1 class="main"
    id="main-999"
    tal:attributes="id python:'main-%s' % (25+25)">
Hello Worldd
</h1>

...that produce...

<h1 class="main" id="main-50">
Hello World
</h1>

When I learned this behavior, years ago, I started as attitude to remove all static attributes that later would be overridden by dynamic ones (like in the first example). Why? Because in this way is impossible for a developer to do something stupid like don't realize that the attribute is dynamic and not static.

Today I changed my idea because when you begin to look at someone else code, you will like documentation.

Sometimes when you look at a page template made by someone else you don't need (want) to understand the meaning of every single attribute, but a general idea can be enough:

<a href="http://myhost/site/folder/document-to-be-updated"
   tal:attributes="href view/getSomething">
Update me!
</a>

In the example above, knowing that "view/getSomething" return a valid URL can be understood because the value is used for an href attribute, but what kind of URL? Is an internal URL? External ones? An URL to a specific view?
To know this we need to investigate the getSomething code, or test the application.
This can be avoided if the original template designer documented an example URL using a static href URL that document more or less the general meaning, exactly like done above.

Playing with attributes order
Let's look at this TAL code that generate XML:

<foo attb="value2"
     tal:attributes="attc string:this is C;
                     atta string:this is A;" />

... that produce...

<foo attb="value2"
     attc="this is C"
     atta="this is A" />

When mixing static and dynamic attributes, static ones are rendered first keeping the original order, then all dynamic ones again keeping the order.

In the example above I'd like to see three attributes in the right logic order, but two of them are dynamic:

<foo atta="value1"
     attb="value2"
     attc="value3"
     tal:attributes="attc string:this is C;
                     atta string:this is A;" />

... that produce...

<foo atta="this is A"
     attb="value2"
     attc="this is C" />

So with static attributes you can control also order of dynamic ones.

OK, this is not really interesting (commonly who cares of order of attributes inside generated code?) but sometimes you need to controls exactly the generated code. For example, when you wrote functional Python tests, then you check output.
Omissis
Another important thing to learn is omit an attribute dynamically, based on expression. Yes, because with tal:attributes you can also want to control if an attribute must be added to a node or not.
This is very popular when you want to work with an HTML select element or checkbox:

<select tal:define="values python:[1,2,3,4]">
    <option name="foo:list" value="1"
            tal:repeat="val values"
            tal:attributes="value val;
                            selected python:val==2 and True or False" >
        <span tal:replace="val" /></option>
</select>

... that produce ...

<select>
    <option name="foo:list" value="1">
        1</option>
    <option name="foo:list" value="2" selected="selected">
        2</option>
    <option name="foo:list" value="3">
        3</option>
    <option name="foo:list" value="4">
        4</option>
</select>

As you can note, we simply return True or False, without specifying what kind of value we want for the attribute. This is the way for obtaining the standard XHTML value equals to the attribute name.

Another example is for attributes where we want to controls also the attribute value:

<div tal:define="foo python:2">
    <span tal:attributes="class python:foo==1 and 'fooClass' or None">First</span>
    <span tal:attributes="class python:foo==2 and 'fooClass' or None">Second</span>
</div>

... that produce ...

<div>
    <span>First</span>
    <span class="fooClass">Second</span>
</div>

There we are also controlling the class attribute value: we need to specify its value or return None to omit the attribute.
Using static default
The last tip! I learned this one recently, looking at 3rd party code.

Sometimes you put in the static node attribute the very-common value of an attribute, for example a class:

<ul>
    <li class="veryCommonClass1 veryCommonClass2"
        tal:repeat="foo python:[1,2,3]"
        tal:attributes="class python: foo==2 and 'veryCommonClass1 veryCommonClass2 selected' or 'veryCommonClass1 veryCommonClass2'"
        tal:content="foo" />
</ul>

... that produce ...

<ul>
    <li class="veryCommonClass1 veryCommonClass2">1</li>
    <li class="veryCommonClass1 veryCommonClass2 selected">2</li>
    <li class="veryCommonClass1 veryCommonClass2">3</li>
</ul>

In the example above we used a real static HTML attribute for our nodes (and this is good, also for documentation) but it's annoying to be forced to repeat the whole default value also in the tal:attributes expression.

This can be simplified a bit, using default!

<ul>
    <li class="veryCommonClass1 veryCommonClass2"
        tal:repeat="foo python:[1,2,3]"
        tal:attributes="class python: foo==2 and 'veryCommonClass1 veryCommonClass2 selected' or default"
        tal:content="foo" />
</ul>

... that produce ... the same output!

Seems that the variable default can be used in the tal:attributes expression to get the static value we used.
Unluckily default is not a string but a Python object, so we can't use it also for compose the class value when we need to add also the "selected" class: trying in the expression a default + 'selected' will raise an error.

Thats all! TAL can be a language sometimes tricky! Hopefully we will use all Chameleon in the near future and things will be a little cleaner!

Debugging a slow site using Apache response time and TinyLogAnalyzer

We met recently a new customer, a company that need help with a slow application composed by:

• a Plone CMS
• a set of external processes (a daemon, an e-mail scan system, ...)
• an complex integration with an external database engine
• an integration with two kind of Visual Basic based client desktop application

With the grow of the company bussines they suffer in years a gradually decay of performance.
Some architectural changes were needed, but the company also asked for some quick fix, to get back a faster application while major changes are planned.

Tune up this application was not simple because the whole system is Windows based, so a lot of well-know software we commonly use are not applicable. The company contacted us because the core of the application seems the Plone CMS and the underlying Zope, but we met a customized version of a very old Plone installation. More or less we were facing something that was not Plone.

We decided to start to look at the code (everyone always hope to find some app-go-slow=True flag activated!), but meanwhile we enabled the response time log on the Apache front-end (and it works also on the Windows version!)

How long does it take to serve a request?
You can change the default Apache HTTP log configuration from something like this:

    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

...to this:

    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T/%D" combined

Suddently you log format change to this:

    [31/Jan/2008:14:19:07 +0000] "GET / HTTP/1.1" 200 7918 "" "..." 0/95491

...adding the time (in seconds and microseconds) needed to handle the request.

Please note that in a complex installation this will measure the total time needed for the request (if you put a reverse-cache proxy and a load balancer behind your Apache, this will raise the total time) but their Windows installation is very simple: Apache and Zope behind.

How to analyze results?
After a month of collected log data in the new format we wanted to see some statistics. The problem now was: how to read this log to learn something?

After some Google search I was a little worried. There are some real-time monitor tool for log analysis, but we are really scared to try them on a production Windows environment.

The only log analyzer found was apache-response-time: the projects description is really promising, however I was not able to use it: I installed it on a Linux and MacOS system but both I get a segmentation fault error.
Neither a stackoverflow question helped me to find something more. 

So I started to write some few Python lines preparing a custom analyzer. This job became interesting, so I spent a pair of week-ends for code refactoring and to make initial foo.py script is something reusable.

The result was TinyLogAnalyzer, a command line HTTP log analyzer that take a log file with response time enabled and extract two kind of statistics: the top total time and the top average time.

The top total time give you an idea of what is the most time consuming part of the application. In out case we found that more than 20% of the total time was spent performing searches and a special call performed by an external daemon.
Some optimization and changes to this code make the customer a little happier.
Also we find that a lot of requests were done for download external JavaScript resources, never cached.

The top average time instead give you an idea of the slower part of the application. With this I found that object creation and traversing in a special part of the application was really slow due to use of basic (not BTree) Plone folders.

All other release of the log analyzer product you can find are bug-fixing and new features still done for the same customer's problem (meanwhile we collected two more months of log).
For example, I found that some of the external tasks run also during the weekend, so collect data during Saturday and Sunday was wrong. The same for the night time.

Conclusion
The script is all but perfect. For example it keep all data in memory (so it's not the good tool for handle gigabytes-long-file) and read the log from the first line to the last (so if you need to skip some months it is still really slow).

However: it helped me a lot, giving some time to us and some breath to the customer, making things going better while major refactoring are planned.

Apart of the script itself, I think that enabling the Apache response time can be a great source of information when things goes wrong.

Plone, security and workflows: learn how to use "Acquire" option

This article is a new episode of my security and workflow common mistake using Plone.

In the first part, "Plone, security and workflows: when rely on Owner role is bad" I introduced a potential workflow problem, that however can be a real problem only in some kind of organizations.

Today I don't want to focus onto roles, but on the use (or missing use) of the "Acquire" check when designing workflows.
Don't know how to use this checkbox can lead to security problem, especially in conjunction with the View permission.

Is also important to be noted that this problem can be inside current workflows distributed using Plone. This depends on how you and your users think that Plone security must work to fill your needs.

What is the "Aquire" meaning?
You can see the "Acquire" check ("Acquire permission settings?") while defining the workflow permissions page, for every permission managed by the workflow.


The simple meaning is "roles that behave this permission are the ones listed there, plus ones defined on the container of this content".

If the parent itself use a workflow and manage same permissions, it can also use Acquire check again, so acquiring other permissions for roles from it's own parent... and so on.
This continue until the Plone site (or sometimes, up again to Zope root object) where all permissions are defined in the "Security" tab.

Use the Acquire settings without flagging additional roles for a permission is ok: the meaning is "this workflow state don't want to manage this permission". Instead, another state of the same workflow will probably add or change something to the permission.
If every state of a workflow simply use the Acquire check without change something, the customization of this permission in the workflow is useless and can probably be removed.

Also: don't manage a permission for a workflow is still ok. You will never find a workflow that handle all permissions. Only manage the ones you care about (so, for workflow for custom content types, please remove the "Change portal events" permission!).

Out of the box security
More than one Plone standard workflows suffer of what we can define "a not perfect configuration". This do not mean that this is a Plone architectural problem: with Plone you can create great, perfectly working and safe, workflows! Just: the provided ones could be not perfect in some situations and for being able to fix them or create new ones, you need to understand why!
Also, if the workflow is ok or not, depends on how you and your customer think that Plone must work.

The problem is commonly associated with the View permission, however all analysis that follow are the same for every other permissions. We will refer to "Simple Publication Workflow", while will be the same for other like "Intranet/Extranet Workflow".
The "Simple Publication Workflow" is used from all Plone default contents, also by folders.

Let's take a look at the published state permissions setting.


Look in detail at the Anonymous column. The View permission is given at the Anonymous role. And this seems good: you are explicitly saying "every content in this state will be visible by the anonymous". Always. Everywhere.

How can be this bad? The content is public, I want that Anonymous see it!

Let think this applied to folders: users (99% of times, in my experience) see the private and the published state like a room with a door:

• A published folder is like a room with an open door. You can see inside the room itself, and other rooms that follow (if those rooms are open)
• A private folder is like a closed door. You can't see inside it, and you don't care if other doors that follow are open or closed. You don't even know if there are other doors!

Users that move from published to private a main folder that contains a deep tree of subfolder and contents, commonly think that this action will hide all the tree!

You probably already get the point. An explicit check of the View permission for a role is like a magic crystal ball. You don't care of the door: you have the power too see inside.

Going back to Plone world: think about having a Plone folder (folder-1) with a document inside (document-1). If the document is published you can see it using your crystal ball. The review state (even if private) of the containing folder doesn't matter.
This is what is not always clear to users.

• Users can reach http://yourcompany.com/
• Users can't reach http://yourcompany.com/folder-1 (he will get Unauthorized)
• Users can reach http://yourcompany.com/folder-1/document-1

During developer training that customers ask us, one of the argument is always how to customize workflows. Sometimes those customers have already installed a Plone site or intranet by their own so when I explain this, is not uncommon to see surprised faces.

O__o

Another bad attitude: customers never tested the workflow deeply and they think that their contents are is a secure review state just because a main folder is private.

How to fix this
As said above: this is not a Plone technological error. There's a way to get exactly what we want: a set of rooms with doors, with no crystal ball, secret passage or teleport bridge that can transport unwanted users inside the room.

The trick is: don't say that the document is visible explicitly, leave this responsibility to the container (to the room's door).

This is traduced in a workflow configuration that remove explicit permission for Anonymous but use the Acquire power.


Note also an important thing: we don't give the check to Anonymous, but we now check permissions for all other roles that matter. We mean there: all roles that must see the content ignoring if the access door is open or closed.

In this way an anonymous user is able to see the content if he can enter the room (published container folder) and will be no more able to see anything when the door is closed (private container folder).

This is really simple!

When not talking of anonymous access
Somewhere above I said that also the "Intranet/Extranet Workflow" suffer the same configuration problem. Can we fix also this?

This kind of workflow is designed for intranets, but also provide some review state for public contents: it add the "internally published" concept to our Plone site.

Note that this workflow is designed to be used with "Intranet Workflow for Folders" associated with folderish content types.

When using those workflows, we are sometimes referring to the Member roles as we did for Anonymous above.

The default configuration of the "internally_published" state is as follow:


The concept behind this state is "having a published state for intranet users" (so excluding Anonymous).
So we have the same problem for the explicit check of the View permission, this time for Member role.

We can fix this in the same way:


We removed the explicit View permission for Member role while enabling the "Acquire".

This time the cure has some limit.

First problem: if we put an internally_published content in the site root, in our "fixed" version of the workflow we are exposing the content to anonymous access, while the original version protects it (know that you will never remove the View permission for Anonymous role from the site root or Plone will work in unpredictable way).

Another problem: take a look at states defined for "Intranet Workflow for Folders". It's composed by two states: private and internal. If we look at the internal description we read "Visible to all intranet users", so it want to be a "published folder for intranet users".

The private state has no View permission for Member role, while the internal state explicitly check it for site members.
As explained above, this mean that if we put an internal folder inside a private folder, site members will be able to skip the private state of the main container folder and access directly the inner ones and all contents inside.

If we fix this removing the explicit check for Member while activating the "Acquire" check we will have a similar problem to the first ones: anonymous users will be able to access the internal folder in the site root...

There is no perfect solution for this problem using only a good workflow configuration. We need to make some default configuration to the site structure, helping the workflow:

• We fix all "internal published like" states of both workflows as described (using "Acquire")
• We put all internal folder in one (or more) main private folder(s) in the site root
• We give to all users (we can also use the "authenticated users" virtual group) the "can read" check (Reader role) from the sharing tab of this private folder
• We remove from all contents/subfolders inside master private folder(s) the roles inheritance from the sharing tab, removing Reader power to all members
• Inner subfolder and contents will live with normal worfklow security settings

With a configuration like this we can be sure that Anonymous users can't access our contents (as they all live in a private folder) while still be able to control the visibility of a subtree changing the main folder review state (relying on "Acquire").

Think about having a dedicated workflow for folders
Designing a workflow can often be design a set of workflows. This is common; the "Intranet Workflow for Folders" is a perfect example. To use it right, it's better to use also the workflow for folders.

I find often useful to create two different workflow for real contents and folders.
Here a common request I get: Reader role must be able to enter inside private area, but inside this being able to only published contents.

Obviously the workflow I will design will be a workflow that rely onto "Acquire".
While the private state for folders will give explicit View permission to Reader, the private state for contents won't give such power. Done!

Conclusion
Before load your new workflow in a production site or intranet, be sure of what workflow you give to contents... and test it.

A simple way for performing those tests is_ search contents using the site search feature.

Plone, security and workflows: when rely on Owner role is bad

Once upon a time Plone was a CMS that:

• was using less roles than today (no Contributor, Reader or Editor and obviously no Site Administrator, the new entry)
• None of few roles available (Manager, Owner, Reviewer and Member) were hidden from the sharing form

Starting from Plone 3 the sharing view were simplified in two ways:

• hiding confusing roles (Member and Owner)
• translating the general meaning of every roles left

In this post I want to talk about the Owner role.

First of all: why it has been hidden from the sharing form? Because this was confusing users a lot.
Who is the Owner of a document? Commonly is a single user, the one you read from the Creator metadata field.

You can manipulate the Owner of a document (you know? If you change the Creator from the edit tab you simply change an information, nothing different as changing the description): you need simply to access http://url/to/document/ownership_form (the link to this page has been removed from the sharing form... I never get why, however it's always there if you need its service).
Also: you can use the very good plone.app.changeownership that also help you to manage the deprecation/removal of a user, transferring ownership of all documents from one to another.

The Dark Side Of The Owner
In a basic Plone installation there's nothing wrong with Owner. It's natural to base part of the document security on the ownership nature and it's right that the document creator still keep special power onto it.
If you use Plone for a open collaboration project this is ok. For example if I publish something on plone.org is good that I can always change it later, or retract it.

You must be warned about using workflows that give special powers to Owner only if the organization using Plone is a bit structured. Follow some examples:

• your Organization is splitted in at least two or more departments and on that division you also create a set of Plone groups. "Department A" can have a personal area where "Department B" can't enter, or simply can't write into.
  What's happen when a user from Department A is moved (let say because get a promotion) to Department B?
• you give temporary special power to a user that isn't really part of your team and for some times he will be able to contribute, creating some documents. Then you revoke this power but you can't simply also remove the users itself

Using Plone default workflows both cases above can leave you security problems.
Take for example the Simple Publication Workflow, current Plone default workflow: when a content is in the published state the Owner can still modify it. Commonly this is not a problem (if it is, you need a different workflow, but Plone is really good for build them quickly) but if you are in a situation like the two described above the real creator will not be the proper user for having this power on the content.

There isn't only a solution to this problem. From now for simplicity we will consider the first example, where the user that is creator of a document has been moved away.

Perfect solution
The perfect solution is to use plone.app.changeownership because in this way you can:

• keep the high level ownership (metadata) of the original user. Commonly this is good, you don't want to change the history and destroy proof that the user one time worked in Department A (but know that with this product you can also do this, like cancel documents creation performed by super user used during site creation).
• give low level ownership (Owner role) to another user that still work in Department A

However the user of this product is not automatically integrated in Plone. Your organization must take care of perform the migration manually every time someone leave one of the departments.
So, even if I name this "the perfect solution", you probably prefer a well Plone workflow integrated ones.

Don't rely on Owner
The other kind of solution is to design and use workflows that don't rely on the Owner role.
Going back to Simple Publication Workflow, you can remove to Owner the power to edit published contents.

If this seems enough for you, remember that this will not cover all cases. For example: the user moved to other department must not be able to edit also document he created that are still in private state (because he didn't published them, or a reviewer retracted).

You can't simply remove all permissions to Owner role. Why not? Because you need to keep special power for the Owner in the workflow initial state.
Look again at Simple Publication Workflow, this time to the private state permissions. Who can edit the content? If you remove administrative roles (Manager and Site Administrator), only Editor and Owner are left. You can't be sure that the user is also in a group that will give him Editor role (the "can edit" from the sharing form)... maybe he is simply a Contributor ("can add").

So, if you remove the Owner power there your user will not be able anymore to create new contents. He will get Unauthorized error when saving for the first time.

Rely on Editor Role
In facts you can still rely only to Editor role, but for what described above this can be only applied to organizations where you don't have the Owner/Contributor/Editor concepts, but simply the "Author" ones, where is not important who create a document, but all the team matters.

In my customer-types experience, we have many organizations like this. It mean that you can create a content, but it isn't yours; it's always of your group (Office, Department, ...). You create it, but your colleague can edit it later, even if it is still in the private state. The Plone history form take track of all changes done.

When your organization works in this way, for you the Editor and Contributor roles collapse because your users can add new content and edit all contents. You commonly have a group that can manage contents on a folder and all users in this group have both roles.
You don't get Unauthorized errors removing to Owner all permissions because the Contributor will be also an Editor so he will be able to save the content.

What's happen when you are moved away? You loose all Editor powers and the Owner roles is giving you nothing special.

Fake initial state
The solution above is OK, but it can't be used if you want to keep the differences between Editor role and Contributor: you could like the fact that a member of the department can add new contents but also that he isn't able to edit other contents, while an editor user can modify all contents.

In this situation you need to go back to the Owner role. But how use it without falling in the security risk and also give to the user the permission to save the content the first time?

The solution can be add a new initial state: let name it "Being created" (stolen from one of the Poi's workflows).

(images above realized using the always great DCWorkflowGraph)

You can leave "Modify portal content" permission to Owner only in this state. The "Post" transition can be "Automatic", instead of "Initiated by user action" so, as soon as the user save the content, it will move in the real initial state: private.

This solution has some limitation: Owner can't be able to edit again the content after saved it (or better: he need also additional roles like Editor).

Fake initial state (lite)
Another solution is very similar to the one above: use again a pre-initial state (let say again "Being created") but leave to the user the choice of move forward. Also, only a Reviewer/Manager power role can move the content back to the pre-initial state, where the Owner can edit it.
The Owner will be able to save it multiple times, the send it, but later can ask permission for edit again. Also: this "Being created" state will be "super private". No other Editor/Reviewer/Reader role will show those contents.

In this way, even if later the user moved to Department B mess up or destroy his old documents, no one will cry. If those document stay there, they will be only a little garbage in your database.

Conclusion
Owner role is a good mate, just know how and when use it!