JCMS 6.0: principal new features

1. Dual data storage mode

In JCMS 5.7 and earlier version, all data managed by JCMS were stored in JStore. These data included objects manipulated explicitly by users (publications and other content, categories, groups, members, etc.) and also a set of small technical objects (workflow notes, reader tracking, archives, poll votes, etc.). These objects could in certain cases become very numerous (e.g. workflow notes) or generate rather unpredictable data volumes (e.g. poll votes).

Since the JStore persistence mechanism is based on the logging of changes in the file store.xml, the growth of this file depends on the number of entries (creations, modifications and deletions). The size of this file directly affects the JCMS boot delay and a number of functionalities (JSync, history access, etc.). For this reason, and in order to reduce the number of writes, it was not recommended to conserve in JStore frequently updated data such as user preferences, usage information, and statistics.

To handle high data volumes and frequent updates more efficiently, JCMS 6 introduces a dual storage system that now exploits both JStore and a relational database referred to as "JcmsDB". The idea is to choose the storage system most suitable for each type of object. Some new types are managed in JcmsDB; some existing types continue to be managed with JStore, but others have been ported to JcmsDB.

1.1 Hibernate

JCMS 6 uses the Hibernate 3.2 object/relational (ORM) mapping framework to manage the data in JcmsDB. Hibernate offers good performance and its dialects system ensures great independence of the code vis-à-vis the underlying base. It supports the most popular RDBMS systems, although in practice each RDBMS has specificities that must be taken into account (e.g. reserved keywords, table name lengths, index uniqueness, CLOBs management, etc.). JCMS 6 has been certified for several RDBMS systems, which is a guarantee of interoperability.

1.2 Supported RDBMS

JCMS is delivered with the integrated RDBMS Derby which is quite suitable and fully functional for JCMS sites managing reasonable data volumes. However, note that Derby is incompatible with JSync replication.

For sites managing very large data volumes or requiring high availability, JCMS 6 is certified for three external RDBMS products:

• MySQL 5.1 (InnoDB storage engine)
• Oracle 10g
• PostgreSQL 8.3

Use of an external RDBMS necessitates the purchase and installation of the corresponding plugin.

Future JCMS releases will be certified for additional RDBMS systems, notably Microsoft SQL Server.

1.3 Réplication (JSync)

The JSync replication protocol operates only on the data managed in JStore. The internal Derby database cannot be used in a replicated architecture. In this case MySQL or Oracle must be used. Various architectures can be chosen, depending on the topology of the servers and the required availability level.

1.4 Centralized database

The leader and replica servers access concurrently. This architecture is very simple but does not guarantee high database availability.

Fig. 1. Centralized database

1.5 Replicated database

The leader and replicas access replicated instances of the database. This architecture guarantees high database availability and enables flexible topologies (e.g. one instance in a DMZ, another in the LAN). On the other hand, none of the RDBMS systems currently supported by JCMS allow simultaneous writing to the same table. What's more, the synchronization of the instances is relachee and conflicts can occur.

Fig. 2. Replicated database

1.6 Clustered database

The leader and replicas access the cluster concurrently. This architecture guarantees high database availability. It is not always a standard features of commercial RDBMS products, although most of them provide for certain cluster-based solutions.

With JCMS 6, if high database availability is essential, we recommend the cluster solution.

Fig. 3. Clustered database

2. Evolution of the data model

2.1 Homogeneity of the two storage modes

A special effort has been made to ensure great homogeneity of the data model regardless of the storage mode chosen. In the workspaces, contributors use exactly the same interfaces to browse, search, consult, create, modify or delete publications stored in JStore or JcmsDB. Likewise, developers have homogeneous methods in the JCMS API for finding and searching for data, saving them and placing controls, regardless of the storage mode.

This is not to say that there are no differences between data managed in JStore and those managed in JcmsDB. For example, fewer functionalities are provided for JcmsDB data than for JStore data; on the other hand, JcmsDB data can be much more voluminous and can be updated very frequently.

2.2 Data types managed in the database

The following internal types are now managed in the database:

• Workflow Note (WFNote)
• Reader list (ReaderTacker)
• Archive (ArchivedPublication)
• Extra Data (ExtraDBData)
• Review

Plugins can also manage their data in the database, notably in the following cases:

• DBForum Plugin
  • DBForumTopic, DBForumPost and DBForumWatcher types
    
• Poll Plugin
  • PollVote type
• DBComment Plugin
  
  • DBComment type
• Bookmark Plugin
  • Bookmark types
    

The Type Editor now enables the storage type to be chosen for content and form types. The icon is used to indicate data stored in JStore, and the icon   indicates data stored in the database.

Fig. 4. Choosing the storage during type editing

2.3 Data identifier

All the data stored in the base implement the DBData interface and have a rowId field containing the row identifier. This identifier is a unique integer in the table, but not in the base. However, since JCMS needs to work on unique object identifiers, it produces unique identifiers for objects managed in JcmsDB by concatenating the row identifier (rowId) with the object's class name. JCMS 6 can therefore recognize JStore identifiers and use them in the same way in the URLs.

2.4 Relations between the database and JStore

Some data in the database may reference data stored in JStore (e.g. a WFNote reference to publication and a member). This is managed by a column containing the identifier of the JStore data. If a data item in the base needs to be referenced by JStore, its unique JCMS identifier is exploited.

2.5 Common fields

All the data deriving from the Data classes have a number of common fields:

• rowId
• creation date
• modification date
• author

2.6 Publications

2.6.1 Constraints on publications stored in JcmsDB

It is possible to store publications in the database, but this imposes several constraints that do not exist with JStore storage.

The supported functionalities are:

• Author
• Creation, modification, publication, expiry and archiving date
• Archiving
• Partitioning by workspace
• Workflow (pstatus)
• Type editor
• Integer, boolean, decimal, date, text and link fields
• Lucene indexing
• Search by type, date, workspace and author
• Management interface in the workspace
• Export and JCMS Open API

The restrictions are:

• No update history
• No access to deleted data
• No rights management (read/write) by access list
• No categories field
• No multilingual field
• No multivalue field
• No management of inverted links
• No version management
• No open roles management
• No work copy
• No caddy (batch-type) operations
• No incremental import
• No choice of display template
• Text fields (textarea) are limited by default to 64 KB.

For this reason, storage in JcmsDB is preferable for technical data or data of simple structure (e.g.  forms or user content types).

2.6.2 Common fields

All publication types managed in JcmsDB have a set of common fields:

• rowId
• creation date
• modification date
• author
• title (monolingual)
• pstatus
• workspace identifier
• publication date
• expiry date
• archive date

2.6.3 User Content types

The amount of content produced by users (forums, comments, opinions, polls, etc.) tends to increase, but its final volume is difficult to anticipate. For this reason, user-generated content (UGC) should preferably be stored in the database.

To organize the content well, JCMS 6 distinguishes UGC from content produced by authors. This is handled by means of a new section in the workspaces:

Fig. 5. Management interface for user-generated content

In the API, a new publication sub-class, UserContent, completes the three existing classes (Content, PortalElement, Form).

2.6.4 Publications Manage

The publications management interface can list publications of a given type stored in JcmsDB or in JStore. The JcmsDB data are highlighted by the icon .

The content manager in a workspace does not list publications from both data stores: the All the types option lists only data types stored in JStore. Most of the publications stored in JcmsDB being user-generated or form submissions, in the standard configuration, they do not appear in these interfaces.

2.6.5 Indexing and searching

Publications in JcmsDB are indexed just like publications managed in JStore; searches can be made in both bases, so the hit list may include publications from either. However, due to the limits imposed on publications managed in JcmsDB, only restricted search criteria can be used (textual, type, author, date, status and workspace) on this database.

2.6.6 Interoperability

Publications in JcmsDB can be exported and manipulated via JCMS Open API. They can be exported and imported but the import mechanism does not support the incremental update.

2.7 ExtraDBData

JCMS 5.7.1 introduced the ExtraData feature which enables addition of fields of existing types. The ExtraData were managed in the form of a HashMap in JStore. JCMS 6 extends this concept by introducing ExtraDBData which are the equivalent of ExtraData but are stored in JcmsDB. They can be exploited for example to manage users preferences (e.g. configuration of a user's Virtual Desktop) or to store usage information (e.g. the date of a user's last login).

Read the article ExtraData et les ExtraDBData (french) for more details.

2.8 ReaderTracker

ReaderTrackers enable the readers of a given publication to be monitored. They replace ReaderNotes. Since they are stored in JcmsDB, there is no longer any volume limit. In addition to the date of first consultation of the publication, the date of last consultation and number of consultations is now memorized for each reader. Non-authenticated accesses are also recorded.

Fig. 6. Example of the readership of a publication

2.9 Archives

Every time a publication is archived, an ArchivedPublication representing this archive is created. Up until JCMS version 5.7, these objects were stored in JStore, a functional choice though not optimal, since one of the objectives of archiving is to reduce the number of objects in memory. JCMS 6 archives are now stored in JcmsDB.

3. Usability

JCMS 6 incorporates the latest technological progress on Web interfaces: drag'n drop, Ajax, contextual menus, suggestions, etc. These functions are integrated in the standard JCMS interface. They are also proposed in the form of components that can be integrated in the business interfaces.

3.1 Modal window

JCMS 6 proposes a new modal windows system. The use of modal windows is generalized throughout JCMS 6 by replacing the native windows of the browser with so-called lightboxes. In addition to better rendering that focuses the user's attention on the window (by graying everything around it), these windows can include rich-text validation or confirmation interfaces (text format, forms, etc.).

Fig. 7. Example of a rich-text modal window

3.2 Tooltips

JCMS 6 incorporates a new tooltip system that provide better ergonomics than the title attribute used in previous JCMS versions. The pop-up explanatory box that appears when the mouse pointer moves over certain objects can now be given a specific format and contain rich text, links, images, etc. These new boxes also help to improve performance by making pages "lighter", since their content is loaded only when the user calls it.

Fig. 8. Example of a tooltip pop-up (here about locked documents)

3.3 Pager

The presentation of the pagination system (or pager) has been reviewed in JCMS 6. It is possible to define new presentations by JSP templates and CSS sheets.

The pager is compatible with the JCMS 6 Ajax-Refresh technology which can refresh only the paginated zone each time a page changes. This is particularly useful for Query/Foreach portlets and search result lists. Despite this partial refreshment, the pager's HTML code is compatible with indexing robots (crawlers), so good referencing is still guaranteed.

Read the article pagination avec le tag <jalios:pager> (french) for more details.

3.4 Calendar portlet

In earlier versions, a "month" view in a Calendar portlet was one of the heaviest JCMS objects in terms of HTML and processing power, especially when the month displayed contained many events.

With JCMS 6, the Calendar portlet exploits the new tooltip features (see above) to display the detail of an event only when the user moves the mouse pointer onto it, which helps to keep the calendar weight and processing time relatively constant. Ajax-Refresh technology is employed to speeds consultation (by month, week, day, etc.).

Finally, a colored background is now used to highlight the days on which there are events concerning the user.

Fig. 9. A Calendar portlet

3.5 Portal editing

A contextual menu is present on each portlet. This enables rapid actions on the portlet (editing, deletion, displacement, etc.) and its configuration (template, skin, cache, etc.).

In the JCMS 6 portal editing interface, the portlets contained in Column Portlets or JSP Collection Portlets can be moved by simple drag'n drop. In order to remain compatible with all the browsers supported, this drag'n drop feature is not active for portlets contained in Portlets Row, although they can be moved using the contextual menu.

Fig. 10. The new portal editing interface

3.6 Other new features

3.6.1 Drop-down menu template

The display template of a drop-down menu in a Navigation portlet has been completely rewritten to make it more portable, accessible and easy to modify using CSS.

3.6.2 Full/summarytemplate

The Query/Foreach portlet has a new display template that presents the first content in detail followed by the others in summary form.

3.6.3 Members display template

The display template of a member has been revised to provide better legibility. The information about the member now includes the date of his last login.

Fig. 11. New member display template

3.6.4 Publications header

The icon bar in the publications header (print, send this page to a friend, etc.) has a new presentation, more discreet and more easily extensible using plugins. For example, the bookmark plugin adds an icon to this bar to enable the displayed content to be bookmarked.

3.6.5 PDA

With JCMS 6, in addition to the default PDA home page, it is possible to designate a specific home page for a particular PDA model. To do this, just declare the home JSP for this PDA via the small-device.home.Name property, with Name set to the PDA name as it is decoded by JCMS (e.g. small-device.home.IPhone).

JCMS 6 now provides automatic detection of the BlackBerry browser, in addition to the browsers already supported: iPhone, Windows Mobile, i-Mode, OperaMini and Android.

3.6.6 Review

The Review content type is now a generated type derived from the UserContent class and stored in JcmsDB.

3.6.7 Open roles

In editing forms, the choice of members for roles has been modified: members are sorted into alphabetical order and only the roles corresponding to the current state are proposed.

3.6.8 Version history

In the history of versions, the text field comparison shows, in addition to the differences, a side by side comparison of the field content.

3.6.9 Spell checker

JCMS 6 integrates the latest version of XMLMind Spell Checker 1.3. It is possible to add new dictionaries (in the WEB-INF/jalios/dictionaries/xsc/ or via the property spellchecker.xsc.dict-path). It is even possible to attach another spell checker, by developing a class implementing com.jalios.jcms.spellchecker.SpellChecker and declared via the spellchecker.class property.

3.6.10 Warnings box

In order to ensure rapid feedback of events occurring on the site, a Warnings Box has been added to the Admin Area to display the last 5 warnings.

Fig. 12. Warnings Box

3.6.11 Export of changes in CSV format

To support migration phases, the JCMS 6 change manager can export in CSV format all the files that have changed in order to manage this list in a spreadsheet.

3.6.12 TinyMCE 3

JCMS 6 incorporates the latest major version of the TinyMCE 3 rich text editor.

4. Plugins

4.1 Virtual Desktop plugin

This add-on plugin is available as an option (see the JCMS Price List).

The Virtual Desktop is designed to give users a personalized portal where they can select and organize the services they want to use.

The webmaster is free to propose any services he wants:

• internal information: new features, ongoing projects, logged users, etc.,
• external information: pre-configured RSS feeds, free RSS feeds (the user can input the URL of the RSS feeds he wants),
• personalized tools: agenda, bookmarks, workflow, document management, etc.,
• business portlets,
• widgets: in a Widget portlet it is possible to use most Google widgets (stock prices, weather, maps, road traffic, etc.). Users can be allowed to choose the widgets that want to use.

Users can organize their Desktop by placing their services in tabs (they choose the number and size of columns in each tab). A theme can be selected for the Desktop.

Fig. 13. Example of a Virtual Desktop

The Virtual Desktop is used very much like services such as Netvibes and iGoogle. Each service can be moved by simple drag'n drop. Users can choose the title and color of each service's box. Some services come with even more options: for example, it is possible to specify the number of articles displayed in an RSS service.

Each service takes the form of a portlet and a personalization layer. The portlets are the same for all users, but the personalization layer of each service and the configuration of the Desktop are specific to each user and saved in JcmsDB.

Fig. 14. Configuring a service on the Virtual Desktop

To propose a new service, just create a new portlet and set the Virtual Desktop aptitude in the Advanced tab of the portlet editor. It is also possible to define read rights to restrict user access to this service.

Many JCMS portlets can be used in the Virtual Desktop:

• RSS
• Workflow
• Calendar
• Bookmarks
• Notification
• Query/Foreach
• Logged Members
• Explorer
• Google Talk
• Widget
• Exchange (mail, calendar, contacts and tasks)
  
• etc.

The services can be proposed either by configuring one of these portlets or by creating a new portlet (or a simple JSP using the JSP Portlet).

Read the article Présentation et Utilisation du Bureau Virtuel (french) for more details.

4.2 Bookmark plugin

This add-on plugin is available as an option (see the JCMS Price List).

The Bookmark plugin provides the Bookmarks portlet which replaces the one in earlier JCMS versions. Bookmarks are specific to each user. In order to allow a very large number of them, they are stored in JcmsDB. Bookmarks can either be URLs of external sites or links to internal publications. For external sites, the site's icon (or favicon) is displayed; for internal publications the editing icon is added.

Internal publications can be added either by using the bookmark addition interface or by clicking the Add to my bookmarks icon in the icon bar present in the display of all publications.

Fig. 15. The Bookmarks portlet

4.3 Forum plugins

This add-on plugin is available as an option (see the JCMS Price List).

Forum management has been externalized to a plugin. JCMS 6 comes with two forum plugins: JStore Forum and DB Forum.

The JStore Forum plugin is identical to the forum function in JCMS 5.7.

The DB Forum plugin manages all the discussions stored in JcmsDB (which means it can therefore handle much larger volumes). The DBForum object encompassing a set of discussions remains in JStore. A few restrictions have been introduced following the migration to JcmsDB:

• discussions are no longer categorized (but forums can be);
• it is no longer possible to attach a document to a message;
• in cases of migration, since the identifiers of discussions are changed, the URLs of old discussions are no longer valid.

4.4 Poll plugin

This add-on plugin is available as an option (see the JCMS Price List).

The Poll function has been externalized to the Poll plugin. To be able to handle large volumes, all votes are now stored in JcmsDB. The Poll portlet and the "all polls" display have been completely revamped to provide a more elegant and more accessible presentation. It is now possible to attach an image to a poll.

Fig. 16. A Poll portlet

4.5 Module Exchange

This add-on plugin is available as an option (see the JCMS Price List).

This plugin provides four portlets to display information taken from a Microsoft Exchange server (Outlook Web Access - OWA - is necessary):

• Mail Inbox: the portlet displays incoming mail and provides a link to consult them on OWA;
• Work List: the portlet displays the list of tasks and allows it to be edited;
• Calendar: the portlet displays events in a Calendar Portlet;
• Contacts: the portlet displays a list of contacts and allows editing and searching in it.

   

Fig. 17. The Exchange portlets

4.6 Other plugins

4.6.1 DBComment plugin

This add-on plugin is available as an option (see the JCMS Price List).

The DBComment plugin replace the Comment plugin of JCMS 5.7. It has the same functionalities but, as its name suggests, the comments are now stored in JcmsDB.

4.6.2 Document Import plugin

This add-on plugin is available as an option (see the JCMS Price List).

This plugin is used to load documents automatically into JCMS from a given folder. Every time a new file (or modified version) appears in this folder, it is uploaded to JCMS in the form of a document. Contextual data are set for the DataController in order to be able to trigger specific processes on the imported document (e.g. categorization, rights, specific indexing, etc.).

4.6.3 Upload Portlet plugin

This add-on plugin is available as an option (see the JCMS Price List).

This portlet provides a very simple interface for uploading documents: the user just selects a file to upload. If the upload is likely to take a long time, the user can ask to be notified by e-mail when it finishes. The portlet can be configured to set default categories and rights. Contextual data are set for the DataController in order to be able to trigger specific processes on the uploaded document.

Fig. 18. The Upload portlet

4.6.4 GoogleMaps plugin

This add-on plugin is available as an option (see the JCMS Price List).

This plugin places JCMS data on a Google Maps display. A "Geolocation" field is added dynamically to data items declared as geolocatable. The geolocation can be expressed either in longitude/latitude or in the form of a postal address or place. The information is presented in a GoogleMaps portlet. The portlet has various presentation parameters (dimension, centering, zoom by default, view by default, control interface, etc.).

Fig. 19. The GoogleMaps portlet

4.6.5 Carrousel plugin

This plugin is used to present a series of images in a "carrousel". Several presentation and motion modes can be chosen.

Fig. 20. Example of a Carrousel

4.6.6 Compatibility of JCMS 5.7 plugins

Plugins developed for JCMS 5.7 must generally be adapted to run under JCMS 6. In the simplest case, it is sufficient to recompile the classes to be compatible with the new signatures of the methods (generic types). For more complex plugins, the declarations of the templates must be reviewed and certain Java and JSP code modified.

Most plugins written by Jalios for JCMS 5.7 are already available in JCMS 6 version.

5. Interoperability

Interoperability is understood to mean the ability to make JCMS compatible with the other components of the enterprise information system. When this is achieved, JCMS can be seen as a specialized IS service providing content and documentary management.

To respond partially to this need, JCMS 5.7 introduced an import/export mechanism to allow JCMS sites to communicate with others. JCMS 6 enhances the interoperability by providing a web service type API, a unitary authentication system (AuthKey), and the preservation of references during imports.

5.1 JCMS Open API

JCMS 6 incorporates JCMS Open API, a REST-compatible API web service providing access to JCMS functionalities. REST web services are an alternative to SOAP and notably simplify the principles by exploiting Web fundamentals (HTTP and URI). Although REST (REpresentational State Transfer) is just a set of principles, not a bona fide standard, JCMS Open API aligns with these principles as they are described in the book RESTFul Web Services.

JCMS Open API now enables:

• all JCMS data (publication, category, member, etc.) to be created, consulted, modified and deleted;
• searches for publications;
• consultation of the structures of publication types and workflows;
• consultation of site information (environment, data status, JSync synchronizations, open sessions, etc.);
• enable/disable data writes;
• imports

Plugins can create their own JCMS Open API accesses.

The data returned by the JCMS Open API methods are in XML with the same structure as export datastreams.

Examples of Open API requests:

The JCMS Open API functionalities will be enriched with each new JCMS release. In particular, it is planned to provide control of JSync synchronization, the Cache Manager, the Index Manager, and the DeployManager.

An article presenting the JCMS Open API in detail will soon be placed on this site.

5.2 JCMS Open API Client

JCMS Open API is language-independent, so it possible to control a JCMS site using Perl, PHP, shell scripts (using cURL for example), etc. In order to simplify the work of Java developers, Jalios comes with the JCMS Open API Client, a Java library that simplifies requests to Open API. It includes a mechanism to manage authentication by session, XPath navigation in the results, and an easy-to-use API.

5.3 AuthKey

Some processes relating to interoperability necessitate authentication (e.g. writing data, administration operations, access to RSS feeds on a private site, etc.). One first solution is to manage the authentication via the J2EE session. This solution requires that the client performs an authentication phase and preserves the J2EE session cookie. This is feasible with the JCMS Open API client, but not with the JCMS import process or with clients external to JCMS. For example, this approach is not suitable for integrating in an RSS client, an RSS stream located on a private JCMS site.

To meet this needs, JCMS 6 proposes a new, unitary authentication mechanism. The principle consists in adding to the URL performing the treatment a parameter containing an authentication key (authKey). This key is produced for a given member and is valid only for this URL and for a predefined duration. When the URL is accessed, JCMS 6 decodes the key and finds the signatory member, then executes the treatment requested in the name of this member. In the example of the RSS feed mentioned above, we just generate an authentication key for the RSS feed URL and add it to this URL (authKeyparameter).

The authentication key can be produced for a specific URL or for a URL prefix, which means it can be reused for all URLs starting with this same prefix. For example, in the case of JCMS Open API, it is possible to generate an authentication key for all requests starting with /rest/data/.

The JCMS 6 Admin Area has a new interface used to generate authentication keys. In addition to specifying the URL and the signatory member, other restrictions can be defined, such as validity duration, control of calling IP address, and authorized HTTP methods.

Fig. 21. Interface used to generate authentication keys

5.4 Import/Export

JCMS 6 enhances the import/export mechanism introduced with JCMS 5.7. During imports JCMS re-establishes the references between imported content. If we import a content item having links to other content, these links will be validated (provided the content items pointed to have also been imported). JCMS 6 can handle all types of links on content: field, Wiki, HTML (WYSIWYG field).

Finally, the ExportPolicyFilter has been enriched to enable finer controls on each export.

6. Accessibility

A great effort has been made to make the JCMS 6 front-office as accessible as possible. The precise meaning of accessibility remains slightly blurred, since several players propose different accessibility criteria (WAI/WCAG, RGAA, AccessiWeb, etc.), although there is fortunately considerable overlap between these baselines. For JCMS 6, we have opted for the RGAA criteria which are well documented and very similar to those of WCAG.

It is important to remember that accessibility issues are not restricted to the actual content management: they must be taken into account throughout the project, from the design phase (the official graphic style in particular) up until the operation phase. Accessibility challenges must be explained to contributors.

JCMS 6 support good accessibility by providing templates, components and functions that make it easier to conform to recent accessibility criteria, for example:

• Complete compliance with the XHTML standard;
• Systematic use of CSSs for presentation;
• A CSS adapted to print media;
• Management of invisible text areas to hold information reserved for screen readers and text-based browsers;
• Enrichment of file information (types, weights, units, etc.);
• Forms compatible with screen readers;
• Indication of the language change in fields;
• Integration of TinyMCE 3 to ensure respect of accessibility criteria.

The DevTools plugin contains a new tool that makes it easier to validate a site. It produces on a single page all the items present on the front-office: portlets, skins, display and abstract templates, forms, and so on. It is then possible to run a validation of this page with tools such as TotalValidator which verifies a series of criteria (XHTML, WCAG, etc.).

Fig. 22. Example of a page validation with Total Validator

Read the article JCMS et l’accessibilité (french), for more details.

7. Enhanced performance

7.1 Front-office optimization

7.1.1 Ajax

Ajax technology makes user interfaces more friendly and also boosts performance. Partial loading of selected parts of the interface and loading only at the user's request reduce the quantity of data transferred; only the necessary content is loaded and subsequent loads are lighter.

7.1.2 Optimized JavaScript loading

The book High Performance Web Sites, a compilation of research work by Yahoo!'s Exceptional Performance team, presents 13 rules for optimizing page loading. The Firefox add-on YSlow can be used to check whether or not these rules are respected by a given page.

JCMS version 5.7 already respected some of these rules: CSS style sheets placed at the top of a page, merging of CSS and JavaScript files, Gzip compression, etc.

JCMS 6 completes the integration of these rules by providing broader compliance in the merging of JavaScript and placing JavaScripts at the bottom of the page (rule 6), which gives users an impression of faster page loading. The JavaScript loading blocks all other loads (images, CSS, etc.). By deferring the other loads, the browser immediately gets all the resources it needs to render the page.

Putting the JavaScript at the bottom of the page has consequences on the integration of the application's specific JavaScript. The JavaScript files must be included in the page with the context.addJavaScript() method and the scripts contained directly in the pages must be modified to replace the <script> tag by <jalios:javascript>.

Thanks to optimizations made by Jalios engineers, JCMS 6 sites generally obtain an "A" score for their compliance with rules 5 (Put CSS at the top), 6 (Put JS at the bottom), 7 (Avoid CSS expression), 9 (Reduce DNS lookup), 11 (Avoid redirect) and 12 (Remove duplicate scripts). The JavaScript / CSS mergers generally earn a good score for compliance with rule 1 (Make fewer HTTP requests). The "A" score can be earned for rules 3 (Add an Expires header) and 4 (Gzip components) too by correctly configuring the Web server. Finally, rule 2 (Use a CDN) does not concern JCMS directly and it is unsuitable for most sites since it imposes a distribution of the static resources by a Content Delivery Network.

Fig. 23. Example of a YSlow performance report

7.2 Reducing the weight of the data

JCMS site data, like people, tend to get a little fatter as time goes by. With JCMS 5.7, creating an empty publication consumed 1,460 bytes, and a new member 880 bytes. With JCMS 6, all objects are now on a strict diet! The table below shows the space gains achieved:

This obviously reduces memory consumption and enables more objects to be loaded – and also improves JCMS performance. The reductions are achieved by deferring the allocation of many collections (Set and Map), which has greatly reduced the number of allocated objects. The garbage collector having less objects to manage, it triggers less often and processes take less time.

It is difficult to estimate precisely the gains when a site is migrated from JCMS 5 to JCMS 6. Not only have objects become lighter but some are now stored in JcmsDB. By way of example, for a site of modest size handling 16,000 objects with less than 2% of these objects placed in the database, a JCMS 5.7.4-to-JCMS 6 migration reduces the memory consumption when loading the store by 30%. The JaliosXperience site, on which 85% of the objects have been moved to the database, has seen its memory consumption on loading divided by three.

7.3 Access to the version history

With JCMS you can consult the version history of any publication. To build the history, JCMS partially reads the store in order to find all the modifications made to the publication. Since versions 5, JCMS optimizes the reading of the store by jumping to the first operation on the publication, but even so finding all the other operations can be a heavy task, especially if the publication is very old and the store is large. JCMS 6 improves the performance by using a cache on the version history which ensures that all browsing operations on it, such as consulting a previous version or comparing two versions.

7.4 Database access

The fact that JStore manages all its objects in memory ensures excellent performance. JStore loads each object only once and has many object indexes, for example all the objects in a given class, all publications attached to a given category, all the publications of a given member, etc. Access to data is not only fast but involves no object creation. Consequently, the memory consumption associated with JStore is very stable and depends only on the number of writes (which are much less frequent than reads). This makes it easier to specify the memory needed by a site and also makes work lighter for the garbage collector.

In JCMS 6, data stored in JcmsDB is accessed using the Hibernate query mechanism. During a request to JcmsDB, a series of treatments is triggered:

1. JCMS builds a Hibernate request (Criteria or HQL).
2. Hibernate analyses this request and builds the SQL query(ies) optimized for the underlying RDBMS (thanks to the dialects).
3. Hibernate opens a connection to the RDBMS and executes the SQL query (via JDBC).
4. The RDBMS treats the query and returns the list of hits.
5. Hibernate scans this list and builds the corresponding Java object for each hit.

This mechanism therefore involves inputs/outputs (connection to the base, transfer of results), calculations and memory allocation in the webapp. JCMS 6 incorporates several optimizations to guarantee good performance and low memory consumption for data managed in JcmsDB.

7.4.1 Simple relational model

The standard data management applied by JCMS in JcmsDB is based on a very simple model (one table per class) with very few relations between tables, since the structure is managed in the objects of the store. SQL queries are therefore very simple and generally necessitate no joins.

7.4.2 Column indexes

Indexes are a proven technique for improving database performance; they avoid the RDBMS having to scan all the records in tables. On the downside, they incur additional tasks during writes and they increase the memory consumption and/or RDBMS disk space requirement.

The RDBMS sets an index on the primary key of a table (rowId in the case of JCMS). Additionally, on all the tables generated by Hibernate, JCMS declares indexes on the main columns used in the WHERE clauses of SQL queries. For example, the Review type includes indexes on the columns authorId, pstatus, workspaceId and publicationId. This means that when a contributor consults the list of his published reviews in his workspace, the indexes workspaceId, authorId and pstatus can be used.

7.4.3 Default sorting on the primary key

Sorting the results of an SQL query (orderby clause) is usually a heavy operation. One improvement technique consists in having an index on the sorting column. In JStore, the default sort is on the creation date (cdate). To avoid having to declare an index on the cdate column of all the tables, the default sort makes use of the primary key (rowId) which is naturally indexed. The rowId is not a date, but it does have the same incremental characteristic, so this ensures a sort identical to one on cdate.  

7.4.4 Results pagination

Paginating search results is one way of helping a user to scan through a large number of results. The JCMS pagination system (<jalios:pager>) has been optimized to enable high-performance pagination of JcmsDB data even for very voluminous tables.

7.4.5 Optimizing the number of SQL queries

All JCMS code (Java and JSP classes) has been analyzed to optimize the number and type of SQL queries.

Counting queries (count(*)) are used when the only need is to know if data exist corresponding to certain criteria.

Caches have been placed on certain data (e.g. the number of reviews of a publication).

7.4.6 Pool de connexions

JCMS 6 is delivered with the C3P0 connection manager which is deactivated by default. To activate it, open the file WEB-INF/data/custom.prop and just decomment the lines starting with hibernate.cfg.common.prop.hibernate.c3p0. Then restart JCMS.

8. Developer tools

8.1 Ajax-Refresh

Ajax technology enables highly reactive web interfaces since only certain parts of a displayed page need to be refreshed, not the entire page. Even though Ajax frameworks exist, Ajax is not always easy to implement and generally requires solid knowledge of JavaScript development. Moreover, this delayed, partial loading of page content modifies the kinematics of exchanges, which raises a number of questions:

• How do we manage the browsing history?
• How do we continue to guarantee good referencing?
• How do we respect accessibility criteria?
• How do we preserve the execution context (current categories, current portal, etc.)?
• How do we reconnect the JavaScript events and load the CSS sheets?

To cope with this need and address these questions, Jalios engineers have developed Ajax-Refresh which simplifies the use of Ajax technology in one specific case: the refreshment of part of a page (e.g. a portlet). Ajax-Refresh is not a new JavaScript framework, just a declarative model with a simple naming convention.

In the general case, the zone to be refreshed must be identified by a tag <div class="ajax-refresh-div">. The refreshment triggers can be links or buttons that must be identified with the CSS class ajax-refresh. It is possible to ask for confirmation before refreshing (CSS class confirm). The zone is refreshed with the content of the JSP targeted by the link or by the form. During the Ajax call of the JSP (XmlHttpRequest), JCMS 6 preserves the execution context (authentication, current category, current portal, etc.). In cases where the refreshment concerns browsing (e.g. the pager), the browser history is checked in order to enable a return to the previous state of the page. The links triggering the refreshments are standard links. Ajax-Refresh enriches them dynamically. In this way the accessibility and referencing criteria can be respected.

For portlet developers, the use is even simpler since all JCMS 6 portlets are pre-configured to be refreshed by Ajax-Refresh. The developer just indicates with the CSS class ajax-refresh the links and buttons that trigger the refreshments.

Example for the general case:

<div class="ajax-refresh-div">
<%= new Date() %><br/>
<a class="ajax-refresh" href="testLink.jsp">Refresh</a>
</div>

Example for a portlet:

<%@ include file="/jcore/doInitPage.jsp" %>
<%= new Date() %>
<a class="ajax-refresh" href="<%= ServletUtil.getResourcePath(request) %>">Refresh</a>

An article presenting the Ajax-Refresh technology in detail will soon be placed on this site.

8.2 JavaScript libraries

JCMS 6 makes life easier for developers by integrating a set of JavaScript libraries, notably

• Prototype 1.6 : a JavaScript toolkit providing a high-level API, homogeneous whatever underlying browser;
• Script.aculo.us 1.8 : a library to support drag'n drop, predictive searching (auto-completion) and visual effects.

8.3 Sprites

In an HTML page it is not rare to find several dozen images and icons, each of them necessitating an HTTP request. The first rule of High Performance Web Sites is to reduce the number of such requests. To do this, one solution is to group the icons in a single image and use CSS rules to make only the required icon appear.

Exploiting this idea, JCMS 6 incorporates the set of icons, FamFamFam Silk Set, with more than 700 icons in a single image. To display one of its icons, we just indicate its name in the image's CSS class.

Example :

jcmsContext.addCSSHeader("css/fff-sprite.css");
<img class="icon ss_sprite ss_add" src="s.gif" alt="Add"/>

The DevTools plugin provides a page listing all the icons accompanied by the name of the associated CSS class.

8.4 Types template

JCMS 6 has a new mechanism enabling dynamic selection of display templates according to their usage. This means that a portlet can have a different appearance according to whether it appears in a portal page column, in a portal full page, on the Virtual Desktop, in a PDA display, in a collaborative space dashboard page, or is vocalized by a screen reader.

Fig. 24. The same portlet may be presented differently according to its display context

This mechanism also concerns editing templates which can differ according to the usage context (workspace, front-office, popup, Virtual Desktop, etc.)

Fig. 25. Example of an editing form contextual to an editing zone (front-office or back-office)

For this purpose, JCMS 6 introduce the notion of usage and enables a presentation JSP to be associated with each usage. JCMS 6 proposes standards usages (full, query, box, edit, editFront). Plugins can be used to add new usages (e.g. PDA, Virtual Desktop, text-to-speech, etc.).

The portlet editing form has been enriched to select a particular template to match the usage:

Fig. 26. Templates selection interface

An article presenting the new templates management system in detail will soon be placed on this site.

8.5 Data Image

In JCMS 6, all objects deriving from the Data Class have the getDataImage() method which returns the image associated with the object. The choice of this image varies according to the class of the object:

• Member: the member's photo;
• Category: the category's image field;
• Document: the thumbnail associated with the document;
• Portlet: it is possible to associate an image with each portlet (Information tab);
• Publication: it is possible to choose in the type editor the image field serving this usage.

In addition to the getDataImage() method, portlets have the getPortletImage() method. This returns a preview image of the selected display template if the portlet image does not exist.

8.6 ExtraData and ExtraDBData

With JCMS 6, the ExtraData and ExtraDBData support inheritance. It is now possible to define a ExtraData valid for all the publications, even for all the JCMS data (Data).

For more details, see the article on the ExtraData.

8.7 Generic types

The JCMS 6 API makes abundant use of the generic types offered by Java 5. This ensure more robust code (by controlling the type during compilation) and generally improves the legibility of the API and the code, notably by identifying the type of objects contained in the collections (List, Set, etc.).

8.8 Compatibility

To know the compatibility of JCMS 6 with operating systems, JVM, application servers, RDBMS, LDAP servers and browsers, see the JCMS 6 Installation and Operation Manual.