Release Notes 10.0

September 17, 2021

Table Of Contents

Upgrading to 10.0

Read This First

Upgrading to 10.0 is a major upgrade, and so you are strongly advised to read everything in the “Upgrading to 10.0” section before attempting it.

It is entirely possible the upgrade will fail the first time. It is therefore critical that you have a backup of the database before you start.

You must also upgrade to the latest 9.0.x before upgrading to 10.0. Unlike earlier releases, there is an incompatibility that means 10.0 cannot apply 9.0 upgrades. Consequently, you must complete all 9.0 database upgrades before upgrading to 10.0. This means you must boot Calpendo with the latest 9.0 so that it can apply those upgrades.

If you have a Calpendo test environment, then you should perform the test there first, using a copy of your production data.

If you do not have a test environment, then you will need to plan for enough down time for the upgrade. See below for an estimate of how long a successful upgrade will take.

Primary Key Changes

Unlike earlier versions of Calpendo, you cannot upgrade directly to 10.0 from any version; you must upgrade to the latest 9.0 version before attempting to upgrade to 10.0.

Upgrading from 9.0.x to 10.0 is a relatively slow upgrade and it manipulates the data in the database fairly extensively. This is because the format for all primary keys has changed from a 32 bit integer to a 64 bit integer. This is in preparation for Calpendo Enterprise which includes the ability for data to be shared between systems.

This has resulted in primary keys being formatted as:

This scheme allows for primary keys to be created up until 19th January 2087 with the largest legal primary key being 2^53-1.

When upgrading a 9.0 system to 10.0, your database will be allocated a random shard number and all primary keys will be modified to the above format. This means all pre-existing keys are multiplied by 1024 and the shard number added to it.

When displaying an ID in the user interface, we display the number in the format yyyymmdd.HHMMSS.sequence.shard unless the ID is small enough that it looks like it came from a pre-enterprise system, in which case it will be displayed as sequence.shard.

The conversion of primary keys is a complex operation on the database, and while we have taken great care to make sure this is done correctly, it is imperative that you create a backup of your database before the upgrade just in case anything goes wrong.

Also, if there are any tables in your database that are not used by Calpendo, it is important to understand that the above primary key translation will be done on them as well. If you want to avoid this happening, then you should drop those tables after you have taken a backup (and verified that it is good) and then upgrade, and then restore your tables. If the tables are used by Calpendo, then you will have to leave them in place so that they can be converted. Otherwise Calpendo won’t be able to continue to work with them.

Before Upgrading To Version 10.0

Unlike previous upgrades, the upgrade to version 10.0 is not entirely automatic; you must examine your system and make certain changes first before upgrading so that the upgrade will then be automatic. These changes are all compatible with version 9.0, which means that you can get ready for upgrading to version 10.0 before you actually do the upgrade itself.

  1. All custom integer properties added to any BiskitDef that store a primary key should be modified. There’s an integer sub-type called “Primary Key” that can be set onto integer properties. This does not change any behaviour in 9.0, but it is a signal to the upgrade procedure that this property stores a primary key, and so should be changed to a long.

    If you have an integer property that stores a primary key but you do not mark it as storing a primary key, then the upgrade procedure will not modify it. This would then mean that it is no longer capable of storing a primary key after upgrading to version 10.0 which will cause problems. It could, however, be manually modified after upgrading to 10.0 to change it to be a long. This would require changing the type of the property and also changing the column used to store it. It is easier to simply mark the property as storing a primary key in version 9.0 instead.

  2. If any of your workflows call the function “toInt”, which converts a string to an integer, and the value concerned is actually a primary key, then the function should be changed. Call “parsepk” instead. This is a function that also converts a string to an integer in version 9.0, but returns a long in version 10.0. However, the names of the inputs and outputs are different, so this isn’t a drop-in replacement. That is, you will need to modify any actions that use the output from “toInt” that has been switched to “parsepk”.

  3. If you have any workflow function actions that call the function ‘getAsInt’ or ‘getAsIntList’ or ‘csvDecode’, then they should be checked to see whether they are actually returning primary keys. If they are, then the actions should be changed to call ‘getAsPrimaryKey’ or ‘getAsPrimaryKeyList’ or ‘csvDecodeAsPrimaryKeys’ as appropriate. These are functions that make no functional change in version 9.0, but which are modified during an upgrade to version 10.0 to return a long or list of longs.

  4. If you have any workflow that uses the ‘Registers’ biskit, and you use one of its integer properties (int1, int2, int3 etc) to store a biskit’s id, then you should modify the workflow so that it uses the equivalent primary key property instead. That means that references to int1 should be changed to pk1, and references to int2 should be changed to pk2, but only for those properties that store a biskit’s id. During an upgrade to version 10.0, Registers properties pk1, pk2, pk3 etc are converted from int to long.

  5. Go to Admin->Bakery and click on “Validate Biskits”. If there are any errors, then please correct them before upgrading. Also click on “Update DB Schema…” to make sure the database schema is fully up-to-date before upgrade. In particular, if the script it generates wants to add foreign key constraints, then it is essential that those changes are made before upgrade. That’s because we use foreign key constraints as an indication of a column that contains a primary key and needs to be converted from an integer to a long integer.

In all cases, if you find from the above that you need to modify anything, then we recommend checking that your workflows are still correctly defined after the upgrade, and that they operate correctly.

How To Perform The Upgrade

Upgrading from any version to the latest (including applying bug fix updates) should be done with the following procedure:

Time Required To Upgrade

The upgrade may take a long time. I rough rule of thumb is that it will take about 20 minutes per gigabyte of a gzip-compressed backup of the database. That is, if you run mysqldump and gzip the output, and it’s about 1GB, then the upgrade will take about 20 minutes. This is only a rough guide.

If Things Go Wrong

If you have a large database, and a small MySQL setting for innodb_buffer_pool_size, then the upgrade may fail with the log file (which may be in /var/log/tomcat*/calpendo.log or similar directory) containing a message in the first error saying:

the total number of locks exceeds the lock table size

If this happens, then you should increase the value of innodb_buffer_pool_size. This probably means you should edit a file under /etc/mysql/ to add or change the directive and restart MySQL. This is the kind of line you would add:

innodb_buffer_pool_size = 2G

where you would choose the size depending on what’s required. What’s required depends on the amount of data you have. So you might want to increase this for now, perform the upgrade and then put it back afterwards.

It should, in general, be no bigger than 80% of your RAM on a dedicated database server. If you also run Tomcat or other services on the server, then you should allocate less memory to the innodb_buffer_pool_size.

You can see what your current setting is by running this on the database:

select @@innodb_buffer_pool_size;

For some online resources giving background about this, see:

If this is your problem, and you’ve increased innodb_buffer_pool_size, then you can try again. This means dropping your existing database, rebuilding from backup:

    sudo mysqladmin drop calpendo
    sudo mysqladmin create calpendo
    sudo mysql calpendo < calpendo.sql

and then restart Tomcat.

Downgrading from 10.0 to 9.0

The general procedure for downgrading from 10.0 to 9.0 is:

Do not merely load an old database on top of an upgraded or partially upgraded database and run an old Calpendo. It will break things, but possibly not until you come to do another upgrade, by which time it may be too late to fix things easily.

New Features

Linked Bookings

Linked Bookings are a feature whereby a booking for one resource can automatically trigger the creation of a booking for some other resource, perhaps at some earlier time or at the same time. When the “parent” booking is moved to a new time, then the “child” booking should be moved too. When the parent booking is cancelled, the child should be cancelled too.

An example would be when you have a PET Scanner and so when you have a scanner booking, you would like an automatic booking for the creation of a radioactive tracer some time ahead of the scanner booking.

Bookings now have a permanent property added called parentLinkedBooking that provides a reference from a child booking to its parent. This value would not be set for any booking that is not a inked booking child.

In earlier versions of Calpendo, we have implemented linked bookings for customers by putting the necessary code into workflows. In version 10.0, it is no longer necessarey to use a complex workflow for this because we now have built-in support for it. This also provides support for all the different flavours of linked bookings that people have requested until now.

For example:

Linked bookings are now managed by going to the resource editor by selecting menu item Admin->Resource Editor and creating a Linked Booking Link.

Two extra properties have been added to Booking to support linked bookings. They are both only populated for bookings that are children of some link. The properties are:

Linked Bookings are not available in Calpendo Lite.

Linked Menu Items

Sometimes, you want some of the same things to appear on menus for different people. When this happens, it’s annoying when you want to make a change to one of them, because it means you have to make a change to all of them.

There’s now a solution to this in the form of “Linked Menu Items”.

This means that you can set up a menu item on a menu that is a linked to a menu item on another menu, or even on the same menu. Then, when the menu is loaded, it replaces the linked menu item with whatever it was pointing to.

This will work for any type of menu item. This means that if you create a submenu and a link to it, then the whole submenu will be replicated. Alternatively, you may want to link a single item such as a custom search page.

Linked Menu Items can be found in the menu editor by adding a custom page of type “Linked Menu Item”.

Kanban Reports

There’s now a new style of built-in report available called “Kanban”. This provides a Kanban board in which items are separated into separate lists depending on the value of a configurable property.

For example, from a search for users, if you select a Kanban report, and then select the property “type”, it will show the users in separate lists with all those users having the same type being in the same list.

Booting With Broken Biskit Definitions

Previously, you could modify the biskit definitions in such a way that the system could not boot. Now, if there’s something fatally wrong with the biskit definitions, or missing tables from the database, then the system will boot with the original set of biskit definitions.

This means the system is in a rescue mode that will allow you to go to the bakery and fix any problems that exist.

This makes it much harder to put the system into a state whereby it cannot boot and avoids the need to modify the database outside the web browser to fix it.

Broken Upgrade Recording

If a database upgrade fails for some reason, there’s now a record of the SQL it was trying to run in the system events.

Agresso

There is now an Agresso module. This makes it easier to integrate with Unit 4 Business World, formerly known as Agresso.

The module provides:

A workflow can then generate GL07 and/or LG04 biskits and generate a file suitable for sending to Agresso.

Custom Tree Pages

For a long time now, there has been support in the menu editor for adding pages that:

There’s now a new custom page option available in the menu editor called “Custom Tree Page” that lets you define a custom tree to appear on the left, where each of the items in the tree can trigger any custom page or built-in page on the right.

However, it can also display child items in the tree for biskits that it finds, with optional grouping depending on the value of one or more properties.

For example, you could create a menu item for a Custom Tree Page and give it two child menu items as follows:

This might then produce a tree that looked like this:

This will be explained more in the documentation and videos.

Requirements

There’s now built-in support for storing system requirements. The intention is to use this for new customers to sort out the requirements they have for their new system. The requirements support provides a means of showing all the individual requirements, along with a mechanism for attaching notes to each requirement so that there can be a discussion based around each requirement.

Requirements can also be split into sections and given a subject.

Requirements can also be given tags to provide a custom means of identifying them.

This is built-in to each Calpendo so that once your system has been built, it will continue to contain the requirements to serve as a specification for what the system contains and what it’s supposed to do. This will help both customers and Exprodo when anybody unfamiliar with the system needs to understand what things are in it and how it’s supposed to work. It will also help with the process of getting new customers up and running.

This will be documented further elsewhere.

When displaying a search page with Requirements, there’s now a new report type, called “Requirements Report” that displays the requirements properly on the screen. It can also be exported to PDF where it will include a copy of all the requirements included in the search, along with all their notes too.

Send SMS Text Messages

You can now send SMS text messages from a workflow. To use this, you will need to create an account with Twilio and buy a phone number from them. Then, create an instance of TwilioPhoneAccount in Calpendo and give it the details of your Twilio account and phone number.

Then you can use the new workflow function called sendTwilioSMS to send an SMS message, giving it the message to send, the phone number to send it to, and referencing your TwilioPhoneAccount.

Draft Projects

There is now a new “Draft” status that can be selected for a project.

This includes the following changes:

New Enterprise Features

Introduction to Enterprise

Calpendo Enterprise is a version of Calpendo that allows multiple separate Calpendos to work together in a federation. Each Calpendo instance is known as a shard.

The idea is that where there are multiple facilities (which you may call cores, labs, facilities, departments or similar things) and each of them needs to be autonomous, but work together in some way, then Calpendo Enterprise will support that. Each of these facilities will have their own Calpendo, but the Calpendos will talk to each other.

Each Calpendo is known as a “shard”, as it forms a part of whole “federation”.

There will be a single “master” shard which provides a place to define some things that are common across all shards. This would be the first shard created in a federation.

This system will allow:

Most of the changes for support Calpendo Enterprise are a part of Calpendo 10.0, and the final elements will ship in a latesr version.

Some of the concepts introduced in Calpendo 10.0 are described in the following subsections.

Enterprise Terminology

Each individual Calpendo is known as a shard. A collection of shards working together is known as a federation. The first shard created is known as the master shard and it controls the standard biskit definitions (such as those for User, Booking and Resource). These are inherited by all other shards, who may create custom sub-types if desired.

For example, a shard might create a subtype of Booking that contains properties required on that shard.

Any shard that creates a BiskitDef is said to own it. Only the shard that created a BiskitDef is allowed to make changes to it.

A slave shard is any shard that is not the master shard.

A biskit is said to be fragmented when that biskit has a main part plus one fragment of data for each shard. Each shard can define what properties exist on the fragment that that shard.

We generally use the term “facility” to refer to the concept of an autonomous (or semi-autonomous) part of a larger organisation, and each facility would normally have its own shard. You can put more than one facility into a shard, but it is more difficult than having them in separate shards.

We will sometimes use the words facility and shard as if they were synonymous. They aren’t quite the same thing though because you can have multiple facilities in one shard. Also, you may have something in your organisation that is politically considered to be a single entity or “core facility”, and which is comprised of more than one part that have different technical rules around how it is run. In that case, you might split one “core facility” across multiple shards.

Pinning

A biskit definition can be pinned. That means that it is the preferred type for biskits. For example, suppose:

Then on the master shard, if you go to Search->Search and try to create an instance of A, it will ask whether it should be an A, AX, AXChild1, AXChild2, AY, AYChild1 or AYChild2.

However, if you search for A on shard X, and then try to create an instance, it will only offer you the chance to create an AX, AXChild1 or AXChild2. That’s because AX is pinned on shard X.

Similarly, trying to create an A on shard Y would only offer the option to create an AY, AYChild1 or AYChild2.

Exporting Biskits

In order to see bookings for another shard’s resources, the other shards resources must be exported to other shards.

Projects can be exported, and in fact must be exported to another facility’s shard if you want the project to use that facility’s resources.

Fragmentation

A user of one facility might well be a user of another. Each user that registers will request access to the facilities they want to use. This means that a single user record will exist that spands the whole federation.

This means there is some facility-specific information that must exist for each user and facility:

From a technical perspective, a user is said to be fragmented.

Similarly, Calpendo projects are fragmented. That’s because each facility will have its own ideas about what information should be stored about a project, depending on what makes sense for that facility. This means each facility will define the properties that exist on the project fragment for the facility’s shard.

If you look in the bakery, you will see a UserFragment and a ProjectFragment.

If you set up an Enterprise system, then for each shard, there will be a subtype of UserFragment and ProjectFragment defining the properties for that shard beyond those defined on UserFragment and ProjectFragment.

This means that when looking at a project, you will see one tab for each shard, which contains the information in the ProjectFragment for that shard.

Local User, Project and Resource Types

A user in Calpendo has a UserType.

The UserType values that can be selected are defined globally, so that every shard would see the same selectable values.

Some facilities would like to segregate their users into types in a manner that suits their facility, and not in a way that suits other facilities, or in a way that suits the whole federation. To make this possible, a user can have a local type.

The way this works is that there is a LocalUserType biskit. Each shard will have instances of LocalUserType created, and only instances of LocalUserType that were created on a particular shard can be assigned as the local type for a user’s fragment on that same shard.

In the same way, a ProjectFragment stores a LocalProjectType, and the values that can be stored in it are all those LocalProjectType instances created on the relevant shard.

Resources also have a LocalResourceType. However, they are different from user and project because Resource is not fragmented. This means that the local type stored on a resource has to be from the shard that created the resource.

Permissions

Permissions have changed very slightly, but the difference is only visible when running Calpendo Enterprise. A user can access multiple shards, and may have different roles on each one. For example, somebody may be an admin for one facility, but a regular user on another.

To make this work, there are three properties that define a user’s roles.

First, there’s the User.roles property as has always existed.

Then, on each UserFragment, there is a roles property that defines the additional roles the user has on the shard that fragment relates to.

For example, if a user is an admin across the whole federation, they could have the admin role in their User.roles property. Whereas, if a user is an admin on only one shard, then they would have the admin role in their UserFragment.roles for the relevant shard, but not in their UserFragment.roles for other shards, nor in User.roles.

Finally, the roles that can be assigned are defined globally by the master shard. If a shard wants to have other roles that exist only on that shard, then it can. Every shard in a federation has a MappedInt that defines the roles that exist for that shard. These are known as “local roles”, and they are stored in UserFragment.localRoles.

When creating permissions in an Enterprise system, the “Roles” section displays a little differently. It now lets you select both the global roles and the local roles (if the shard on which you’re creating the permission has some local roles defined in its MappedInt).

If you indicate that a permission applies to somebody with the global “admin” role, then it does not matter whether a user has the admin role in their User.roles or their UserFragment.roles; they are treated the same.

Expiry Date

A user has an expiry date in the ExprodoUser.expiryDate property. However, each facility may want to set an expiry date on when the user can access their shard. For this, there’s also now a UserFragment.expiryDate property to store the expiry date for a user on the shard for that fragment.

Workflow Changes

Workflow Manager User Interface

There are new events and workflow actions that have been introduced in this version.

As a consequence, the old method of adding events and actions by clicking a button that gives a drop-down list, has become cumbersome. It generally took too much scrolling and searching to find the type of item you wanted.

So there’s now a new tab labelled “Add” that appears for each event and action. On that new tab, you will find one button for each type of event and action. Those that are legal to be added in the current context will be enabled, and the others greyed out.

The previous “Add Event” and “Add Action” buttons have both been removed so that you must now use the new “Add” tab instead.

Workflow Debugger

This provides a means of properly debugging workflows so that the use of system events should no longer be necessary.

This is different from most debuggers, in that it does not operate live. Instead, you record some activity and then use the debugger to examine the recording to see what happened. This has the advantage over a “live” debugger that you can go backwards in time through a recording, as well as forwards.

There’s a new item added to all menus that previously had Workflow Manager on them, and you can also reach it by clicking the “Debugger” button from the Workflow Manager.

To use this, you must first go to the debugger, go to the recording options, and enable recording. This can be enabled for all workflows or just some.

Map Workflow Action

Map Workflow Action is a new type of action. It lets you group a list of biskits by one or more keys that you can define.

This is best explained by example. Suppose you need to generate PDF reports or invoices of bookings, and you want one PDF for each project owner (the principal investigator).

This is an ideal situation for the Map Workflow Action. You given it a list of the bookings, and set up the keys as “project.owner”.

The result is a list of entries, each of which has a user (the project owner) as a key, and a list of bookings whose project.owner matches the key. This lets you loop through the entries, and produce one PDF for each entry.

This leads to a workflow snippet that looks something like this:

Now suppose you want the PDFs to group the bookings by project. This means you need the bookings grouped by project as well. So change the map action to have keys “project.owner” and “project”.

Now the map action will return a different list of entries. Each entry will have a user (the project owner) as a key, which is the same as before. It will also have a list of values (bookings) for that project owner, as before. However, each entry will also contain a list of sub-entries. Each of those will have the project as a key and a list of bookings that have the project.owner and the key as project.

This leads to a workflow snippet that looks something like this:

Import Workflow Action

This action can be used to import data.

This works in a similar way to the file import page, except you give it a a list of biskits, and specifications on how to resolve properties from those in the input list to something that will be on the biskit imported.

You also specify how to match properties in the input list to pre-existing biskits. This is so that we know which biskits to update, when the import is set to update existing biskits rather than create new ones, or when the import is set to create new biskits that don’t match a pre-existing one.

You might generate your input list of biskits from a file you’ve parsed or some other source, and then want them to be converted into something that can be saved to the database.

Macro Workflow Event

Defining and using Subroutines

The following actions have been added:

A subroutine has inputs, and produces outputs, much like a function. Unlike a function, a subroutine can be passed as an argument to another subroutine, allowing you to provide plug-in behaviour to a subroutine.

For example, you might have a complicated subroutine that generates an output file, but you might want each line to be either comma separated or HTML. You can handle this by using a subroutine to generate each individual line in the format you want, and passing that as an argument to the subroutine that builds the file.

You can define an interface. This is something that lets you define inputs and outputs, but an interface has no child actions so it can’t do anything.

When you define a subroutine, you have the option to indicate that you implement a particular interface, which means it controls what your inputs and outputs are. Alternatively, you can define the inputs and outputs directly in the subroutine.

When you define a subroutine that takes another subroutine as an input, you should specify the type of one of those inputs as a Define Interface Workflow Action. That will allow you to use it in a Call Subroutine Workflow Action.

Then, when you pass a subroutine as an argument to another subroutine, the subroutines that you pass in must specify that they implement the interface required for the argument.

Subroutines and interfaces can only be added below a Macro Workflow Event, just like custom functions actions.

The differences between a subroutine and a function are:

Defining and using Constants

It’s common that you need workflows to operate using some values that control how it works, where these values rarely change. In these circumstances, it’s preferable not to bury the values deep inside a workflow because that makes it difficult to maintain if those values do need to change.

So we now have new actions:

This is like a Create Variables, but one that can be shared across different events or even across different workflows.

Constants can only be added below a Macro Workflow Event. However, you can add any action as a child of a Define Constants Workflow Action.

This means you can define what constants you want, and then the child actions can calculate something and store their value onto the constants biskit.

Constants are calculated once when the system boots, and whenever the workflow containing the constants is modified. They are not recalculated at any other time. So the values you generate will be used many times without reclculating them.

Find Function (Breaking Change)

The find function is given a BiskitDef and a primary key or list of primary keys, and returns you the biskit or list of biskits they represent.

Since primary keys are not longs and not integers, this means the signature of the find function has changed so it expects a long or list of longs. If you have used the find function in any of your workflows, then after an upgrade, you may need to change your workflows so they continue to work.

To help with this, there’s a check when the database is upgraded to see whether you’re using the find function. If there is, then an upgrade warning is issued which will cause a message on the first few logins after the upgrade.

getAsPrimaryKey and getAsPrimaryKeyList Functions (Breaking Change)

These two functions were added in 9.0 specifically because they would return an integer (or integer list) in 9.0 and a long (or long list) in version 10.0.

Workflows using these functions should be checked after upgrade to make sure that everything still works.

User Workflow Event Message Types

When a User Workflow Event is run, the workflow can set a message onto the event’s response biskit to send a message back to the user. There’s also a MessageType that lets you choose whether this is an error, warning, or an informational message that appears in the bottom right corner for a few seconds before automatically disappearing.

There are now two more options for the message type. Both of them are considered informational messages, but they require the user to manually click an OK button for the message to go away. The only difference between the two new message types is that one expects the message to be written in plain text and the other in HTML.

JSON and XML Processing Support

We have had support for generating and manipulating JSON files for quite some time. This has now been extended by a workflow function called jslt that can invoke JSON transformations using JSLT (there’s a tutorial for it here).

We now also have tools for processing XML. This is typically used (within Exprodo) for generating XSL-FO files that are in turn used to generate PDF files, or for working with XML-based external APIs.

We now support both XQuery and XSLT.

XSLT and XQuery are both languages for transforming XML documents into other XML documents or other formats.

Our XSLT support is implemented with a new workflow function called xslt.

Our XQuery support is implemented with an extension to a TemplatedTextWorkflowAction. Make the template be an XQuery program, and enable the XQuery option on the action, and it will run XQuery after replacing any data references in the usual way.

You may also enable the FreeMarker templating engine to process the contents of the template. The expansions are performed in the following order:

  1. Regular expansion of references to data
  2. XQuery
  3. FreeMarker

To support this, Templated Text actions have new options to choose whether to enable FreeMarker or XQuery.

For more information about XQuery, see:

New Workflow Functions

The following functions have been added:

User Interface Changes

Selecting biskit from list

Whenever you have a list of biskits displayed, for example in a list report or when you click on a row in a group report, you can click on a biskit in the list to display it.

Previously, the displayed biskit would appear below the list which meant a lot of scrolling up and down if you changed which biskit you clicked on.

It now appears (by default) in a pop-up instead. Click the red X icon to close the pop-up, or press the Escape key to close it.

This behaviour can be modified by pressing the control key (command on an Apple Mac) or by pressing the shift key:

Adding Events and Actions To Workflows

A new “Add” tab now exists in the workflow manager for adding events and actions. The existing “Add Event” and “Add Action” button still remain for now, but will be removed in a future release.

Changes

Primary Keys

All primary keys (integer identifiers) on all biskits have been changed from 32 bits to 64 bits. This is in preparation for a federated system where up to 1023 databases can talk to one another, and data from one database can be exported to be accessed within another.

In this scenario, each database is known as a shard, with each one being given a unique number from 1 to 1023 to identify it.

This includes a new format for the idenitifier within the available 64 bits as follows:

Every database that is upgraded is assigned a randomly selected shard number. Then, every column that stores a primary key is modified by the upgrade in two ways:

This puts it into the new key format, leaving the time part of the key is set to zero.

Whenever you see an identifier in the user interface, you might see it in one of the following formats:

We display the number in this new format to make it easier to read. Otherwise, it’s just a very long number.

The top 11 bits are always zero, making this a 53 bit number. The reason this is done is that JavaScript does not have an integer data type, and uses floating point values for all numbers. Everything up to 53 bits can be represented as a floating point value without losing any resolution.

Primary Key Translation

Primary keys now exist in two forms - there’s a numeric value stored in the database, and a formatted version of them displayed in the user interface. If you need to convert between them for some reason, then you can do so with a new page called “Primary Key Translation” which you can reach either by adding it to your menu or by going to the page #pktranslate

Long Property Subtype

When you define a property in the bakery of type Long, you can now specify a subtype to say that it stores a primary key.

If you do this, then it will mean that the property is rendered like a primary key, using the new format described above.

Also, when adding a long property to a Create Variables Workflow Action or Custom Function Workflow Event, you can now specify whether it is a primary key.

Stored Functions

For anybody who runs direct queries against the database, we now create two functions to assist with primary keys. The UI will now display primary keys in one of the formats described above. But when running queries, you need to use the numeric values.

You can use the #pktranslate page to convert for you, but you can also use one of the new functions:

These convert between the numeric and formatted versions of primary keys. For example:

MariaDB [calpendo]> select formatpk(135003325370560);
+---------------------------+
| formatpk(135003325370560) |
+---------------------------+
| 20200108.125501.36.192    |
+---------------------------+
1 row in set (0.00 sec)

MariaDB [calpendo]> select parsepk('20200108.125501.36.192');
+-----------------------------------+
| parsepk('20200108.125501.36.192') |
+-----------------------------------+
|                   135003325370560 |
+-----------------------------------+
1 row in set (0.00 sec)

Meta Properties

All meta-properties available in conditions now provide for a new property called “shardNumber” which is the current system’s shard number.

Installation

Installation of brand new systems is now done differently. You must now create a database which can be empty, and tell the system which database to use and how to connect to it.

When the system boots, it will check to see if there is a table giving configuration options. If it’s there, it will use it. Otherwise, it will start a web server so that if you point your web browser at it, it shows a configuration page where you can set the options needed to build the system. This includes giving a name for the shard and its shard number.

Templates

Template Conditions

A new option when configuring templates it that you can include conditions that control which users each group within a template will apply to.

Previously you could only choose users by their role, user type or user groups. This now allows you to do this based on any property on a user.

Duplicating Templates Across Resources

It is common when setting up templates that you might want to create identical templates across many different resources all at the same time.

You can now create one template, then click on it in the templates calendar, and one of the items in the pop-up menu is “Duplicate across resources” which will offer you a choice of all resources, and then create identical copies of the selected template across all selected resources.

Subtypes of Misc Biskits

Some biskits now support the creation of subtypes that did not support it before. They are:

This is a part of the Calpendo Enterprise changes, with the idea being that each system within a federation might want to have different flavours of each of the above biskits. So you can now create subtypes of the above biskits, and then control which subtype should be created on your system.

This is done by editing a BiskitDef and setting its pinned property to true. This then means that creating a ServiceProvider (for example) will create whichever BiskitDef subtype of ServiceProvider has been pinned for your system.

You can also specify on a resource which subtype of ResourceUsage should be created for that resource.

Database Dump

The built-in database dump process now also saves stored functions and stored procedures. Previously it would ignore these.

This is necessary now that we have stored functions (parsepk and formatpk) which should be saved in the database dumps. If you create your own stored procedures or functions, they will now be included in dumps created through the user interface.

Group Reports and Collections

Group reports will now let you select a property that contains a set of items. For example, when creating a report on projects, you can show the “users” property. This does not let you add a column to the report that belongs to items within the collection. That is, you cannot add a column for “users.roles”.

Also, when creating a group report, you can choose to aggregate a column. For example, you might display a list of bookings and choose to show the maximum value of the duration, or even the standard deviation.

There are now two new aggregations that have been added:

List Report Sorting

When you have a list report, you can sort the rows by any column by clicking on the header cell for that column. If that column were to contain a biskit-valued property, then the sorting was previously done alphabetically.

However, some biskits define their own custom sort order which may not be alphabetical. When you now sort a list report by clicking on the heading of a biskit-valued column, it will sort the rows using the sorting method defined for the type of biskits in that column.

This will still be alphabetical by default, but sometimes it might be something else.

Hiding Search Bar On Predefined Reports

For a long time, you’ve been able to use the menu editor to add a menu item that will display a report. There’s now an option to hide the search bar when the report is displayed. This is for cases where you want a menu option to display a report, but want to keep things simple for the people who view the report so that there aren’t any options to play with it.

Linking to Biskits From External System

For a long time now, you’ve been able to use URLs that link directly to viewing a particular biskit - so long as you’ve know the ID of the biskit you want to link to. The format is like this:

#ba&type=Calpendo.CalpendoUser&action=view&id=9876543

where the ID will also now accept the new format for primary keys:

#ba&type=Calpendo.CalpendoUser&action=view&id=20200504.123456.981.1

However, we have now added support for links where you don’t know the ID of the biskit you want, but you do have some conditions that could nail down precisely which one it is. You can now put those conditions directly into the URL with a fragment like this:

#ba&type=Calpendo.CalpendoUser&action=view&condition=userIdentity.loginName/eq/'fred'

The format for these conditions also support nested conditions, so the value for the “condition” part of the URL can look like this:

or when creating conditions on bookings:

THe full syntax supported is beyond the scope of these release notes, and will be published.

As well as linking directly to particular biskits by specifyng conditions in a #ba page, you can now also have a URL that will initiate a search page with conditions in a similar way. This works for all search pages. For example, with the general search page, you can initiate a search with a URL fragment like this:

#search&searchType=Attachment&condition=created/gt/20200413.0000/to-the/second

The conditions specified here are the same format as for the #ba page.

It’s also worth pointing out that the “&searchType=Attachment” part is also new. You can now link to the general search page with something that initiates a search for a particular biskit type, regardless of whether you specify conditions.

As well as the general search page, you can also specify conditions on any other page. Other search pages would normally be configured to know what biskit type they’re searching for - or at least know what the base type is.

For example, “#bookingSearch” searches for Bookings. If you have subtypes of bookings configured, then you can search for a particular subtype of Booking. This means you can specify a &searchType to #bookingSearch, but it has to be a subtype of Booking.

Layout Changes

For a long time now, it’s been possible to specify in the layout editor that you want a boolean property to display with a checkbox instead of a drop-down. However, that was ignored when boolean properties were actually rendered.

This is now done properly so booleans will now display with a check box if that’s what you want. You can also display them with two radio buttons instead if you prefer.

Also, in the layout editor you can now ask for biskit-valued properties to be displayed with radio buttons instead of the normal method. This is useful for presenting multi-choice questions, for example with some sort of a training quiz.

Theme Changes

Added support for theme targets “splitter.left” and “splitter.main”

Double Booking Rule Covid-19 Changes

The double booking rule has been extended so that when you set the maximum number of bookings that are allowed at the same time, the number no longer has to be a fixed number set in the rule. Instead, it can now be a variable value picked up by some property path from the booking being made.

This is principally to make it easier to support social distancing in shared offices. Suppose you have an office that is used by multiple people, and you want to set a maxiumum number of occupants who must make a booking to reserve a desk.

You can now add a property to the resource that indicates the capacity of the room represented by that resource. Then, you can have a single rule that will work for many rooms with different capacities by picking up the capacity from the room itself.

This is intended to support new ways of working where people no longer have their own desk with their personal belongings, but they are shared in a permanent hot-desking arrangement with daily cleaning between each use.

Processes (aka Stepped-Edit Process)

Initiating A Stepped-Edit Process

We have supported the ability to edit biskits in steps for a long time now. This means you edit a biskit in a page with a Next and Previous button, and a layout controls which properties are visible on which screen, and a workflow can control the flow of which page you go to when you click on one of those buttons.

It is now much easier to initiate editing in this way.

Wherever there is a “Create” button that would create a biskit for which a process is defined, then the create button can now initiate the process. This also works for Calpendo’s “New project” menu item, and any custom menu item for creating a new biskit.

A “Create Copy” button will also behave the same way.

Previously the only way to create a new biskit via a process was by going to a suitably crafted URL.

Also, any “Edit” button that is there to edit a biskit that has a process defined on it can initiate editing via the process. In this case, you might want the edit session to start from the first step or from the step most recently used. The user can either have the choice, or the person that sets up the process definition can specify what the default should be.

If the process is configured to allow the user to choose, then pressing the Edit button will display a drop-down to choose whether you want to edit without steps, edit from the current step, or edit from the first step.

Choosing The Biskit To Edit

Another change that has occurred is that when a user starts a stepped-edit process via any method, then a Process Workflow Event will be triggered.

Previously, this event would only be triggered by a user clicking a button in a stepped-editor. Now it is triggered at the start of the process as well.

Now a workflow is being called, this provides an opportunity for a workflow to decide the process should edit a particular biskit instead. This might be a pre-existing biskit, or one that the workflow creates and does some configuration on before letting the user process continue. In the workflow, you can tell whether you’re being triggered at the start of a session or not by looking at the button that was pressed or the step performed:

The response biskit on the process event’s output now has an additional property called biskitToEdit, and you should set this to the biskit that should be the object to be edited in the user’s session.

To help you, the Process Workflow Event’s output also now includes a reference to the ProcessDef this relates to. Also, the response’s nextStep property will be populated with the step that is considered to be the next step to perform. This will either be the ProcessDef’s initial step or a step specified by the user’s browser.

Simpler Process Configuration

It’s now easier to set up a process this working as we’ve removed the requirement for a layout to be configured on the initial step.

It’s definitely a good thing to set up a layout, but without one, you can at least confirm that the stepped editor is being used.

Exiting A Process

When using a workflow to control what happens when a user clicks a button on one of the stepped-edit pages, you now have the option of setting the nextPageToken property on the process workflow event’s response.

This allows you to choose that a user clicking one of the buttons can result in the user’s browser switching to any page that you want to select.

Misc Changes

Back to Top # Updates #

10.0.0 March 31, 2020

First released version.

10.0.1 July 9, 2020

Changes

Bug Fixes

Back to Top

10.0.2 August 21, 2020

Changes

Bug Fixes

Back to Top

10.0.3 October 14, 2020

Changes

Bug Fixes

Release Candidate rc2

Release Candidate rc3

Release Candidate rc4

Release Candidate rc5

Release Candidate rc6

Release Candidate rc8

Release Candidate rc9

Release Candidate rc10

Back to Top

10.0.4 November 4, 2020

Changes

Bug Fixes

Back to Top

10.0.5 November 17, 2020

This version incorporates all changes in 9.0.68

Changes

Bug Fixes

Back to Top

10.0.6 November 22, 2020

Changes

Bug Fixes

10.0.7 December 2, 2020

This includes everything in version 9.0.69

Changes

Bug Fixes

RC2

RC3

10.0.8 December 8, 2020

This includes everything in version 9.0.70

Changes

Bug Fixes

10.0.9 December 17, 2020

This includes everything in version 9.0.71

Changes

Bug Fixes

10.0.10 December 22, 2020

This includes everything in version 9.0.72

Changes

Bug Fixes

Back to Top

10.0.11 December 23, 2020

This includes everything in version 9.0.72

Changes

Bug Fixes

Back to Top

10.0.12 January 4, 2021

This includes everything in version 9.0.73

Changes

Bug Fixes

Back to Top

10.0.13 January 6, 2021

Changes

Bug Fixes

Back to Top

10.0.14 January 8, 2021

Bug Fixes

Back to Top

10.0.15 January 12, 2021

This includes everything in version 9.0.74

Changes

Bug Fixes

Back to Top

10.0.16 January 21, 2021

Changes

Bug Fixes

Back to Top

10.0.17 January 26, 2021

This includes everything in version 9.0.75

Changes

Bug Fixes

Back to Top

10.0.18 February 1, 2021

This includes everything in version 9.0.76

Changes

Bug Fixes

Back to Top

10.0.19 February 5, 2021

This includes everything in version 9.0.77

Changes

Bug Fixes

Back to Top

10.0.20 February 10, 2021

Changes

Bug Fixes

Back to Top

10.0.21 February 15, 2021

This includes everything in version 9.0.78

Changes

Bug Fixes

Back to Top

10.0.22 February 17, 2021

This includes everything in version 9.0.79

Changes

Bug Fixes

Back to Top

10.0.23 February 23, 2021

Changes

Bug Fixes

Back to Top

10.0.24 March 3, 2021

This includes everything in version 9.0.80

Changes

Bug Fixes

Back to Top

10.0.25 March 5, 2021

This includes everything in version 9.0.81

Changes

Bug Fixes

Back to Top

10.0.26 March 11, 2021

Changes

Bug Fixes

Back to Top

10.0.27 March 17, 2021

Bug Fixes

Back to Top

10.0.28 March 23, 2021

Changes

Bug Fixes

Back to Top

10.0.29 April 6, 2021

This includes everything in version 9.0.83

Changes

Bug Fixes

Back to Top

10.0.30 April 25, 2021

This includes everything in version 9.0.84 and has no other changes.

Back to Top

10.0.31 May 6, 2021

Bug Fixes

Back to Top

10.0.32 May 24, 2021

Changes

Bug Fixes

Back to Top

10.0.33 May 31, 2021

This includes everything in version 9.0.85.

Changes

Bug Fixes

Back to Top

10.0.34 June 7, 2021

Changes

Bug Fixes

Back to Top

10.0.35 June 18, 2021

This includes everything in version 9.0.86.

Changes

Bug Fixes

Back to Top

10.0.36 June 21, 2021

Bug Fixes

Back to Top

10.0.37 June 25, 2021

Bug Fixes

Back to Top

10.0.38 June 28, 2021

Bug Fixes

Back to Top

10.0.39 July 9, 2021

This includes everything in 9.0.87 and no other changes.

Back to Top

10.0.40 August 3, 2021

Bug Fixes

Back to Top

10.0.41 August 25, 2021

This includes everything in 9.0.88 and 9.0.89 (all listed below).

Changes from 9.0.88 and 9.0.89

Changes

Bug Fixes From 9.0.88

Back to Top

10.0.42 September 16, 2021

This includes everything in 9.0.90 (all listed below).

Changes

Bug Fixes from 9.0.90

Back to Top