Small steps with big feet

Introduction

With the 4.2 release coming close (somewhere beginning next week), I feel is time to write about one the new nice features of this particular release.

As a software engineer, I’m always eager to try out a new release when of one of the products/frameworks/… I use on a daily basis hits the shelves. I’ve also learned that I’m not alone on this matter and that it is a common thing among developers to have this burning itch to try out new stuff. But with every new release it’s also always keeping fingers crossed and hoping that everything stays working as it was without actually having the system blow up in your face.

With a new jBPM release, there are three ‘change-areas’ which could disrupt your current system:

* API changes. As I wrote a few months ago, starting from jBPM 4.0, there is a new API. Our policy is is to keep changes to the API (extremely) limited and deprecate any to-be-changed operations for two or three releases before actually replacing them completely. This means you have plenty of time do execute the change. So no blow-up on this front.

* JPDL schema changes. A New jBPM release often means additional features, which could lead to jPDL schema changes that make existing process definitions incompatible with the latest release. This issue has been addressed in the upcoming 4.2 release: old process definitions will remain backwards compatible with newer releases of the engine. Not a single change to your precious process definitions will be needed when upgrading your jBPM4 installation. So for example process definitions coded in 4.1 style will be deployable to a 4.7 engine. For more info, see the Jira issue. So again, no fear for blow-ups here.

* Database changes. Since jBPM uses a relational database as a persistence mechanism for processes, tasks, audit history, etc… new features in the engine can lead to changes in the database. We try to keep these changes as limited as possible, but sometimes they are simply needed to evolve. Of course, with every release we indicate which changes need to be done to the database, but if you skip a few releases before you upgrade this tends to become cumbersome.

It’s this last point which I want to demonstrate in this post. The 4.2 and every subsequent release have a new mechanism on board to determine the current version of the database schema and update it to the fresh release automatically.

Setting up the ‘old’ data

We’ll start by downloading and unzipping an older release of jBPM, jBPM4.1:

[jbarrez@jenova 4.2_upgrade]$ unzip jbpm-4.1.zip

We’ll now create the jBPM schema using one of the install task shipped in the distribution. Since I’m using a MySQL database, I first need to update the jdbc properties that the install script will be using:

[jbarrez@jenova 4.2_upgrade]$ your-favourite-editor jbpm-4.1/install/jdbc/mysql.properties

and change it to my local database:

We can now create the schema in my local database (notice how we just need to add the ‘database‘ parameter, the script will then take the connection properties from the mysql.properties file we’ve just edited):

[jbarrez@jenova 4.2_upgrade]$ cd jbpm-4.1/install/
[jbarrez@jenova install]$ ant create.jbpm.schema -Ddatabase=mysql
Buildfile: build.xml
 [echo] database......... mysql
 [echo] tx............... standalone
 [echo] mail.smtp.host... localhost
 [echo] current dir = /home/jbarrez/dev/blog/4.2_upgrade/jbpm-4.1

create.jbpm.schema:
 [echo] creating jbpm schema in db jdbc:mysql://localhost:3306/jbpm
Executing resource: /home/jbarrez/dev/blog/4.2_upgrade/jbpm-4.1/install/src/db/jbpm.mysql.create.sql
79 of 79 SQL statements executed successfully
BUILD SUCCESSFUL Total time: 3 seconds

Which gives us a fresh jBPM 4.1 schema in our database:

We now use a basic ‘Hello World’ process definition:

<process name="helloWorld" xmlns="http://jbpm.org/4.0/jpdl">
  <start>
    <transition to="display hello world" />
  </start>
  <custom name="display hello world" class="be.jorambarrez...DisplayHelloWorld">
    <transition to="theEnd" />
  </custom>
  <end name="theEnd" />
</process>

… to start a process instance through the jBPM API:

public static void main(String[] args) {

ProcessEngine processEngine = new Configuration().buildProcessEngine();
RepositoryService repoService = processEngine.getRepositoryService();
ExecutionService executionService = processEngine.getExecutionService();

NewDeployment deployment = repo.createDeployment()
.addResourceFromClasspath("process.jpdl.xml");
deployment.deploy();
executionService.startProcessInstanceByKey("helloWorld");
 }

The debug log shows us the actual execution trace, where our ‘Hello World’ message is displayed between the engine logs.

17:53:15,219 FIN | [ProcessDefinitionImpl] creating new execution for process 'helloWorld'
17:53:15,224 FIN | [DefaultIdGenerator] generated execution id helloWorld.1
17:53:15,229 FIN | [ExecuteActivity] executing activity(25788693)
17:53:15,229 FIN | [ExecuteActivity] executing activity(display hello world)

Hello World!

17:53:15,230 FIN | [ExecuteActivity] executing activity(theEnd)
17:53:15,230 FIN | [ExecutionImpl] execution[helloWorld.1] ends with state ended

Upgrading in one minute

Suppose now that we take this process to production. Life is happy and peace. But then comes a new jBPM release. The developer hands start to itch again … So we take the jBPM 4.2 distribution and unzip it:

[jbarrez@jenova 4.2_upgrade]$ unzip jbpm-4.2.zip

And we change the dependencies of our little Hello World project, by either replacing the 4.1 jar with the 4.2 jar in the build path. Or if you are using Maven:

<dependency>
<groupId>org.jbpm.jbpm4</groupId>
<artifactId>jbpm-jpdl</artifactId>
<version>4.2</version>
</dependency>

<repositories>
<repository>
<id>jboss</id>
<url>http://repository.jboss.com/maven2</url>
</repository>
</repositories>

Of course, since we are adventurous developers, we don’t care about upgrading and just run our Hello World process with the new jBPM version. However, the new upgrade check will now kick in:

Exception in thread "main" org.jbpm.api.JbpmException: jBPM DB schema not in sync
 with library version: no JBPM4_PROPERTIES table.   Run the upgrade target
 first in the install tool.
 at org.jbpm.pvm.internal.cmd.CheckDbCmd.execute(CheckDbCmd.java:54)
 at org.jbpm.pvm.internal.svc.DefaultCommandService.execute(DefaultCommandService.java:42)
 at org.jbpm.pvm.internal.tx.StandardTransactionInterceptor.execute(StandardTransactionInterceptor.java:54)
 at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.executeInNewEnvironment(EnvironmentInterceptor.java:53)
 at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.execute(EnvironmentInterceptor.java:40)
 at org.jbpm.pvm.internal.svc.RetryInterceptor.execute(RetryInterceptor.java:55)
 at org.jbpm.pvm.internal.svc.SkipInterceptor.execute(SkipInterceptor.java:43)
 at org.jbpm.pvm.internal.cfg.ProcessEngineImpl.checkDb(ProcessEngineImpl.java:177)
 at org.jbpm.pvm.internal.cfg.ProcessEngineImpl.buildProcessEngine(ProcessEngineImpl.java:170)
 at org.jbpm.api.Configuration.buildProcessEngine(Configuration.java:140)
 at be.jorambarrez.jbpm4.upgrade.Main.main(Main.java:15)

As you can see in the stacktrace, during the creating of the jBPM ProcessEngine a check on the database schema will take place (tech detail: it uses the JBPM4_PROPERTIES table and some other checks to do this). And since the exception is thrown during ProcessEngine creation, this also stops us from executing processes in a old schema – which is a good thing.

So we follow the instructions of the exception above and run the upgrade script. Don’t forget to change the database properties (as we did above).

[jbarrez@jenova install]$ ant upgrade.jbpm.schema -Ddatabase=mysql
Buildfile: build.xml
 [echo] database......... mysql
 [echo] tx............... standalone
 [echo] mail.smtp.host... localhost

upgrade.jbpm.schema:
 [echo] upgrading jbpm schema...

 ... // *snip* Hibernate setup etc

 12:30:02,501 INF | [DbHelper] --- Executing DB Commands -------------------------
 12:30:02,501 INF | [DbHelper] create table JBPM4_PROPERTY (
 KEY_ varchar(255) not null,
 VERSION_ integer not null,
 VALUE_ varchar(255),
 primary key (KEY_)
 ) type=InnoDB
 12:30:02,544 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,544 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HACTI
 12:30:02,564 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,564 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HPROCI
 12:30:02,578 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,578 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HVAR
 12:30:02,595 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,596 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HTASK
 12:30:02,610 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,611 INF | [DbHelper] create index IDX_HSUPERT_SUB on JBPM4_HIST_TASK (SUPERTASK_)
 12:30:02,625 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,736 INF | [PropertyImpl] nextDbid is initialized to 4
 12:30:02,740 INF | [Upgrade] jBPM DB upgrade completed successfully. 
 12:30:02,740 INF | [SessionFactoryImpl] closing
 12:30:02,741 INF | [DriverManagerConnectionProvider] cleaning up connection pool: jdbc:mysql://localhost:3306/jbpm
BUILD SUCCESSFUL Total time: 4 seconds

The output of the script show us that some upgrades were done: a new table was created, some indexes were altered and some colum values were updated.

When we now retry running the Hello World process, everything works fine:

12:41:19,275 FIN | [DatabaseIdComposer] generated execution id helloWorld.10010
12:41:19,279 FIN | [ExecuteActivity] executing activity(28678425)
12:41:19,279 FIN | [ExecuteActivity] executing activity(display hello world)
12:41:19,280 FIN | [ExecuteActivity] executing activity(theEnd)
12:41:19,280 FIN | [ExecutionImpl] execution[helloWorld.10010] ends with state ended
Hello World!

And if you pay close attention to the logs, you’ll see that the schema check is OK now:

12:41:18,687 INF | [CheckDbCmd] jBPM version info: library[4.2], schema[4.2]

NOTE: if you got a grumpy DBA: the jBPM distro also ships with plain .sql files to use for upgrading the schema, which you can hand over to him/her.

Conclusion

And that’s it, folks. We’ve got API stability, backwards schema compatibility and as I just showed you, upgrading the database takes only a minute of your time… leaving more time for you to actually code business processes!

QA

To finish the post, I just want to show you a screenshot from our Hudson continuous integration job. Since the upgrade feature is extremely important for us, we test it against all our supported databases and JDK versions. The test job that is executed is actually very close related to the explanation of the previous section. With every new release, this configuration matrix will continue to grow. Which means that in every future release are certain of backwards compatibility and upgradeability.

Happy jBPM coding!