Posts in Category: jBPM

Alive and Kicking!

You may have seen the Open letter to the jBPM community explaining that Tom and I step down from the jBPM project. We just want to let you know that we’re alive and kicking. We’re building a new BPM platform that’s architected for new IT requirements. It will be Apache licensed and it will run BPMN 2.0 natively. I can assure you …. exciting times are ahead!

That’s all we can share at this point. Stay tuned for more updates on ‘the what and the how’ very soon!

Book review: jBPM Developer Guide by Mauricio Salatino

I’ve managed to give the jBPM Developer Guide by Mauricio Salatino, which was sent to me less than a mont ago by the people of Packt, a thorough read. Reviewing a book is not a simple task, certainly not when the topic is quite familiar. So I started every chapter by writing down some notes of what I would expect to find, and checked while reading if the content could cover my expectations.

Since many people will simply skip to the conclusion part (at least, that’s what I do when I read a book review), I’ll start off with The Short Review.

The Short Review

Basically a book review needs to answer the following question:

Would you recommend the book?

Yes. I would certainly recommend the book to any novice jBPM user. The book covers practically all jBPM3 concepts and gently builds up the complexity of the content. Having this book a few years ago when I started out with jBPM, would have saved me tremendous time. Granted, the book has some minor flaws (see The Long Review), but it currently is the best guide and reference when you want to start using jBPM. period.

The Long Review

In the following sections, I will summarize for each chapters the notes I’ve taken while reading the book.

The book is published by “Packt Publishing”. The nice thing about Packt is that they are generally fast to publish books about new and exciting, mostly open source technologies. On top of that, they also invest a part of the profit back into the project, which is a noble deed if you ask me.

In general

  • This book is about jBPM3, not jBPM4. One might think this is a bad thing, but do remember that jBPM3 is extremely stable and is part of the JBoss SOA-Platform for many years to come (today jBPM3 is the only official supported version by Red Hat – these things take quite a time). People that are familiar with jBPM3, will have no problems to make the jump to jBPM4.
  • A general nuisance is the language usage throughout the book: although the sentences are syntactical correct, they often not feel ‘right’. Others have already commented on this fact, so I’m not going into it here. I don’t blame the author: he shows a great amount of enthusiasm and jBPM knowledge, but with a little more editorial reviewing, this book could have gotten that little extra edge.
  • The book is great for jBPM starters, no single doubt about that. However, I would have liked to see more ‘best practices’ and advanced configurations (JMS/SOA/etc.). Let’s hope for a next jBPM book that covers those topics 😉

Preface and Chapter 1: Why developers need BPM

This chapter is a bit overwhelming for beginners. The author tries very hard to convince people of the rightness of open source software (which is perfectly valid ), but it’s too heavy as a starter chapter. This chapter would have been a good appendix, I guess.

Oh yeah, I don’t necessarily agree with Mauricio’s definition of a ‘BPM suite’. You’ll understand it when you read it 😉

Chapter 2: jBPM for Developers

Good introduction to some important concepts like wait states, nodes, process definition vs instance, etc. The downside of this chapter is that it tries to explain these concepts by building a simple state machine in Java code. In my opinion, 99% of the jBPM starters will find this too overwhelming. I found it fun to read, but then again my day job is to build such “state machines” 😉

Luckily, it goes uphill from here. My advice would be to read the chapters 1/2 after starting from chapter 3, and only giving them a quick read either way.

Chapter 3: Setting up Our Tools

Mandatory chapter for every beginner. I really liked the fact that nothing is “taken for granted’, as tools like SVN and Maven are also covered with some text. Many technical books assume that this is common knowledge, but experience has taught me this certainly is not the case. I also really liked the ‘community’ take on things in this chapter. The author explains and demonstrates that by knowing maven, svn, etc. every one can contribute to open source ecosystem. As a core jBPM dev, I was happy to find these encouragements.

This chapter demonstrates in depth how tools (Maven, Eclipse, etc.) need to be configured to get the maximum out of your jBPM experience. Also the jBPM project structure is touched, which is in reality a true knowledge advantage I’ve not yet seen in many jBPM users.

A small side remark (has no influence on jBPM novices, ofc.): the history of jBPM is a bit skewed, which is a bit of a pity if you’re closely involved with the project (eg. jBPM was orginally not created by JBoss)

Chapter 4: jPDL language

When I was reading this chapter, I noted down “To the point! This is the content you need”! I can’t express it better than that. A must-read chapter for every jBPM adept.

I was delighted too see that the author often explains functionality by referring to snippets in the source code. In my opinion, this is the way open source tech books should be. When I was still a jBPM consultant, I often looked at the source code to learn how corner cases where handled, or what exactly a certain operation does, etc. As Ronald often states on the jBPM User Forum: the best documentation is the source.

Chapter 5: Getting Hands Dirty with jPDL

In this chapter, the topics explained in the previous sections are used in a real project. The use case is interesting and has great didactical value. I had some reservations when I read the use case, and noted down some questions on how the author would tackle them (if they were tackled at all). I was happy to find them all answered by the end of the chapter. Perhaps a small remark could be here that the jBPM configuration isn’t showed in this chapter. This could be a bit confusing for some starters, since it involves some “magic” that is easy to explain.

Chapter 6: Peristence

Really thorough and detailed explanation of jBPM persistence. Excellent transaction explanation and great diagrams that explains what happens in which situation (automatic nodes, wait states, when updates happen, etc.). Really every aspect is covered (for example even Hibernate caching is touched), nice!

Imo, perphaps the best chapter of the book.

Chapter 7: Human Tasks

Perfect follow-up for the previous chapter. Good, detailed content about probably the most important construct of any BPM engine. I also really like the webapp approach at the end of the chapter: this is the stuff why people buy the book.

Chapter 8: Persistence and Human Tasks in the Real World

This chapter perfectly aligns with the previous Human Task sections: good and very detailed content.

Chapter 9: Handling information

Great explanation about variable usage in jBPM. The content is good for starters and should get you going quite far. But in my opinion, some things should have gotten more attention, since variables are an area in which many mistakes are made in real life projects. Some topics that could have gotten more love: hibernate objects as variable (+ why !), serializable (what about synchronisation problems?), business keys for process instances (why do you need them).

Chapter 10: Advanced features

This chapter definitely covers some advanced features. It even managed to surprise me: I had totally forgotten about the superstate indirections using “../” in the node names (altough I’ve used it a lot on a certain project) and the book mentioned this little detail! Also start tasks were covered – nice!

Some topics which I have needed a lot in my past projects weren’t covered (but this is really advanced stuff, and definitely not for startes): subprocess binding, more database schema focus, e-mail node and custom HQL queries using the jBPM objects. So altough I really liked the chapter, it could have gotten extra points when dealing with those topics.

Chapter 11: Advanced topics in practice

Not to miss chapter since here asynchronous execution is explained. The books goes really deep in the nitty gritty details of async executions – which I like. However in my opinion, the book doesn’t stress enough the crucial importance of the async functionality enough. But after reading this chapter, you’ll definitely be able to understand and try it for yourself.

Chapter 12: Going Enterprise

Explanation of what the jbpm-enterprise jar offers. Some good practices are states here, like the EJB intermediate layer with ids (jBPM4, anyone?). Also remote client usage is covered, together with Timers and job (VERY good explanation btw).

As my conclusion already can be found in the beginning of this post, I’ll repeat here the one-sentence summary. If you are starting out with jBPM, there is no better book or guide to get you guys. Highly recommended!

Updated hello world tutorial for jBPM 4.3

While checking my tweets this weekend, I found out that Niklas Gustavsson updated the jBPM 4.0 Hello World example to jBPM 4.3. Altough not complicated if you know jBPM and Maven, I’m sure many people will find it helpful. Since my Hello World article is still one of my top posts (visitor-wise), I decided to post here how you can get Niklas’ work in your Eclipse (thanks Niklas!).

You can find the source code on git:

http://github.com/protocol7/jbmp4-helloworld

(small remark, you’ve misspelled jbpm, Niklas ;-))

Clone the repository locally:

git clone git://github.com/protocol7/jbmp4-helloworld.git

And prepare the project for your IDE (I’m using Eclipse):

cd jbmp4-helloworld

mvn eclipse:eclipse

You can now import the project into Eclipse (File -> import -> Maven project). Note that you need a Maven plugin, such as m2Eclipse. Find the Main.java class and run it.

The content of my previous post about this example are still valid today (hurrah for API stability :-)), so please read further there.

jBPM Screencast: from the drawing board straight to executable BPMN 2.0

In my last blogpost about BPMN 2.0, I already showed how BPMN 2.0 processes could be executed natively on the jBPM4 engine. Of course, the main advantage of BPMN is that we have a standardized and industry-accepted language to describe business processes – executable and non-executable ones. This also means our story is but complete if we have the modelling part covered.

As described in many previous posts, we’re closely collaborating with the guys from Signavio. They have created a fantastic web based BPMN editor and develop the core of the editor in open source.

In this screencast, I show how a BPMN 2.0 process created with the Signavio editor can be exported, dropped into Eclipse and deployed to the jBPM engine – without any change to the process file!

Major milestone:

BPMN 2.0 processes modelled with Signavio

can now directly be executed on the jBPM4 engine

The process I’m using for the screencast is shown below (it is an advanced process we use to test our Inclusive Gateway implementation). Also note that I’m using the cool new embeddable process model feature in Signavio – try to zoom in and out or click on the header :-).

Inclusive gateway example

The example is a bit abstract, but shows clearly how well our BMN 2.0 implementation copes with something as advanced as the inclusive gateway. By the way, many kudos go out to the jBPM PVM (Process Virtual Machine) created by Tom, which made the implementation a breeze. I haven’t found anything in the BPMN 2.0 spec yet which I could throw against it and make it sweat.

All right, time for the screencast. In the screencast, I’m using the online version of Signavio. We’re currently discussion with the Signavio team when and how we’ll ship a BPMN 2.0 editor with the jBPM distribution. Expect some news on this front for the 4.4 or 4.5 release! You can however, try out the fully functional modeller already for a trial period.

I have deliberately been sparse with information in this post, because I’ve added audio to the screencast (and yes, I do have a serious cold :p), so don’t forgot to pump up your speakers as you’ll miss a lot of the fun otherwise. Click on the image below to view the screencast.

jBPM talk @ JDD09 online on Youtube

I just found out that the friendly people of Proidea (organisation behind the Java Developers Day or ‘JDD’ in short) have put the jBPM talk online which we did back in October.

The talk is divided into four parts, all nicely organized in a playlist on the Proidea Youtube channel.

Pictures of the talk (and the rest of the event) can be found here.

Enjoy!

Native BPMN 2.0 execution with the freshly released jBPM 4.3

Happy 2010 to all the folks who read this blog! I’m extremely pleased to start this  year with a bang by announcing that jBPM 4.3 has been released!

As we announced last month, the major achievement of this release is the native BPMN 2.0 execution on top of the PVM. See the announcement itself for more information, examples and a shiny movie.

The main goal of the first release of our BPMN 2.0 implementation was to implement the ‘basic’ part of the specification. Using the basic constructs, one should already get quite an impression how BPMN 2.0 can be used in practice with jBPM. We’re extremely proud that we’ve managed to include the BPMN 2.0 implementation as an integral part of the jBPM framework. This means that no database or API changes are required when you want to start using BPMN 2.0 with jBPM. Heck, enabling BPMN 2.0 in jBPM requires one single line of configuration. Just check the example of the previous BPMN 2.0 blogpost to see all this sweetness.

The complete distribution zip file can be found on http://sourceforge.net/projects/jbpm/files/

This distribution includes quite a few examples to get you started. The developers guide that is shipped with the distribution also contains a whole new chapter about how to implement BPMN 2.0 processes using jBPM 4.3. The content of the documentation is detailed and people with no prior BPMN 2.0 knowledge should be able to understand it without much problems. See the subdirectory /doc/devguide to find this guide. Or you can take a look online(link to my own version, online doc on docs.jboss.org isn’t yet updated due to past holiday season – but this can change any moment now).

To finish up, let me quote one of our prominent Community Users, Bernd Rücker:

“2010: the year of BPMN 2.0”

Of course, BPMN 2.0 isn’t the only thing we’ve been doing the last two months. Check our JIRA for a complete changelog. Most noteworthy new features are the ejb and jms invocation for JPDL – features that are easy to leverage for a future jBPM BPMN 2.0 release …

As always, thanks for reading!

Upcoming review ‘jBPM Developer Guide’ by Mauricio Salatino

Building an open source framework sure is a rewarding experience. You see community people logging issues, contributing patches, answering forum posts, joining the community days, etc.

And sometimes there is a crazy enthusiast user who sacrifices *a lot* of his free time to write a jBPM book. Heck, writing a jBPM book has been a long-lived dream for me … but some things always seem to come between that dream and reality (you know, like building a BPMN 2.0 implementation on top of jBPM). But that’s probably just an excuse, I know.

Writing a book takes a lot of time and effort and it’s an adventure that’s not to be taken light-heartedly. So many, many, many kudos to Mauricio Salatino for writing the just released “jBPM Developers Guide”! Just click on the book cover on the left to get more information or to order your own copy.

The people of Packt publishing were so kind to send me a review copy of this book, which I received this morning (see picture below). So expect a review coming up soon (I’m going to give it a thorough reading).

jBPM goes BPMN!

For those who follow the jBPM internal discussions a bit, the marriage of jBPM and BPMN will come as no surprise. Since this summer we’re coding away at a native BPMN2 implementation on top of our Process Virtual Machine (PVM), and you might have seen tweets or forum posts passing by where the term ‘BPMN’ pops up. However, the only proof were some classes in our SVN trunk …. until now!

In this post I’ll give you a sneak peek of our current BPMN2 execution effort which will be documented, QA’d and incorporated in the jBPM 4.3 release (scheduled January 1st). Many thanks go out to our community members Ronald Van Kuijk and Bernd Rücker, who have contributed a significant amount of  ideas and code (and in fact, still are contributing to the BPMN2 implementation now as we speak). This is open-source power at its fullest…

Happy times indeed!

What is BPMN2?

Basically, the Business Process Modeling Notation (BPMN) started out a pure graphical notation standard for business processes, maintained by the Object Management Group (OMG). Version 2.0, which currently is in beta, adds execution semantics to the specification and this is of course where it gets interesting for jBPM.

The primary benefit of BPMN2 is that it is a standard accepted by the IT industry, which means that process models become portable (graphical and execution-wise) across process engines such as jBPM. Since process executions are the raison-d-être of jBPM, it is only natural we are now investing in BPMN2. People who are familiar with JPDL (the current native language of jBPM) will have generally no difficulties in learning the BPMN2 language, as many constructs and concepts are shared. In fact, from a high-level point of view, BPMN2 and JPDL are in concept solving the same problem (which is a biiiiig difference with BPEL – but that’s a story I’ll let other tell).

Of course, this is just the nutshell explanation. You can find a lot more information in our wiki.

Enough talking … show me how it works!

The business process we’re going to implement looks as follows (click to enlarge – created using the wonderful editor from our friends at Signavio).

You might recognize this example, since we’ve also implemented it in JPDL as an example in our distribution.

The business process is simple: an employee can start a new process and make a request for a certain amount of vacation days. After the request task has been completed, the manager will find a verification task in its tasklist. The Manager can now decide to accept or reject this request. Depending on the outcome (that’s the little rhombus on the outgoing transitions, or better said ‘sequence flow’ as it is called in BPMN. It means that there is  a condition that must be true to follow it), a rejection message is send or the process ends.

As is the case with JPDL, BPMN2 uses an XML language to define the process definition. Our business process looks like this in BPMN2 XML:


<?xml version="1.0" encoding="UTF-8"?>
<definitions id="vacationRequest"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://schema.omg.org/spec/BPMN/2.0 BPMN20.xsd"
 xmlns="http://schema.omg.org/spec/BPMN/2.0"
 typeLanguage="http://www.w3.org/2001/XMLSchema"
 expressionLanguage="http://www.w3.org/1999/XPath"
 targetNamespace="http://jbpm.org/example/bpmn2/vacation_request"
 xmlns:jbpm="http://jbpm.org/bpmn2">

 <process id="vacationRequestProcess" name="BPMN2 Example process">

   <startEvent id="start" />

   <sequenceFlow id="flow1" name="fromStartToRequestVacation"
     sourceRef="start" targetRef="requestVacation" />

   <userTask id="requestVacation" name="Request Vacation"
       implementation="other">
     <potentialOwner resourceRef="user" />
     <rendering id="requestForm">
       <jbpm:form>request_vacation.ftl</jbpm:form>
     </rendering>
   </userTask>

   <sequenceFlow id="flow2"
     name="fromRequestVacationToVerifyRequest" sourceRef="requestVacation"
     targetRef="verifyRequest" />

   <userTask id="verifyRequest" name="Verify Request"
       implementation="other">
     <potentialOwner resourceRef="manager" />
     <rendering id="verifyForm">
       <jbpm:form>verify_request.ftl</jbpm:form>
     </rendering>
   </userTask>

   <sequenceFlow id="flow3" name="fromVerifyRequestToEnd"
       sourceRef="verifyRequest" targetRef="theEnd">
     <conditionExpression xsi:type="tFormalExpression">
        ${verificationResult == 'OK'}
     </conditionExpression>
   </sequenceFlow>

   <sequenceFlow id="flow4"
       name="fromVerifyRequestToSendRejectionMessage" sourceRef="verifyRequest"
       targetRef="sendRejectionMessage">
     <conditionExpression xsi:type="tFormalExpression">
        ${verificationResult == 'Not OK'}
     </conditionExpression>
   </sequenceFlow>

   <scriptTask id="sendRejectionMessage" name="Send rejection Message"
       scriptLanguage="bsh">
     <script>
       <![CDATA[System.out.println("Vacation request refused!");]]>
     </script>
   </scriptTask>

   <sequenceFlow id="flow5"
     name="fromSendRejectionMessageToEnd" sourceRef="sendRejectionMessage"
     targetRef="theEnd" />

   <endEvent id="theEnd" name="End" />

 </process>
</definitions>

Most of the constructs are straightforward, certainly if you’re coming from a JPDL background. Do note that the Resource construct isn’t correctly implemented (yet), but that should be solved for the 4.3 release. Also note that you already can add taskforms to your user tasks (see line 33-35 for example).

Also interesting to note is the way how the outcome of the user task is used is for guiding the flow through the process (line 41 and 49). We could have used an exclusive gateway to do the same (which is also already implemented), but we’re using formal expressions on the outgoing sequence flows coming from the user task, just to show you we can.

That’s cool … but I’m a Java developer!

Rest assured: everything that makes jBPM a blessing to use is here to stay. Tom has put a tremendous effort in the jBPM PVM and we now are reaping the fruits of his work. BPMN2 is implemented on top of the PVM, which means that many things come for free: database persistency, database transactions, the service and query API’s, etc. it all just works.

So, when we want to test a business process, we use the same Java approach which we always had with jBPM:

public class VacationProcessTest extends JbpmTestCase {

  protected void setUp() throws Exception {
    super.setUp();

    NewDeployment deployment = repositoryService.createDeployment();

    deployment.addResourceFromClasspath("vacationrequest.bpmn.xml")
    String deployId = deployment.deploy();
    registerDeployment(deployId);

  }

  public void testRequestRejected() {

    ProcessInstance pi = executionService.startProcessInstanceByKey("vacationRequest");

    // After process start, peter (employee)
    // should be a candidate for the task
    Task requestTasktask = taskService.createTaskQuery()
              .candidate("peter").uniqueResult();
    assertNotNull(requestTasktask);

    // After this task is completed,
    // a manager (eg alex) is be able to see a 'verify' task
    TaskQuery verifyTaskQuery = taskService.createTaskQuery()
                                           .candidate("alex");
    assertNull(verifyTaskQuery.uniqueResult());
    taskService.completeTask(requestTasktask.getId());

    Task verifyTask = verifyTaskQuery.uniqueResult();
    assertNotNull(verifyTask);

    // When completing the verification task,
    // we also need to store the result of the decision
    // (will be done through the taskform normally)
    Map<String, Object> vars = new HashMap<String, Object>();
    vars.put("verificationResult", "not OK");
    taskService.completeTask(verifyTask.getId(), vars);

    // With a rejection, we should see
    // the 'sendRejectionMessage' task in the history log
    HistoryActivityInstance hai = historyService.createHistoryActivityInstanceQuery()
      .activityName("sendRejectionMessage").uniqueResult();
    assertNotNull(hai);
    assertProcessInstanceEnded(pi);

  }

}

So what you’re seeing here is

  • Testing BPMN2 business processes in pure Java code that everybody understands
  • in a way which is exactly the same as for JPDL
  • and using the very same API which we are all familiar with
  • and using the very same database schema as before!

What’s there not to like? In my opinion, the argument of the same API is really really really powerful. The PVM is showing its power at its fullest! To configure your jBPM installation for BPMN2, you just have to add this line to your jbpm.cfg.xml file:

<import resource="jbpm.bpmn.cfg.xml" />

Conclusion: using BPMN2 processes in your application with jBPM is extremely easy. Everything you know about embedding and configuring jBPM is just as before … except that you’re now using a portable and standard process language.

What else?

A lot. Just check our wiki to see what’s planned for the 4.3 release. When talking in pure BPMN2 constructs, these are all planned for next month: Exclusive Gateway (XOR), Parallel gateway (AND), Service Task (Java based), Manual Task, Receive Task (Java only), Script Task, User Task, None start event, None end event, Terminate end event, Sequence Flow, Conditional Sequence flow, Formal expression, Interface (Java), Resource + ResourceParameter. More is possible, if we go to bed a bit later than usual 😉

To finish this sneak peek, here is a screencast of our jBPM console executing the BPMN2 example process which is described above. In this screencast, you’ll see Peter (the employee) requesting 5 days of vacation using our business process. After he finishes the request task, he logs out. Mike, a manager, now logs in and finds the vacation request in his task list. He rejects the request since there is too much work. Peter will now be notified with a rejection message (for this demo, just a printout to the console). The forms you see are using the same taskform mechanism which I described earlier.

The last part of the screencast shows that the reporting functionality, which was already available for JPDL, also works for the BPMN2 processes. Even better, since the PVM defines a shared database schema, reports can aggregate data based on JPDL and BPMN2 processes without any problem!

Until next post!

Joram

New feature in jBPM 4.2: Auto-upgrade your jBPM installation

Introduction

With the 4.2 release coming close (somewhere beginning next week), I feel is time to write about one the new nice features of this particular release.

As a software engineer, I’m always eager to try out a new release when of one of the products/frameworks/… I use on a daily basis hits the shelves. I’ve also learned that I’m not alone on this matter and that it is a common thing among developers to have this burning itch to try out new stuff. But with every new release it’s also always keeping fingers crossed and hoping that everything stays working as it was without actually having the system blow up in your face.

With a new jBPM release, there are three ‘change-areas’ which could disrupt your current system:

* API changes. As I wrote a few months ago, starting from jBPM 4.0, there is a new API. Our policy is is to keep changes to the API (extremely) limited and deprecate any to-be-changed operations for two or three releases before actually replacing them completely. This means you have plenty of time do execute the change. So no blow-up on this front.

* JPDL schema changes. A New jBPM release often means additional features, which could lead to jPDL schema changes that make existing process definitions incompatible with the latest release. This issue has been addressed in the upcoming 4.2 release: old process definitions will remain backwards compatible with newer releases of the engine. Not a single change to your precious process definitions will be needed when upgrading your jBPM4 installation. So for example process definitions coded in 4.1 style will be deployable to a 4.7 engine. For more info, see the Jira issue. So again, no fear for blow-ups here.

* Database changes. Since jBPM uses a relational database as a persistence mechanism for processes, tasks, audit history, etc… new features in the engine can lead to changes in the database. We try to keep these changes as limited as possible, but sometimes they are simply needed to evolve. Of course, with every release we indicate which changes need to be done to the database, but if you skip a few releases before you upgrade this tends to become cumbersome.

It’s this last point which I want to demonstrate in this post. The 4.2 and every subsequent release have a new mechanism on board to determine the current version of the database schema and update it to the fresh release automatically.

Setting up the ‘old’ data

We’ll start by downloading and unzipping an older release of jBPM, jBPM4.1:

[jbarrez@jenova 4.2_upgrade]$ unzip jbpm-4.1.zip

We’ll now create the jBPM schema using one of the install task shipped in the distribution. Since I’m using a MySQL database, I first need to update the jdbc properties that the install script will be using:

[jbarrez@jenova 4.2_upgrade]$ your-favourite-editor jbpm-4.1/install/jdbc/mysql.properties

and change it to my local database:

We can now create the schema in my local database (notice how we just need to add the ‘database‘ parameter, the script will then take the connection properties from the mysql.properties file we’ve just edited):

[jbarrez@jenova 4.2_upgrade]$ cd jbpm-4.1/install/
[jbarrez@jenova install]$ ant create.jbpm.schema -Ddatabase=mysql
Buildfile: build.xml
 [echo] database......... mysql
 [echo] tx............... standalone
 [echo] mail.smtp.host... localhost
 [echo] current dir = /home/jbarrez/dev/blog/4.2_upgrade/jbpm-4.1

create.jbpm.schema:
 [echo] creating jbpm schema in db jdbc:mysql://localhost:3306/jbpm

Executing resource: /home/jbarrez/dev/blog/4.2_upgrade/jbpm-4.1/install/src/db/jbpm.mysql.create.sql
79 of 79 SQL statements executed successfully
BUILD SUCCESSFUL Total time: 3 seconds

Which gives us a fresh jBPM 4.1 schema in our database:

We now use a basic ‘Hello World’ process definition:

<process name="helloWorld" xmlns="http://jbpm.org/4.0/jpdl">
  <start>
    <transition to="display hello world" />
  </start>
  <custom name="display hello world" class="be.jorambarrez...DisplayHelloWorld">
    <transition to="theEnd" />
  </custom>
  <end name="theEnd" />
</process>

… to start a process instance through the jBPM API:


public static void main(String[] args) {

ProcessEngine processEngine = new Configuration().buildProcessEngine();
RepositoryService repoService = processEngine.getRepositoryService();
ExecutionService executionService = processEngine.getExecutionService();

NewDeployment deployment = repo.createDeployment()
.addResourceFromClasspath("process.jpdl.xml");
deployment.deploy();
executionService.startProcessInstanceByKey("helloWorld");
 }

The debug log shows us the actual execution trace, where our ‘Hello World’ message is displayed between the engine logs.

17:53:15,219 FIN | [ProcessDefinitionImpl] creating new execution for process 'helloWorld'
17:53:15,224 FIN | [DefaultIdGenerator] generated execution id helloWorld.1
17:53:15,229 FIN | [ExecuteActivity] executing activity(25788693)
17:53:15,229 FIN | [ExecuteActivity] executing activity(display hello world)

Hello World!

17:53:15,230 FIN | [ExecuteActivity] executing activity(theEnd)
17:53:15,230 FIN | [ExecutionImpl] execution[helloWorld.1] ends with state ended

Upgrading in one minute

Suppose now that we take this process to production. Life is happy and peace. But then comes a new jBPM release. The developer hands start to itch again … So we take the jBPM 4.2 distribution and unzip it:

[jbarrez@jenova 4.2_upgrade]$ unzip jbpm-4.2.zip

And we change the dependencies of our little Hello World project, by either replacing the 4.1 jar with the 4.2 jar in the build path. Or if you are using Maven:


<dependency>
<groupId>org.jbpm.jbpm4</groupId>
<artifactId>jbpm-jpdl</artifactId>
<version>4.2</version>
</dependency>

<repositories>
<repository>
<id>jboss</id>
<url>http://repository.jboss.com/maven2</url>
</repository>
</repositories>

Of course, since we are adventurous developers, we don’t care about upgrading and just run our Hello World process with the new jBPM version. However, the new upgrade check will now kick in:

Exception in thread "main" org.jbpm.api.JbpmException: jBPM DB schema not in sync
 with library version: no JBPM4_PROPERTIES table.   Run the upgrade target
 first in the install tool.
 at org.jbpm.pvm.internal.cmd.CheckDbCmd.execute(CheckDbCmd.java:54)
 at org.jbpm.pvm.internal.svc.DefaultCommandService.execute(DefaultCommandService.java:42)
 at org.jbpm.pvm.internal.tx.StandardTransactionInterceptor.execute(StandardTransactionInterceptor.java:54)
 at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.executeInNewEnvironment(EnvironmentInterceptor.java:53)
 at org.jbpm.pvm.internal.svc.EnvironmentInterceptor.execute(EnvironmentInterceptor.java:40)
 at org.jbpm.pvm.internal.svc.RetryInterceptor.execute(RetryInterceptor.java:55)
 at org.jbpm.pvm.internal.svc.SkipInterceptor.execute(SkipInterceptor.java:43)
 at org.jbpm.pvm.internal.cfg.ProcessEngineImpl.checkDb(ProcessEngineImpl.java:177)
 at org.jbpm.pvm.internal.cfg.ProcessEngineImpl.buildProcessEngine(ProcessEngineImpl.java:170)
 at org.jbpm.api.Configuration.buildProcessEngine(Configuration.java:140)
 at be.jorambarrez.jbpm4.upgrade.Main.main(Main.java:15)

As you can see in the stacktrace, during the creating of the jBPM ProcessEngine a check on the database schema will take place (tech detail: it uses the JBPM4_PROPERTIES table and some other checks to do this). And since the exception is thrown during ProcessEngine creation, this also stops us from executing processes in a old schema – which is a good thing.

So we follow the instructions of the exception above and run the upgrade script. Don’t forget to change the database properties (as we did above).

[jbarrez@jenova install]$ ant upgrade.jbpm.schema -Ddatabase=mysql
Buildfile: build.xml
 [echo] database......... mysql
 [echo] tx............... standalone
 [echo] mail.smtp.host... localhost

upgrade.jbpm.schema:
 [echo] upgrading jbpm schema...

 ... // *snip* Hibernate setup etc

 12:30:02,501 INF | [DbHelper] --- Executing DB Commands -------------------------
 12:30:02,501 INF | [DbHelper] create table JBPM4_PROPERTY (
 KEY_ varchar(255) not null,
 VERSION_ integer not null,
 VALUE_ varchar(255),
 primary key (KEY_)
 ) type=InnoDB
 12:30:02,544 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,544 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HACTI
 12:30:02,564 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,564 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HPROCI
 12:30:02,578 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,578 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HVAR
 12:30:02,595 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,596 INF | [DbHelper] alter table JBPM4_HIST_DETAIL drop index IDX_HDETAIL_HTASK
 12:30:02,610 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,611 INF | [DbHelper] create index IDX_HSUPERT_SUB on JBPM4_HIST_TASK (SUPERTASK_)
 12:30:02,625 INF | [DbHelper] --- Result: 0 --------------------------
 12:30:02,736 INF | [PropertyImpl] nextDbid is initialized to 4
 12:30:02,740 INF | [Upgrade] jBPM DB upgrade completed successfully. 
 12:30:02,740 INF | [SessionFactoryImpl] closing
 12:30:02,741 INF | [DriverManagerConnectionProvider] cleaning up connection pool: jdbc:mysql://localhost:3306/jbpm
BUILD SUCCESSFUL Total time: 4 seconds

The output of the script show us that some upgrades were done: a new table was created, some indexes were altered and some colum values were updated.

When we now retry running the Hello World process, everything works fine:

12:41:19,275 FIN | [DatabaseIdComposer] generated execution id helloWorld.10010
12:41:19,279 FIN | [ExecuteActivity] executing activity(28678425)
12:41:19,279 FIN | [ExecuteActivity] executing activity(display hello world)
12:41:19,280 FIN | [ExecuteActivity] executing activity(theEnd)
12:41:19,280 FIN | [ExecutionImpl] execution[helloWorld.10010] ends with state ended
Hello World!

And if you pay close attention to the logs, you’ll see that the schema check is OK now:

12:41:18,687 INF | [CheckDbCmd] jBPM version info: library[4.2], schema[4.2]

NOTE: if you got a grumpy DBA: the jBPM distro also ships with plain .sql files to use for upgrading the schema, which you can hand over to him/her.

Conclusion

And that’s it, folks. We’ve got API stability, backwards schema compatibility and as I just showed you, upgrading the database takes only a minute of your time… leaving more time for you to actually code business processes!

QA

To finish the post, I just want to show you a screenshot from our Hudson continuous integration job. Since the upgrade feature is extremely important for us, we test it against all our supported databases and JDK versions. The test job that is executed is actually very close related to the explanation of the previous section. With every new release, this configuration matrix will continue to grow. Which means that in every future release are certain of backwards compatibility and upgradeability.

Happy jBPM coding!

Upgrading in one minute

jBPM4 real-life example: The Train Ticket Demo (part 5: BI and BAM)

Business processes are never finished

As a business changes or grows, business processes will need to incorporate these changes or they will become a drawback for the business. There are the natural changes that come as the business or the market change, but some changes are related to the actual feedback that is received from measuring metrics about the business process executions.

If you take a look at the BPM lifecycle, gathering this feedback means that the process executions must be monitored. Using the metrics gathered there, it will be possible to discover bottlenecks in the process (eg some task is taking long due to a human resource problem) and to act upon them. The next step will then be optimizing the business process, with the new knowledge as background. Now, the lifecyle is completed and we’re back to modeling the actual changes in the process in our BPMN Signavio editor. So, as a developer, it might be interesting investing in jBPM, since your work will never be done (this applies certainly for contractors ;-))

Traditionally, there are two concepts that are often used when one discusses ‘monitoring’ business processes. I’m talking here about Business Intelligence (BI) and Business Activity Monitoring (BAM). Typically, BI is used when the data that is used is historical (eg. monthly reports), whereas BAM is used when the data is real-time. Often, BAM is coupled with Complex Event Processing (CEP) to allow for automatic actions that are triggerd once a certain event or pattern is discovered. Yes, I know these are lot of fancy words, but they all come down to one thing: displaying data of business processes in some way or another.

In this article, I’ll demonstrate how BI and BAM can be done with jBPM. Today, I also have 2 screencast for you, so be sure to read the article to the end (which should be the case anyway 😉 )

Business intelligence with jBPM

I’ve already discussed this topic in a previous article, so I’ll be short here. Using the jBPM console, it is very easy to write your own BI reports and display it in the console. Since we are using the Eclipse BIRT engine, this means that the reports are also usable without the console or embedded within your own application. Take a look at the following screencast, where two things are shown:

  • The cellhone emulator (aka the jBPM-phone) is used to send text messages to the Ticket Handling Platform and the jBPM console is used to inspect the actual process instances.
  • The default BI report is rendered and displayed through the jBPM console.

Business activity monitoring with jBPM explained

For implementing BAM, we’ve used a setup as displayed in the following picture (click for bigger version):

Conceptually, the story goes as follows: inside the process, events (= pojos) are fired to a JSM queue. On this JMS queue, a message driven bean is parsing the events and handing them over to a CEP engine. In the demo, we are using the Esper engine, wrapped in an EJB3 such that the CEP handling scales with our existing architecture. Depending on the type of event, Esper will execute an action by invoking a so-called Subscriber. In our demo, this is mostly writing aggreagations to the database.

Note that this way of working is based upon the ideas from the book ‘Open Source SOA’, by Jeff Davis. Which also includes two big chapters about jBPM, by the way.

We’ll now look more closely into how this is actually done in Java code:

On certain activitites, we define an event listener. In the following example, we listen to the ‘end’ event of the ‘chargecustomer’ activity and send an event using the SendEventToQueue handler class. Also note that the event is send asynchronously, which means that the actual sending of the message is not done in the original thread. This way, the response towards the customer can be send more quickly.

<custom name="charge customer ...">
  <on event="end" continue="async">
    <event-listener>
      <field name="eventClass">
        <string value="org.jbpm.trainticketdemo.event.TicketSoldEvent"/>
      </field>
      ... (more fields, see source for details)
    </event-listener>
  </on>
  <transition to="end"/>
</custom>

The TicketSoldEvent that is fired here, is just a regular pojo:

public class TicketSoldEvent implements Serializable {

  private String from;

  private String to;

  private Double amount;

  // getters and setters

}

These events are received by a message driven bean and handed over to the Esper runtime engine (see line 17):

@MessageDriven(activationConfig =
{
 @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
 @ActivationConfigProperty(propertyName="destination", propertyValue=JmsConstants.EVENT_QUEUE)
})
public class EventQueueProcessor implements MessageListener {

 // field declarations omitted

  public void onMessage(Message message) {

    try {
      if (message instanceof ObjectMessage) {

        ObjectMessage objMsg = (ObjectMessage) message;
        Object obj = objMsg.getObject();
        epsProvider.getEPRuntime().sendEvent(obj);

      }
    } catch (JMSException e) {
      LOGGER.error(e);
    }
 }
}

Without going into much details about Esper (read their excellent documentation if you are interested). We use Esper in the demo to cope with ‘pojo streams’. These pojo streams can be aggregated and queried upon using regular SQL, which I think is really cool:

EPStatement moneyTracking = admin.createEPL(
 "select count(*) as nrOfTickets, sum(amount) as total, " +
 "avg(amount) as average, stddev(amount) as stddev " +
 "from TicketSoldEvent.win:time(15 sec)");

 moneyTracking.setSubscriber(new MoneyTrackerSubscriber());

Note that we are selecting here the count, sum, average and standard deviation aggreagtion of the ticket sale data, based upon the TicketSoldEvent pojos that were received during the last 15 seconds. The results of that query are handed over to another pojo, the MoneyTrackerSubscriber, which will just store this results in the database for displaying them in the GUI.

Note that we are barely scratching the power of Esper here. It is for example also possible to join pojo streams, based on pojo field values. Take for example a look at the following snippet, where we join the TicketRequestReceived event with the QuoteSentEvent which have the same processInstance id, to calculate the time between receiving a cellphone text message and actually replying on it (which might be an SLA).

// SLA for the time between the ticket request receival and the quote sending
 EPStatement responseTimings = admin.createEPL(
 "select TicketRequestReceivedEvent.timestamp as ticketRcvTime, QuoteSentEvent.timestamp as quoteSentTime, " +
 "TicketRequestReceivedEvent.processInstanceId as processInstanceId " +
 "from TicketRequestReceivedEvent.win:time(24 hours) inner join QuoteSentEvent.win:time(24 hours) " +
 "on TicketRequestReceivedEvent.processInstanceId = QuoteSentEvent.processInstanceId");

But you can go much further with Esper. I particulary like the pattern detection mechanism for streams. I’ve used it in the past a lot, for example to build a fraud-detection system: when a certain pattern in the events is found (I believe it had to do with money that was first refused and than we checked on events that were related to the same customer trying to get smaller amount of money approved), an email would be send out to all the account managers involved. But such features are probably for a next iteration of the demo. By that time, we might be able to leverage the SAM solution which Heiko described in a blogpost.

Bam: The end result

The following screencast shows the end result of the explanation of the previous section. The data that is gathered by the subscribers of our events, are displayed using a Seam application, with Richfaces for the general GUI and Primefaces for the flash chart rendering. Using some ajax magic, the charts are rerendered in real-time.

The demo BAM application shows following information:

  • The average and standard deviation time between receiving the text messga eand sending a response (SLA)
  • The time it took the customer to respons with an ‘accept message’, shown as a pie chart with catagories (eg < 30 seconds)
  • The number of customers who had enough money on their balance vs those who didn’t
  • The total money earned (with average and stddev)  in the last 10 windows of 15 seconds

It is easy to see how the information from the BI reports and the BAM application can be used to enhance the business process. Eg when a lot of customers aren’t able to pay because they forgot to add money to their account, it could be wise to add a step to the process that ask ‘do you want to add more money?’, using the credit card registered with the customer. So, your imagination is the only limiting factor here 😉

Like before, the source code can be found in our svn repo: