In the past years, often the request on how to swap the persistence logic of Activiti from relational database to something else was made. When Activiti 6 was announced, one of the promises was exactly making this possible.
People that have dived into the code of the Activiti engine will know that this is a serious refactoring, as the persistence code is tightly coupled with the regular logic. Basically, in Activiti v5, there were:
The problems in version 5 were the following:
Now don’t get me wrong. The v5 code powers many awesome stuff all over the world. But when it comes to swapping out the persistence layer … it isn’t something to be proud of.
And surely, one could hack your way in into the version 5 code (for example by swapping out the DbSqlSession with something custom that responds to the methods/query names being used there), but it still would be not quite nice design-wise and quite relational-database-like. And that doesn’t necessarily match the data store technology you might using.
No, for version 6 it needed to be done properly. And oh boy … it was going to be a lot of work … (just look at the commits on the v6 branch for the last couple of weeks). But … the end result is just beautiful (I’m biased, true). So let’s look at the new architecture in v6 (forgive me my powerpoint pictures. I’m a coder not a designer!):
So, where in v5 there were no interfaces, there are interfaces everywhere in v6. The structure above is applied for all of the Entity types in the engine (currently around 25). So for example for the TaskEntity, there is a TaskEntityImpl, a TaskEntityManager, a TaskEntityManagerImpl, a TaskDataManager and a TaskDataManagerImpl class (and yes I know, they still need javadoc). The same applies for all entities.
Let me explain the diagram above:
Moving all of the operations into interfaces gave a clear overview of what methods were spread across the codebase. Did you know for example there were at least five different methods to delete an Execution (named ‘delete’, ‘remove’, ‘destroy’, etc…)? They did almost the same, but with subtle differences. Or sometimes not subtle at all.
A lot of the work over the past weeks included consolidating all of this logic into one method. Now, in the current codebase, there is but one way to do something. Which is quite important for people that want to use different persistence technologies. Making them implement all varieties and subtleties would be madness.
To prove the pluggability of the persistence layer, I’ve made a small ‘in-memory’ prototype. Meaning that, instead of a relational database, we use plain old HashMaps to store our entities as {entityId, entities}. The queries then become if-clauses.
The code can be found on Github: https://github.com/jbarrez/activiti-in-mem-prototype
(people have sometimes on the forum asked how hard it was to run Activiti purely in memory, for simple use cases that don’t mandate the use of a database. Well, now it’s not hard at all anymore! Who knows … this little prototype might become something if people like it!)
As expected, we swap out the DataManager implementations with our in-memory version, see InMemoryProcessEngineConfiguration :
@Override protected void initDataManagers() { this.deploymentDataManager = new InMemoryDeploymentDataManager(this); this.resourceDataManager = new InMemoryResourceDataManager(this); this.processDefinitionDataManager = new InMemoryProcessDefinitionDataManager(this); this.jobDataManager = new InMemoryJobDataManager(this); this.executionDataManager = new InMemoryExecutionDataManager(this); this.historicProcessInstanceDataManager = new InMemoryHistoricProcessInstanceDataManager(this); this.historicActivityInstanceDataManager = new InMemoryHistoricActivityInstanceDataManager(this); this.taskDataManager = new InMemoryTaskDataManager(this); this.historicTaskInstanceDataManager = new InMemoryHistoricTaskInstanceDataManager(this); this.identityLinkDataManager = new InMemoryIdentityLinkDataManager(this); this.variableInstanceDataManager = new InMemoryVariableInstanceDataManager(this); this.eventSubscriptionDataManager = new InMemoryEventSubscriptionDataManager(this); }
Such DataManager implementations are quite simple. See for example the InMemoryTaskDataManager who needs to implement the data retrieval/write methods for a TaskEntity:
public List<TaskEntity> findTasksByExecutionId(String executionId) { List<TaskEntity> results = new ArrayList<TaskEntity>(); for (TaskEntity taskEntity : entities.values()) { if (taskEntity.getExecutionId() != null && taskEntity.getExecutionId().equals(executionId)) { results.add(taskEntity); } } return results; }
To prove it works, let’s deploy, start a simple process instance, do a little task query and check some history. This code is exactly the same as the ‘regular’ Activiti usage.
public class Main { public static void main(String[] args) { InMemoryProcessEngineConfiguration config = new InMemoryProcessEngineConfiguration(); ProcessEngine processEngine = config.buildProcessEngine(); RepositoryService repositoryService = processEngine.getRepositoryService(); RuntimeService runtimeService = processEngine.getRuntimeService(); TaskService taskService = processEngine.getTaskService(); HistoryService historyService = processEngine.getHistoryService(); Deployment deployment = repositoryService.createDeployment().addClasspathResource("oneTaskProcess.bpmn20.xml").deploy(); System.out.println("Process deployed! Deployment id is " + deployment.getId()); ProcessInstance processInstance = runtimeService.startProcessInstanceByKey("oneTaskProcess"); List<Task> tasks = taskService.createTaskQuery().processInstanceId(processInstance.getId()).list(); System.out.println("Got " + tasks.size() + " tasks!"); taskService.complete(tasks.get(0).getId()); System.out.println("Number of process instances = " + historyService.createHistoricProcessInstanceQuery().count()); System.out.println("Number of active process instances = " + historyService.createHistoricProcessInstanceQuery().finished().count()); System.out.println("Number of finished process instances = " + historyService.createHistoricProcessInstanceQuery().unfinished().count()); } }
Which, if you run it gives you this (blazingly fast as it’s all in memory!):
Process deployed! Deployment id is 27073df8-5d54-11e5-973b-a8206642f7c5
Got 1 tasks!
Number of process instances = 1
Number of active process instances = 0
Number of finished process instances = 1
In this prototype, I didn’t add transactional semantics. Meaning that if two users would complete the same user task at the very same time the outcome would be indeterminable. It’s of course possible to have in-memory transaction-like logic which you expect from the Activiti API, but I didn’t implement that yet. Basically you would need to keep all objects you touch in a little cache until flush/commit time and do some locking/synchronisation at that point. And of course, I do accept pull requests 😉
Well, that’s pretty much up to you … I myself still have a special place in my heart for Neo4j … which would be a great fit as it’s transactional). But the most important bit is: in Activiti v6 it is now possible to cleanly swap out the persistence layer.
Awesome! Just one comment. DataManagerImpl is a lame name. Call it IbatisDataManager or something. I mean it should be clear from the name what kind of implementation it is. Just Impl tells nothing.
Hi Joram,
this is great. An old dream fulfilled.
Based on the types of uses I have been involved, I believe it is profitable to bring it to make it official.
It makes it usable in embedded usages and highly transaction environments.
@Oleksandr : yes. Caching and naming things are the hardest problems of software development. I’ll discuss if we can change it to something like that.
@saeid: Thanks!
Obviously the Anonymous above was me 😉
Hi Joram,
Phantastic, thanks a lot.
Markus
Great! This was the moment I was waiting for. The only way to support “true” embedding is providing a custom persistence implementation, nearly no other “advanced” workflow engine is capable of this. Great!
Björn
After some testing there is an persistence implementation detail lurking in the interfaces: “ByteArrayRef”
This class is exposed in the (…Variable…) interfaces, but it can not be replaced by another/overwritten class because it is a final class. Maybe this class also needs to be replaced by an interface ?