The performance impact of scripting in processes

We often see people using the scripting (for example in a service task, execution listener, etc.) for various purposes. Using scripts versus Java logic makes often sense:

  • It does not need to be packaged into a jar and put on the classpath
  • It makes the process definition more understandable: no need to look into different files
  • The logic is part of the process definition, which means no hassling to make sure the correct version of logic is being used

However, it’s important to also keep in mind the performance aspect of using scripting within the process definition, and balance those requirements with the benefits above.

The two scripting languages we typically see being used with Activiti is Javascript and Groovy. Javascript comes bundled with the JDK (Rhino for JDK 6 and 7) and Nashorn for JDK 8, which makes it easy to pick up. For Groovy, the Groovy scripting engine needs to be added to the classpath.

But let me tell you, I’m no fan of using Javascript as the scripting language choice, as there are subtle changes when moving between JDK versions (read more in a previous post of me here and here, and those are the ones that were documented …). So that means you could write your logic one day and it all happily works and the next day after a JDK upgrade it all fails. I rather spend my time on actually coding.

To verify the performance I made a very small microbenchmark:

Screenshot 2015-09-08 23.06.34and where the script did something silly like (the point was to have a getVariable() and setVariable() in there and something extra like getting the current day):

var input = execution.getVariable(‘input’);
var today = new Date().getDay();
execution.setVariable(‘result’, input * today);

The same code in a Java service task:

public class MyDelegate implements JavaDelegate {

    @Override
    public void execute(DelegateExecution execution) throws Exception {
        Integer input = (Integer) execution.getVariable("input");
        int today = Calendar.getInstance().get(Calendar.DAY_OF_MONTH);
        execution.setVariable("result", input * today);
    }
}

and the Groovy counterpart:

def input = execution.getVariable('input');
int today = Calendar.getInstance().get(Calendar.DAY_OF_MONTH);
execution.setVariable('result', input * today);

I started that process instance 10.000 times and simply noted down the total execution time, I think the numbers speak for themselves:

  • JavaDelegate: 6255 ms
  • Groovy: 7248 ms
  • Javascript: 27314 ms

The JDK version used was the latest version (1.8.0_60). The first time I ran the tests I was on 1.8.0_20, and the Javascript results were 25% higher (I read that performance improvements went in in JDK 1.8.0_40). For Groovy I used version 2.4.4 (which you should be using giving older versions have a security problem!)

Just to give a visual idea of the difference between the options:

chart2

 

Using Groovy for the scripting language seems to be a far better choice performance-wise compared to using Javascript. Do take in account this is a microbenchmark for one very simple use case. But given our troubles in the past with JDK upgrades that break Javascript scripts and this result, it’s very hard to make a case for selecting Javascript by default.

 

UPDATE 11 SEPT ’15: Quite a few people have asked me why the difference is of that magnitude. My assumption is that it’s because the javascript engine in the JDK is not thread safe and thus cannot be reused nor cached, thus having a costly bootup of the ScriptingEngine every time. If you take a look at http://docs.oracle.com/javase/7/docs/api/javax/script/ScriptEngineFactory.html, you can read that there is a special parameter THREADING, which we use in Activiti: https://github.com/Activiti/Activiti/blob/master/modules/activiti-engine/src/main/java/org/activiti/engine/impl/scripting/ScriptingEngines.java#L111 to determine if the scripting engine can be cached. Nashorn (and Rhino) returns null here, meaning it can’t be used to execute scripts on multiple threads, i.e. each thread needs it’s own instance. I can only assume that the ScriptEngineManager in the JDK does something similar.

 

4 Comments

  1. Gethin James September 10, 2015

    I am not (yet) convinced by your tests. Did you use the invoke dynamic (indy) jar of Groovy? Also, if you are not using any of Groovy’s dynamic features you can @CompileStatic to improve performance. You are also micro-benchmarking 3 lines of code, is that a valid test?

  2. Mike Dias September 10, 2015

    Hi Joram,

    I have a little question: Why the javascript test is using “new Date().getDay()” and the others is using “Calendar.getInstance().get(Calendar.DAY_OF_MONTH)”? I didn’t analyze both methods implementations, but those differences can be relevant for the experiment.

    Best,
    Mike Dias

  3. Joram Barrez September 10, 2015

    @Gethin: I did use the -indy jar, given your comment on the Alfresco jira last week, but it made no difference here vs the regular -all jar. Does @CompileStatic work with the JSR 223 scriptingEngine approach? I have tried it in the past (and other annotations) but never got them to work.

    @Mike: I had the same thought before posting this, and verified. With or without that line does not make a difference.

    @Both: I mention twice ‘microbenchmark’ above, so I’m fully aware of the usefulness of the test. The point is that Activiti uses the JSR 22/ScriptingEngine approach, and invoking the javascript engine is dog slow, even doing simple stuff as above. Given that, plus my bad experiences in the past with js inline scripts in processes just breaking, this post about my preference for Groovy if there is a need for scripting in a process (but i still prefer a regular java service task).

  4. Owen Rubel March 16, 2016

    @CompileStatic can work but it’s a different paradigm; one scripts to write fast code and that means that often on needs a more ‘dynamic’ approach.

    HOWEVER… once something is ‘production ready’ DEFINITELY ‘statically compile’. This is the HUGE benefit of Groovy

Leave a Reply

Your email address will not be published. Required fields are marked *