Self-Optimizing Workflow and BPM Engines

Posted by Kevin Sweeney on Aug 23, 2013 10:43:23 AM



Self-Optimizing Workflow .NET Runtime Optimization is Here!


Given today’s demanding software applications, BPM engines must provide self-optimizing workflows in order to process high volumes of transactions quickly and accurately.  Next generation BPM engines such as cDevWorkflow have these self-optimizing workflow performance improvements that optimize runtime system performance.   Here are some of the Self-Optimizing workflow tasks  that the software performs:

  • Runtime dynamic path generation
  • In-memory compilation and execution of decision evaluation
  • Runtime path optimization
  • Negative path traversal
  • Throughput optimization
  • Step-to-step optimizations
  • Runtime optimizations based on similar inputs
  • Runtime improvements based on statistics – future
  • Nano paths - future

Runtime dynamic path generation is a concept that very few workflow / BPM engines can perform.  Most workflow / BPM products are limited to a 1 to 1 relationship matching workflow design with the workflow definition and the instance.  This updated design scenario falters within a dynamic environment where the user determines the workflow at time of use based on his or her needs.

A fine example of this design limitation occurs every time you attempt to navigate your way through an automated telephone attendants series of prompts.   We have all arrived at the point where we are frustrated simply because the menu items do not offer a prompt for what information we need.  It is for this reason, and many reasons just like it, that we need BPM engines where self-optimizing workflows can handle a dynamic # of paths from a step, where the # of paths a users needs can not be pre-determined at runtime.

The good news is that the Next Generation of BPM tools has arrived.  Self-Optimizing Workflows are real.  And cDevWorkflow is able to perform this function by morphing the design at runtime with information gathered from runtime data.

In-memory compilation and execution can provide the most dynamic execution of code at runtime.  cDevWorkflow handles all evaluation decisions making this a routine matter.  From simple to complex routine decisions can be evaluated in real-time by first performing an in-memory compilation and then executing that generated code.  Most workflow BPM engines perform this function by interpreting code, this method negatively impacts system performance.  cDevWorkflow's In-memory compiled expressions are optimized and executed machine code to perform the operation, which provides optimized performance at runtime.  Self-optimizing workflow is available today.

Runtime path optimization is a pre-process that runs against the workflow instance to determine what optimizations can be performed before the process starts to execute.   Just like code, many times workflow processes have dead paths, redundant paths, circular paths, steps that will never produce the configured output, duplicate steps and mostly paths that need serious optimization.  Its human nature to design a processes based on its paper process, when a process is designed, it’s very difficult to optimize the process upfront, as the process executes, most organizations use a revision process to change and optimize the process.  cDevWorkflow’s pre-processor takes care of this by pre-processing your workflow and optimizing it.  Self-optimizing workflow is a powerful tool.

Negative path traversal, this is a concept that do not exist in any other workflow engine, except in cDevWorkflow.  Negative paths are considered as paths that return back in a process, but causes the process to halt in the middle.  Return paths are very important, but if these are not configured properly for the path to return back and perform a forward process, this can have a negative effect on the process it-self.  cDevWorkflow’s pre-processor evaluates these paths and optimizes the process such way that it is able to process the workflow by minimizing the # of halts that can stop the process in the middle.   Self-optimizing workflow is here.

Throughput optimization handles how data is passed between steps within the process.  cDevWorkflow utilizes in-memory matrices to perform this.  Since memory is the fastest accessible data storage, cDevWorkflow utilizes high performance matrix algorithms to store control flow and process data in memory.  Given today’s hardware, memory is an abundance.  Self-optimizing workflow supports your existing hardware.

Step-to-step optimization – a workflow process may contain some engine steps and some user/custom steps.  Alike steps performing similar operations with same data will be optimized to handle these steps in less number operations.  Just like many compilers optimize the code, cDevWorkflow is able to optimize a process to execute in fewer number of operations.  Self-optimizing workflow gets it done faster.

Runtime optimizations based on similar inputs – just like many of the caching algorithms, cDevWorkflow utilizes some specialized algorithms to optimize itself for similar inputs.  A workflow process that is pretty static with a common path will produce same results for same inputs.  Given this case, cDevWorkflow pre-processor is able to analyze a process and produce a faster result based on other executions.  Self-optimizing workflow is logic based.

Runtime improvements based on statistics (future) – even though statistics are collected at many levels by the cDevWorkflow engine, these are not being used in workflow process decision making.  In the future, statistics such as KPI and other data will be made available to the process for better decision making.  Statistics also provide data for predictive models, where a process can predict the outcome based on previous execution data.  This is definitely coming to cDevWorkflow, and we hope to build the best algorithm for this feature, where cDevWorkflow will surpass any other product on the market when it comes to predictive model based processing.  Self-optimizing workflow improves with information.

Nano paths (future) – very interesting concept.  It has been proven that any problem that can be broken into smaller problems can be solved faster and with a higher precision.  Given this theory, we hope to enhance cDevWorkflow to use Nano technology to optimize a process.  In the future, cDevWorkflow pre-processor will be able to figure out Nano-paths that can take a piece of functionality and break it down into smaller set of steps and to solve it faster.  This is an interesting optimization, since most optimizations within cDevWorkflow starts large and becomes smaller to optimize, in this case, a step expands itself to many more steps to optimize.  The reason this optimization is so powerful is that, by expanding a step to multiple steps, it not only produces more efficient steps and paths, it utilizes more parallel resources such as multi core processing / threads to execute and produce a result much faster.  Self-optimizing workflow at work today and tomorrow.

A small sampling of the Workflow Technology for BPM Solutions contained within our cDevWorkflow product offering can be found here: Workflow Technology That Works.

[hs_form id="9"]

Learn more about our Process Automation & IT, QA Services or Software Development products and solutions on the Web!  Visit us at: Innovative Process Solutions


Topics: Self-Optimizing BPM, Runtime path optimization, Throughput optimization, Blog, Self-Optimizing workflow, .net runtime optimization, self-optimizing bpm workflow engine, self-optimizing workflow and bpm engines