Modelling Choreography (with events, states and business rules)

This week the BCS SPA group held a fascinating session titled “Modelling Choreography” by requirements analyst Ashley McNeile.

Ashley described some of the past efforts to model and implement choreographies, using types of process algebra such as  Robert Millner’s Calculus of Communicating Systems (CCS) and its derivative Pi-Calculus. However, Ashley used sequences of events and states (i.e. a state diagram) which he also compared to Michael Jackson’s formalised object lifecycles (e.g. JSD  / Jackson Diagrams). Various W3C efforts have described choreographies too – e.g. WS-CDL. Of course the latest modelling construct for choreography is BPMN2!

As an example of his practice, Ashley described an example – modelling bank account transactions via Protocol Modelling (using simple state diagrams):

  • state model 1: defined the close and withdraw events on an active account
  • state model 2: defined the freeze and release account events
  • state model 3: this had no state transitions, but defined the state by the associated constraints (or business rules)
    • if balance < 0 then account state is overdrawn
    • one cannot close an account if it is overdrawn
  • all 3 state models operate in parallel.

To analyse these state models they can be combined into a single state models (with all combinations of states, and all events), and then the unreachable states can be filtered out. The interesting thing here is (1) the analysis of state models for completeness and (2) the use of incomplete state diagrams as a business notation for textual (policy or constraint) business rules.

Other observations:

  1. These types of business rule apply to states and data; they can be extracted and modified (by a developer, or state modeller) into event rules or guards in a state transition diagram. Is it interesting to specify these business rules up front before mapping to events and processes? Yes from a business perspective, as new events or states might affect or be affected by existing business rules.
  2. Using a state to specify a business rule (in terms of the state and output) is an interesting notation that lends itself well to mapping to appropriate events (or indeed processes). Could it catch on in the business rule community?
  3. The use of an explicit choreography language has not had  much success it seems. Google WS-CDL and most entries are dated 2009 or earlier. BPMN2′s choreography may yet prove useful but possibly the concepts are too difficult for business modellers yet imply a co-operative design process for developers that rarely occurs in practice (beyond “this is the interface”!).
  4. At the end of the day, the sequence of events in a business system is just a complex event – which maybe can tell you if the choreography is valid or not.

I’ll add a link to the slides to help explain all the above when they become available…

Annex: a Distributed System Choreography Development Process:

This process describes a development process of state diagrams for choreography purposes:

  1. Define participants and messages (/events) that interact between them
  2. Define states with events as messages from and to, with only 1 sender per state
  3. Project the states out to individual participants – i.e the parts of the state model for each participant – allowing ambiguous states but ensuring these have no sends
  4. Merge the states for each participant
  5. Enact – check each event at a time to prove feasibility of the interacting state models


DMN: promoting decision-based software

Last week TIBCO hosted the first Face2Face meeting of the decision modelling / management experts from FICO, IBM and Oracle representing one of the submission teams for the OMG Decision Model and Notation standard. Although we cannot divulge any details of the meeting results, suffice to say that progress was considered as “good” by all concerned… publication (to OMG) of the first draft of this and the other submissions is due in May 2012.

From an event processing perspective, current decision-based methodologies mostly focus on the “decision as a service” (DaaS?) type models and associated stateless executable artifacts – think TIBCO ActiveMatrix Decisions and TIBCO BusinessEvents Decision Manager, and their equivalents Oracle Business Rules, IBM Websphere Ilog JRules, and FICO Blaze Advisor. Here a decision is something that occurs at a point in time, based on data / information valid at the time of decision, with some explicit process required to revisit said decision.

However for event-based decisions, we can also consider decisions as a continuous operation. This is best illustrated by the “decision” made when some event pattern is identified: the pattern may be identified and/or disqualified multiple times for the same entity, based for example on complex and sophisticated business rules (that may themselves be changing over time). A typical mechanism for this might be inference rules (as seen in TIBCO BusinessEvents). I would expect later versions of DMN to be extended to cover rule-based and continuous-evaluation decisions, but DMN version 1 will likely cover the simpler “Decision as a Process Task” role (where “process task” could be a BPMN Business Rule Activity or a dynamic rule-driven activity). The differences in decision models for continuous versus decision-point based decisions is an interesting area for future discussion!

My thanks again to the participants of the F2F for a successful meeting.

BBC2011 – quick review

As is now “tradition”, the Business Rules Forum / Building Business Capability conference followed RulesFest (from last week), each covering the opposite side of the rules spectrum in terms of audience (analysts versus rules programmers), size (~1000 vs ~120), and even location (California vs Florida). It’s an interesting fracturing of the “decision and rules community”, which actually makes sense (i.e. business vs IT) – and both deserve to grow next year (RulesFest should be 2x-3x in size, BBC could probably easily grow 30%).

One of the interesting trends here at BBC2011 is the growth of the business analyst focus, which of course means “rules engine vendors” make much less sense to be seen here versus “rules management” or even MDM vendors, and those less so than the business modelling community. In the CEP space, vendors tend to cover both business and IT camps: for example TIBCO BusinessEvents includes event / concept modeling, state modeling and decision modeling, but for automation not business documentation. And both IBM and Red Hat have business user interfaces with event processing (with a rumor of a 4th CEP vendor about to buy their way into the decision management space).

So the main focus for business analysts attending BBC were very much at the TIBCO Nimbus level of process documenting / discovery (and indeed, decision documenting / discovery). And the surprise was the lack of business modeling tools (with the notable exception of Sparx systems – folks known from the UML world). Maybe this will change next year?

TIBCO Software exhibited this year too (with the aforementioned TIBCO Nimbus as well as ActiveMatrix BPM), and we contributed to 2 sessions (“The OMG Decision Model and Notation standard” co-presented with IBM, and “What Analysts need to understand about Business Events“). The latter was interesting to present, as events participate in many analyst models: I’ll post a precis of this session later.

TUCON2011: PJM controlling the flow of electricity

Richard Brenton Jr gave an excellent presentation on PJM’s use of TIBCO BusinessEvents for operational visibility – the control rules for the electricity supply grid, executing against operational events. The main point made here was that for true business control of rules, you need to not only business rules in the vocabulary of the business, but represented in the form the business requires (i.e. one shape of rules management does not fit all).

PJM have built a custom operations dashboard that includes the ability to control and update the supply control rules. Their rules management system or BRMS? MS Sharepoint! Sharepoint provides the workflow, version control, etc, for the “rule objects” that in turn are sent to the running rule engine as rule parameters. This exploits the power of the underlying Rete rules engine with the programmed rules being business rule templates. So Sharepoint is the rule repository, and new rules are fed to the system as events (with an update latency of 2ms). So a Sharepoint form provides the BRMS user interface…

Richard gave a good example of the power of the rule-driven, event-based approach. This past Summer’s East Coast US Earthquake (and subsequent shut-down of a major nuclear power plant) was followed by Hurricane Irene. PJM operators swiftly updated their control rules to suit. Given not many businesses in the E USA would have predefined rules for an earthquake followed by a hurricane, the ability to adapt “on the fly” using business rule technology proved pretty useful.

TUCON2011: AllState – yes insurance is event-driven!

Doug Safford from AllState Insurance gave a fascinating overview of a large Insurance IT department’s migration from a high-overhead ($1B pa budget!) mainframe shop to an architecture-driven organisation. This primarily revolved around the introduction of a standard ESB – a business event bus in this case, with a standard middleware solution (TIBCO EMS) with a standard framework (to handle the governance needs of tracking, logging, etc). It also involved much simplification of the IT processes: going from 10 to 2 data centers, for example, and from 15,000 (!!) to just 4 (!!!) AS400-class servers.

Doug mentioned that the enablers for their Business Process Management were frameworks, services, events and rules. Of these:

  • Services are carefully managed: they have deployed 4,000+ TIBCO BusinessWorks engines in the service and framework areas!
  • Moving to an Event Driven Architecture allowed much more business control of their business events, such as standardised routing and de-duping. The events feed their CEP engines as well as their Data Warehouses for analytics.
  • Separation of the business logic / rules was seen as critical in such a regulated industry; they could not afford to have volatile rules replicated in software code around their systems. AllState have standardised on TIBCO BusinessEvents as their rules engine (with 800+ engines today, and the fastest growing tool deployment for them).

In terms of the ROI for their investment, Doug mentioned:

  • Marketing campaigns can now be deployed from new in weeks rather than 6-12 months, or from existing campaign templates in days rather than months.
  • Rule changes, such as for routing leads to agents, can be done overnight, usually from the BAM reports. In one example they monitored the success rates for agents in closing business, re-routed more leads to them, to gain a 15% increase in closing rates…

I would not be at all surprised to see this AllState IT transformation to be covered in business management books and MBA programs in future!

TUCON2011: kicking off with ActiveMatrix Decisions

Spy shot of AMX Decisions...Tonight is the start of TUCON 2011 – the TIBCO User Conference – being held at the new Aria Hotel in Las Vegas, NV USA. Publicly unveiled to attendees tonight for the first time was TIBCO ActiveMatrix Decisions – the decision modeling component for TIBCO ActiveMatrix BPM that executes under TIBCO ActiveMatrix SOA. This allows the same Decision Table modelling environment to be shared between those modeling processes as orchestrations / BPMN, and those building rule-driven CEP-based processes.

Congratulations to the AMX Decisions team!

Another “city trader” incident, $2Bn, and the need for better risk controls

I happened to be visiting a TIBCO CEP client in the banking industry last week when the UBS “trading fraud” story gained steam. Some of the smarter investment banks are using technologies like CEP to get a handle on risk management, which has been, and will always be, a significant area of interest for banks and their regulators. As far as CEP technologies go, risk management is usually concerned with trade-state management make rule-based approaches more interesting than the trade-stream approach typically used in automated trading systems (and hence the useage of TIBCO BusinessEvents rules technology in this space over the usual ESP stream processing engines).

Risk and governance are of course often seen as aspects of the problem: situation awareness and track and trace of trading events. I hear that at least one TIBCO customer for example is using CEP for Dodd-Frank reporting (also considered a TIBCO Hawk monitoring rule engine as well as TIBCO BusinessEvents CEP use case).

Forrester on the Future of Business Rules Platforms

Last week Forrester analysts Mike Gualtieri and John Rymer published a briefing on the “Future of Business Rules Platforms”, looking at the rules engines that have been so successful in providing business logic platforms for the BPM and SOA worlds. Their opinions are worth noting as they reflect what we are seeing in the marketplace… and their findings are somewhat given away in the report’s subtitle “Customers Are Moving To Event And Decision Management”…

Mike and John mention 2 patterns for CEP and business rule convergence:

  1. CEP can drive business rules via separate platforms. This is analagous to CEP (complex events) driving business processes and decisions as required, and where bolting CEP onto the front of mature BPM and SOA environments can make sense (note [1]).
  2. CEP can use rules to provide event pattern detection merged with business (decision) rules. This is the use of the same platform to provide an event-decision-action process (note [2]).

Notes.

[1] These patterns will be familiar to TIBCO customers through TIBCO BusinessEvents calling TIBCO BusinessWorks, and TIBCO ActiveMatrix BPM calling TIBCO BusinessEvents Decision Manager.

[2] Forrester also say, on the second pattern, “only Tibco is in a strong position to expand from use of rules to define event handlers into general business rules applications“!

Emergence of Case Management

govtcasemgmtI was at OMG this week when the submissions for the proposed “case management standard” (called Case Management Process Modelling or CMPM) were discussed. What was interesting was that one of the submissions spent a lot of time positioning ECA – Event Condition Action – rules as a means to model and execute the launching of activities and achievement of milestones in cases.

ECA is of course a widely used CEP paradigm: in a tool like TIBCO BusinessEvents the ECA rules are defined as conventional (in OMG parlance, PRR-type) production rules, where an event is just like any other fact to drive the rules. ECA rules also drive the traversal of state models (and state machines at runtime), as well as coordinate the conventional manual business processes. It is therefore interesting that Case Management – considered by many to be just an extension to BPM and mostly achievable with BPMN – is also proving to be an example of business process mechanism convergence (e.g. BPMN-type BPM orchestration of explicit activities, for example representing a plan, working alongside declarative rule constructs and technologies like ECA rules). For a real-world example of the latter, take a look at TIBCO Fulfillment Orchestration.

Of course, what neither the “rule technology world” nor these CMPM submissions have defined is a good business modeling technique to represent ECA rules. Such a business model may take the form of something as simple as a rule template, or something like a decision model. On the latter front, standardisation of decision models via the OMG DMN (Decision Model and Notation) effort seems to be gaining steam with pretty much all the main players in the process of registering (via Letters Of Intent) to take part. A good sign for process *and* rule standardisation … and of course decisions are somewhat useful in case management too!

ebizQ Interview with the Smart Enough Systems guys

EbizQ’s Joe McKendrick published an interview this month with the authors of “Smart Enough Systems” – Neil Raden and James Taylor (specialists in BI and decision management respectively) – titled “Decision Management technology enters the mainstream”. Joe writes:

Enter decision management systems, which not only are capable of analyzing the value of data to an organization, but then engage rules engines to automate the decision outcome.

James mentions event-driven systems in his comments about the types of solutions in the decision management space:

The decision management systems market – products that support automated decisioning in some fashion – consists of platform and application technologies. Platform products are things like business rules management systems, optimization engines and predictive analytic workbenches.  … In addition, there are pre-configured decision management systems that provide a decision engine -real-time offer management system or a fraud detection system for instance. These typically embed some rules and analytics technology as well as actual rules and models and handle a small group of related decisions. These are mostly in consumer risk, which includes underwriting, loan origination, and retail credit; fraud detection, which includes claims, credit card, and medical; and marketing, which includes next-best offers and retention. Of course, some CRM or marketing systems include these capabilities, just as some BPM or event-processing platforms include business rules and analytic capabilities.

James is quite correct there: indeed event-driven decision management seems to be the increasingly popular overlap between CEP and decision management. CEP technologies like the TIBCO BusinessEvents platform provide both such event-driven rule / decision automation (and management).