Ideas on Enterprise Information Systems Development

This blog is devoted to ideas on Enterprise Information Systems (EIS) development. It focuses on Lean Thinking, Agile Methods, and Free/Open Source Software, as means of improving EIS development and evolution, under a more practical than academical view. You may find here a lot of "thinking aloud" material, sometimes without scientific treatment... don't worry, this is a blog!
Every post is marked with at least one of Product or Process labels, meaning that they are related to execution techniques (programming and testing) or management techniques (planning and monitoring), respectively.

Thursday, August 18, 2011

Enterprise Information Systems Patterns - Part XII

Step by Step Programming using EIS Patterns
This post will describe, step by step and using the example "bank system", how to program using the framework and Fluidity + Extreme Fluidity.

Step -1
You must know the following techniques and tools:
a) BDD using Lettuce.
b) The Decorator Design Pattern.
c) Should-dsl, to define expectations in both acceptance and unit tests.
d) Python Unittest Library.
e) How to use Python Decorators (@).

Step 0
Understand how and why things are done in the framework:
a) Start at Part IV, the previous parts focus on discussions and experimentations which are not used in the framework anymore.
b) Why Decorators are used to extend classes, explained in Part V.
c) Why should-dsl is also used, besides in tests, to define contracts between objects, explained in Part VI.
d) How things described in parts IV to VI are implemented in general is described in Part VII -  if you feel it too abstract, I suggest going to Part IX.
e) Part VIII only describes the status of the project by that time, you can ignore it, but it is necessary to accompany the change log to become aware of important changes in the code.
f) Part IX describes conceptually how to program using the framework. Part X describes in more detail the use of Fluidity, and Part XI explains the evolution of the concepts. This post does the same of Part IX, but showing the code instead.

Reading the posts will give you an idea of the modus facendi of the framework, however, it is necessary to start programming... probably it is necessary to go back and forth from code to these posts. The following steps are a kind of guide, but in no way it is meant to describe a complete development process, this is one thing that you developer should infer. Some details may be omitted to avoid too long descriptions.

*It is strongly recommended that you check the unitary tests to understand how the framework core classes work in detail*

Right now we are using BDD because we don't have a nice BLDD tool to work with. Therefore you are supposed to describe the business process in BDD terms, by interpreting a template workflow, and then incrementally define the features' steps.

The following steps were developed on top of a specific example, however, they can be generalized.

Step 1
Based on the bank system example, let's suppose that the code is part of a bigger system which have already core concepts such as Employee and Bank Account.
First the process is defined in Fluidity, them, incrementally, the scenarios are defined and finally their steps.  
Fluidity:
    state('requested')
    transition(from_='requested', event='create_loan_request', to='request_created')
    state('request_created')
Scenario:
Scenario Outline: Individual Customer asks for loan
    Given I am a registered Credit Analyst
    And an individual customer with account number <account number> asks for  a personal loan
    And the loan request is of <desired value>
    When I confirm the loan request
    Then a new loan request with the <account number> and <desired value> is created
    And the new loan request is associated to the Credit Analyst
Steps:
@step(u'Given I am a registered Credit Analyst')
def given_i_am_a_registered_credit_analyst(step):
     world.a_person = Person()
     an_employee_decorator = EmployeeDecorator()
     an_employee_decorator.decorate(world.a_person)
     world.credit_analyst = CreditAnalystDecorator('09876-5')
     world.credit_analyst.decorate(world.a_person)

Step 2:
By analyzing the code above it is identified the necessity of developing a credit analyst business decorator. You will do only the basic necessary to the first step pass: create a skeleton of CreditAnalystDecorator, and go back to next step afterwards.
At this point it is already important to define the Rule of Association for this decorator, which is "to be a credit analyst, someone must be a bank employee first". Before writing the rule code, a test for checking if the rule is being enforced and if the decoration is working properly must be prepared:
    def it_decorates_a_person(self):
        #should fail
        (self.a_credit_analyst_decorator.decorate, self.a_person) |should| throw(AssociationError)
        #should work
        an_employee_decorator = EmployeeDecorator()
        an_employee_decorator.decorate(self.a_person)
        self.a_credit_analyst_decorator.decorate(self.a_person)
        self.a_credit_analyst_decorator.decorated |should| be(self.a_person)
        self.a_credit_analyst_decorator.decorated |should| have(2).decorators

Now the rule's code can be written:
    @rule('association')
    def rule_should_contain_employee_decorator(self, decorated):
        ''' Decorated object should be already decorated by Employee '''
        decorated |should| be_decorated_by(EmployeeDecorator)

This rule will assert that only if the person is an employee then he/she can be a credit analyst, and it is implemented as a method of the credit analyst decorator class. The be_decorated_by matcher was created specifically for this framework.

Attention: as of versions 0.7.3 of the core and 0.1.3 of the example, a singleton Rule Manager object started to be used, in order to promote reuse of rules. This object is composed by a "rules" object, which in the core is called Core Rules. Checking the tests of the example it is possible to see that other rule base is used, in the case a subclass of the core rules, called Bank System Rule Base. It is important to note that some tests use the core rules, simply because they don't need to use rules specific to the bank system. In other words, if you need a new rule, it must be created into your rule base object - making it hard-coded, however it is envisioned a way of loading them from a file or other configuration object, maybe using an even higher DSL to define them.


Step 3:
Now that the basic credit analyst decorator is defined and tested, we go back to the acceptance level, in this example, we will jump straight to the "Then" step because the others are straightforward. At this point, the business process starts to be assembled.
@step(u'Then a new loan request with the (.+) and (.+) is created')
def then_a_new_loan_request_with_the_account_number_and_desired_value_is_created(step, account_number, desired_value):
     (...)
    Creates the business process object:
    world.an_individual_credit_operation = Process('Individual Customer Credit Operation')
    Defines the process' source and destination:
    world.an_individual_credit_operation.set_source(world.the_company)
    world.an_individual_credit_operation.set_destination(world.a_client)
    Configures the process using a template state machine:
    template = LoanProcess()
    Uses xFluidity's to get a configuration based on the template:
    configurator = StateMachineConfigurator(template)
    configurator.configure(world.an_individual_credit_operation)
    At this point, the process object (an_individual_credit_operation) has absorbed the template state machine
    Now it is time to configure the first movement - loan request creation
    The object the_movement will be a proxy for CreditAnalystDecorator
.create_loan_request. This movement will be a Transformation, having credit_analyst.decorated (the associated person) as both source and destination, and an_individual_credit_operation.create_loan_request as the activity - that's an important point: create_loan_request is a method, symmetric to a workflow transition, which was injected into the process object by xFluidity.
    the_movement = world.an_individual_credit_operation.configure_activity_logger(world.credit_analyst.decorated, world.credit_analyst.decorated, world.an_individual_credit_operation.create_loan_request, CreditAnalystDecorator.create_loan_request)
    After this, the collection of movements of the process should contain the recently configured movement (found by the activity's name)
    world.an_individual_credit_operation.movements |should| contain(the_movement.activity.__name__)


Step 4:
At this point the CreditAnalystDecorator class has no create_loan_request method, thus it is time to go back to the unit level and write a test to this specific requirement:
    def it_creates_a_loan_request(self):
        an_employee_decorator = EmployeeDecorator()
        an_employee_decorator.decorate(self.a_person)
        self.a_credit_analyst_decorator.decorate(self.a_person)
        self.a_credit_analyst_decorator.create_loan_request(self.an_account, 10000) #self.an_account was created in the setUp()
        self.a_person.input_area |should| contain('1234567-8')

Now it is implemented as an @operation:

    @operation(category='business')
    def create_loan_request(self, account, value):
        ''' creates a loan request '''
        loan_request = LoanRequest(account, value, self)
        #Places the loan_request in the node's input area
        self.decorated.input_area[loan_request.account.number] = loan_request

Note that a LoanRequest class must be defined, which drive us to the next step.

Step 5:
The first thing to do is to create a unit test for LoanRequest and then create the class itself, which are trivial tasks. After running all the necessary tests, we can go back to the acceptance level, given that the loan request class now exists and it is operating properly.

Step 6:
    Time to run the loan request creation and store the execution context:
    the_movement.context = world.an_individual_credit_operation.run_activity(the_movement, world.credit_analyst, world.account, desired_value)
    Since the create loan request transition was fired, the process should be on state 'request_created'
    world.an_individual_credit_operation.current_state() |should| equal_to('request_created') 


Step 7:   
Now it is time to check the 'And' clause of this step, which is:

@step(u'And the new loan request is associated to the Credit Analyst')
def and_the_new_loan_request_is_associated_to_the_credit_analyst(step):
    #... this association was done during loan creation, just checking
    world.a_person.input_area[world.account.number].analyst |should| be(world.credit_analyst)

This step was easy, since the way the loan request creation was implemented already places it on the decorated (the Person) processing_area.


The next step would be go to the next process transition and implement it using a sequence similar to the one presented above.


In summary, developing in the framework involves:
1) Describing a user story guided by a business process template.
2) Developing the code for the story steps, which will identify new responsibilities, to be implemented into the decorators and resources.
3) Developing the code for the decorators, including defining their association rules and @operations.
4) Developing the code for resources, including defining their association rules.
5) Detailing the steps: use movements to wrap and associate @operations to each workflow activity, providing full control through the workflow engine while logging the execution context.

The use of the process template and the association of decorators' methods to business process activities provides the adherence of the application logic to the process' logic. Currently, BDD is in use, therefore the linking to the process logic is done manually. However, it is envisioned the creation of a tool to generate the steps skeletons in a way adherent to the process logic.

Thursday, August 11, 2011

Enterprise Information Systems Patterns - Part XI

Evolving the interpretation of the concepts
As the framework evolves, and we play with it through the examples, new interpretations of the concepts appear. In fact, after playing with Fluidity, we realized that Transformation and Transportation were not necessarily implemented as classes, but categories of movements instead, given that they behave in the same way - it is only a question of explicitly saying if the work item is transformed into a Node (a transformation) or simply moved from one node to another (a transportation).

The history behind this is that, when developing the example, it was noticed that even a "simple" resource movement would need to be logged, as well as it could be associated to, or even generate, other resource(s). When we realized this, it was clear that transportations would potentially need the same machinery used by transformations, hence, they share a common behavior, which should be implemented into Movement class. Therefore, they were transformed into categories - although the Category class is not in use yet. Figure 1 shows how the framework's ontology is interpreted now.

Figure 1: EIS Patterns Ontology

Another important point is that the relationship "is_a" between Operation and Resource does not represent a classic object oriented inheritance relationship. In other words, Operation is not a class, instead, it is a Python Decorator which is applied to Person and Machine methods to mark them as Business Operations. Therefore, when it is said that "a Node performs an Operation", it means that some of the Node's method is decorated as an operation - or, this node is able of performing this operation. Finally, the relationship "operates_on" between Operation and Resource means that operations are always transforming or transporting resources, through movements which encapsulates them (the operations).

In summary:
a) An operation is an abstract representation of a production operation.
b) A node's method is decorated by an operation to inform that the original abstract concept is now represented concretely by the method.
c) Inside a business process, a movement is used to encapsulate and log the execution context of the concrete operation defined in (b).
d) Running a process consists of executing and logging the movements defined in (c).

Part IX of this series explains in less abstract terms how to deal with operations and movements. Part X explains how to do this using Fluidity and Extreme Fluidity. As expected, in order to understand how the whole thing is implemented, it is necessary to check the EIS Patterns Examples project, in special, how BDD steps are implemented.