A DSL for Rules of Association
The use of rules of association may facilitate configuration: by querying them it is possible to check, for a given concept, which responsibilities can be assigned to it. The definition of a Domain Specific Language (DSL) for creating rules of association would be interesting, because it would allow the construction of more readable rules. In fact, a branch of should-dsl can be used. Should-DSL is a DSL for assertions, which are commonly used by coding techniques based on design by contract.
In fact, should-dsl's Predicate Matchers and some of the already available "ordinary" matchers such as respond_to, which checks if an object has a given attribute or method, and be_instance_of, which verifies if an object is of a given type, can be used for formatting rules. For instance, let's suppose we have a decorator that is supposed to work only with persons:
class A_person_decorator:
def __init__(decorated):
...
#checks if decorated is compliant to the rules
self.check_rules_of_association(decorated)
#if it is compliant, sets a pointer to it
self.decorated = decorated
...
def check_rules_of_association(decorated):
try:
#uses be_instance_of matcher to check if decorated is compliant
decorated |should| be_instance_of(Person)
except:
raise RuleOfAssociation('Person instance expected, instead %s passed' % type(decorated))
...
In the example above, type checking is necessary given the nature of Python language, however, the focus is in creating more complex rules related to the business rules.
Another interesting possibility is rule querying:
(i) For a given decorator, what are its rules of association (list its rules)?
(ii) For a given decorated, which decorators it can use (list which responsibilities a given class can assume)?
Query results could be presented in both human and machine readable ways. In other words, it would be some mechanism for querying rules, for both automatically checking associable objects in a object base, as well as informing people what is needed for configuring a concept to receive a given type of decoration.
A funny situation is that should-dsl matchers are used in unit tests to check the correct functioning of the rules... written in should-dsl.
The next post will describe some more details that decorators need to implement.
Friday, March 11, 2011
Wednesday, March 9, 2011
Enterprise Information Systems Patterns - Part V
Decorators X Subclasses
In the previous post, I redefined the level of abstraction for the framework, and now it is time to discuss a bit about Decorators. In order to facilitate the communication, a simple UML Class Diagram is provided by Figure 1, representing the structure after this redefinition.
An example
To better understand this, I will use a simple example: a teacher & researcher career. Let's suppose in our institution we have two kinds of teachers, the 20-hour and the 40-hour. While the first one is supposed only to teach, the second is also a researcher, and therefore holds more responsibilities. Given that there is vacancy, a teacher can change from one to another category. Since both types are "teachers", the ordinary object oriented solution would be to create a basic class named Teacher, which would hold the common features of the teaching career, and two subclasses, named 20-hour Teacher and 40-hour Teacher. With this architecture, I can see no simple solution than creating objects of the different classes and copying attributes back and forth every time someone changed his/her option of working 20 or 40 hours.
Moreover, imagine that teachers can also be assigned to administrative positions, such as department dean or university president, with a lot of new responsibilities associated, while still teaching. Keeping track of all these assignments and withdraws of responsibilities would be complex and error-prone. Also, I like to think that being a dean is getting a new responsibility, instead of becoming a different type of employee, in special when we think that this is a temporary assignment.
Now imagine that we have Persons and I decorate them as they are assigned to new responsibilities. In that way one can use the same object to register all career assignments: 20-hour, 40-hour, dean who still teaches, and even represent some administrative employee who teaches in a specific course - without loosing his/her attributions in the university's management. Teaching, researching, and administering would be decorators that could be associated and dissociated to objects as the business processes require (pay-as-you-go).
Another point is that each decorator must keep a set of rules of association, which is responsible for allowing or prohibiting objects to be assigned to responsibilities. Each "decorated" object is also responsible for keeping a record of its responsibilities (bi-directional relationship). If a given object respects the rules of association of a given decorator, it can be decorated by it, allowing a very flexible way of creating new "types" (mix-and-match).
This reasoning is also valid for other concepts, for instance, a given manufacturing cell (Machine), can run new operations as new tools are attached to it. The same is valid for organizations, as new machines and persons are associated to it.
An interesting point on using decorators is when we think of business entities ruled by laws and regulations, such as contracts, government agencies, or even governmental economic measures: as regulation or environment changes, things can be added and/or deleted from the entities' scope. Rule/Law decorators can be programmed to function during certain period of time, detaching themselves from the decorated objects when this period is over. As an example of this last case, let's think of a tax refunding mechanism that is valid only during recession periods and for specific types of sales.
Of course, a problem is that we have to deal with a potentially big set of decorators. However, given the mix-an-match possibilities of decoration, this number is smaller than the number of classes that would be created to map all possible combinations. Using an extreme example, if I have three concepts that can, each one, be assigned to four responsibilities, we would have 12 classes - and possibly by the use of multiple inheritance. By using decorators we would have 3 classes and 4 decorators, or 7 entities to manage (in 12 possible combinations, of course).
Said that, the next step is to discuss the rules of association and details for implementing decorators.
In the previous post, I redefined the level of abstraction for the framework, and now it is time to discuss a bit about Decorators. In order to facilitate the communication, a simple UML Class Diagram is provided by Figure 1, representing the structure after this redefinition.
Figure 1: UML Class Diagram of the new structure
In Figure 1, each of the (now) three abstract concepts (resource, node, movement) has two "opposite" subclasses (operation X material, person X machine, transformation X transportation) and one aggregator subclass (kit, organization, process). It is important to note that aggregators are Composite objects.
By (re)checking the Decorator pattern documentation, both on Wikipedia and on the classic book from Gamma and colleagues, we can find that:
- While subclassing adds behavior to all instances of the original class, decorating can provide new behavior, at runtime, for individual objects. At runtime means that decoration is a "pay-as-you-go" approach to adding responsibilities.
- Using decorators allows mix-and-matching of responsibilities.
- Decorator classes are free to add operations for specific functionalities.
- Using decorators facilitates system configuration, however, typically, it is necessary to deal with lots of small objects.
Therefore, by using decorators it is possible to, during a business process realization, create an object, associate and/or dissociate different responsibilities to it - in accordance to the process logic, and log all this. In that way, I have two main benefits:
i) The same object, with the same identifier, is used during the whole business process, there is no need for creating different objects of different classes.
ii) Given (i), auditing is facilitated, since it is not necessary to follow different objects, instead, the decoration of the same object is logged. Moreover, it is possible to follow a single object during all its life-cycle, including through different business process: after an object is created and validated - meaning that it reflects a real-world business entity - it will keep its identity forever.
Summarizing...
Thus, the benefits of using Decorators are:
Thus, the benefits of using Decorators are:
i) More dynamic and flexible enterprise systems, through the use of configuration and pay-as-you-go features.
ii) Easier auditing, given the fact that objects keep their class and identification while get new responsibilities.
An example
To better understand this, I will use a simple example: a teacher & researcher career. Let's suppose in our institution we have two kinds of teachers, the 20-hour and the 40-hour. While the first one is supposed only to teach, the second is also a researcher, and therefore holds more responsibilities. Given that there is vacancy, a teacher can change from one to another category. Since both types are "teachers", the ordinary object oriented solution would be to create a basic class named Teacher, which would hold the common features of the teaching career, and two subclasses, named 20-hour Teacher and 40-hour Teacher. With this architecture, I can see no simple solution than creating objects of the different classes and copying attributes back and forth every time someone changed his/her option of working 20 or 40 hours.
Moreover, imagine that teachers can also be assigned to administrative positions, such as department dean or university president, with a lot of new responsibilities associated, while still teaching. Keeping track of all these assignments and withdraws of responsibilities would be complex and error-prone. Also, I like to think that being a dean is getting a new responsibility, instead of becoming a different type of employee, in special when we think that this is a temporary assignment.
Now imagine that we have Persons and I decorate them as they are assigned to new responsibilities. In that way one can use the same object to register all career assignments: 20-hour, 40-hour, dean who still teaches, and even represent some administrative employee who teaches in a specific course - without loosing his/her attributions in the university's management. Teaching, researching, and administering would be decorators that could be associated and dissociated to objects as the business processes require (pay-as-you-go).
Another point is that each decorator must keep a set of rules of association, which is responsible for allowing or prohibiting objects to be assigned to responsibilities. Each "decorated" object is also responsible for keeping a record of its responsibilities (bi-directional relationship). If a given object respects the rules of association of a given decorator, it can be decorated by it, allowing a very flexible way of creating new "types" (mix-and-match).
This reasoning is also valid for other concepts, for instance, a given manufacturing cell (Machine), can run new operations as new tools are attached to it. The same is valid for organizations, as new machines and persons are associated to it.
An interesting point on using decorators is when we think of business entities ruled by laws and regulations, such as contracts, government agencies, or even governmental economic measures: as regulation or environment changes, things can be added and/or deleted from the entities' scope. Rule/Law decorators can be programmed to function during certain period of time, detaching themselves from the decorated objects when this period is over. As an example of this last case, let's think of a tax refunding mechanism that is valid only during recession periods and for specific types of sales.
Of course, a problem is that we have to deal with a potentially big set of decorators. However, given the mix-an-match possibilities of decoration, this number is smaller than the number of classes that would be created to map all possible combinations. Using an extreme example, if I have three concepts that can, each one, be assigned to four responsibilities, we would have 12 classes - and possibly by the use of multiple inheritance. By using decorators we would have 3 classes and 4 decorators, or 7 entities to manage (in 12 possible combinations, of course).
Said that, the next step is to discuss the rules of association and details for implementing decorators.
Thursday, March 3, 2011
Enterprise Information Systems Patterns - Part IV
Changing the abstraction level
As expected, the high abstraction level brought by the use of only 4 concepts to try to define a whole system would bring side effects*. The basic one is that, if on one hand abstract concepts are highly reusable, on the other, they need to be extended a lot in order to become useful for representing concrete concepts, even the basic ones, such as Person or Product. I mean, a Node can be the abstract concept behind a Person object as well as a Factory object. However, you will need to provide a lot of extra code to make these concepts work. Given that I don't want to create lots of subclasses, the alternative is to use Decorators. However, in the same way, it is not good to abuse of a given technique, even if this technique is a well known design pattern. What I was going to do was create a lot of decorators instead of creating a lot of subclasses. In other words, I would be going against my main goal, which is to create a framework extensible by configuration, instead of programming.
Therefore, I stopped a bit to rethink the framework's abstraction level. One thing I realized is that it would be very hard to configure too abstract concepts, thus, by creating one more class level, I could find less generic but easier to reuse concepts. A side effect is that I can get rid of one more of the original concepts - Path. In other words, I shortened the width and broadened the length of the class hierarchy. The idea now is to use three classes: Resource, representing the production resources; Node, representing the entities that transform and transport these resources; and Movements, which represent transformations and transportations of resources. Each of these abstract concepts has three subclasses, representing two "opposite" concepts and an aggregator of these first two. Thus, the structure will be like this:
1)Resource()
-Material(Resource): product, component, tool, document, raw material...
-Operation(Resource): human operation and machine operation, as well as their derivatives.
-Kit(Resource): a collective of material and/or immaterial resources. Ex.: bundled services and components for manufacturing.
Comments:
a)Alternative terms such as Object (material) and Action (immaterial) can also be considered, however, the term Object can bring a lot of trouble in programming.
b)Another point is information, which is not material, neither it is something like operation. Document is used as the physical representation of information. c)Structural and Behavioral would be alternatives, however, they seem to be too academic to be used.
2)Node()
-Person(Node): employee, supplier's contact person, free lancer...
-Machine(Node): hardware, software, drill machine...
-Organization(Node): a collective of machines and/or persons. Ex.: manufacturing cell, department, company, government.
3)Movement()
-Transformation(Movement): an "internal" movement. Ex: transforming raw material, writing a report
-Transportation(Movement): a movement of resources between two nodes. Ex: moving a component from one workstation to another, moving a document from one department to another
-Process(Movement): a collective of transformations and/or transportations, in other words, a business process.
Comments
a) There maybe some confusion between Transformations and Transportations and Operations, but they represent different things. Movements use operations to transform or to transport resources. For instance, an industrial transformation uses a given machine operation during a given period of time to transform a given quantity of raw material into a component.
The next step is to define how to extend the basic concepts to make them work properly in concrete business processes representations, which will be discussed in the next post of this series.
* As I said in my first post of this series, if you need a "real-world" and flexible Python framework, give ERP5 - from which the basic ideas for this framework were taken - a try. Remember that this framework is a didactic one, therefore, some assumptions are simplified.
(The Change Log maps this series of posts to commits in GitHub)
As expected, the high abstraction level brought by the use of only 4 concepts to try to define a whole system would bring side effects*. The basic one is that, if on one hand abstract concepts are highly reusable, on the other, they need to be extended a lot in order to become useful for representing concrete concepts, even the basic ones, such as Person or Product. I mean, a Node can be the abstract concept behind a Person object as well as a Factory object. However, you will need to provide a lot of extra code to make these concepts work. Given that I don't want to create lots of subclasses, the alternative is to use Decorators. However, in the same way, it is not good to abuse of a given technique, even if this technique is a well known design pattern. What I was going to do was create a lot of decorators instead of creating a lot of subclasses. In other words, I would be going against my main goal, which is to create a framework extensible by configuration, instead of programming.
Therefore, I stopped a bit to rethink the framework's abstraction level. One thing I realized is that it would be very hard to configure too abstract concepts, thus, by creating one more class level, I could find less generic but easier to reuse concepts. A side effect is that I can get rid of one more of the original concepts - Path. In other words, I shortened the width and broadened the length of the class hierarchy. The idea now is to use three classes: Resource, representing the production resources; Node, representing the entities that transform and transport these resources; and Movements, which represent transformations and transportations of resources. Each of these abstract concepts has three subclasses, representing two "opposite" concepts and an aggregator of these first two. Thus, the structure will be like this:
1)Resource()
-Material(Resource): product, component, tool, document, raw material...
-Operation(Resource): human operation and machine operation, as well as their derivatives.
-Kit(Resource): a collective of material and/or immaterial resources. Ex.: bundled services and components for manufacturing.
Comments:
a)Alternative terms such as Object (material) and Action (immaterial) can also be considered, however, the term Object can bring a lot of trouble in programming.
b)Another point is information, which is not material, neither it is something like operation. Document is used as the physical representation of information. c)Structural and Behavioral would be alternatives, however, they seem to be too academic to be used.
2)Node()
-Person(Node): employee, supplier's contact person, free lancer...
-Machine(Node): hardware, software, drill machine...
-Organization(Node): a collective of machines and/or persons. Ex.: manufacturing cell, department, company, government.
3)Movement()
-Transformation(Movement): an "internal" movement. Ex: transforming raw material, writing a report
-Transportation(Movement): a movement of resources between two nodes. Ex: moving a component from one workstation to another, moving a document from one department to another
-Process(Movement): a collective of transformations and/or transportations, in other words, a business process.
Comments
a) There maybe some confusion between Transformations and Transportations and Operations, but they represent different things. Movements use operations to transform or to transport resources. For instance, an industrial transformation uses a given machine operation during a given period of time to transform a given quantity of raw material into a component.
The next step is to define how to extend the basic concepts to make them work properly in concrete business processes representations, which will be discussed in the next post of this series.
* As I said in my first post of this series, if you need a "real-world" and flexible Python framework, give ERP5 - from which the basic ideas for this framework were taken - a try. Remember that this framework is a didactic one, therefore, some assumptions are simplified.
(The Change Log maps this series of posts to commits in GitHub)
Friday, February 25, 2011
The obvious, but yet ill applied, relation of Open Source and Auditability
In this post I comment two blog entries related to the use of open source software for auditing purposes in the banking area.
In the first one, Open Source Banking for Financial Stability, Jean Paul Smets affirms that some serious problems presented by financial products not compliant to the regulations are caused by software systems which "are not subject to any kind of regulatory control and, therefore, may be used to circumvent regulations either through functional inconsistencies or through temporal inconsistencies." Therefore, he defends the idea of using a reference open source software, without getting rid of the software in which banks invested for decades. This reference, besides offering an auditable software representation of the regulations rules, would be used "to provide to regulation authorities a very accurate picture of [banks] activity, much more precise than what accounting can provide." - by importing the data handled by existing bank's software.
In the second post, Python Could Become the Language of Finance, Jonathan Allen informs that SEC is proposing that Asset Backed Securities should include a “program that gives effect to the flow of funds, or “waterfall,” provisions of the transaction.” Waterfall in this case "refers to how some bonds are broken into levels, where those who bought into the higher level must be paid off before the lower level sees any money." The interesting part is that SEC is planning to mandate Python as the language for building this reporting tool, because it is open source, uses a standalone interpreter, and is supported by many platforms, including proprietary software such as .NET.
Although the first post is an opinion, the second isn't, and both present good arguments to motivate regulatory organs to use more open source for auditing tasks in the financial area, and, why not say, in other areas. The use of open source reference softwares would (i) help people compare their implementations of regulatory rules with a standard implementation, (ii) provide cheaper software that could be customized for specific adopters, while following the regulation, (iii) provide automated tools for regulatory agencies to audit data. In that way, maybe the number of financial scandals could be reduced, as well as regulatory work in other areas, such as environment, could be simplified and streamlined.
In the first one, Open Source Banking for Financial Stability, Jean Paul Smets affirms that some serious problems presented by financial products not compliant to the regulations are caused by software systems which "are not subject to any kind of regulatory control and, therefore, may be used to circumvent regulations either through functional inconsistencies or through temporal inconsistencies." Therefore, he defends the idea of using a reference open source software, without getting rid of the software in which banks invested for decades. This reference, besides offering an auditable software representation of the regulations rules, would be used "to provide to regulation authorities a very accurate picture of [banks] activity, much more precise than what accounting can provide." - by importing the data handled by existing bank's software.
In the second post, Python Could Become the Language of Finance, Jonathan Allen informs that SEC is proposing that Asset Backed Securities should include a “program that gives effect to the flow of funds, or “waterfall,” provisions of the transaction.” Waterfall in this case "refers to how some bonds are broken into levels, where those who bought into the higher level must be paid off before the lower level sees any money." The interesting part is that SEC is planning to mandate Python as the language for building this reporting tool, because it is open source, uses a standalone interpreter, and is supported by many platforms, including proprietary software such as .NET.
Although the first post is an opinion, the second isn't, and both present good arguments to motivate regulatory organs to use more open source for auditing tasks in the financial area, and, why not say, in other areas. The use of open source reference softwares would (i) help people compare their implementations of regulatory rules with a standard implementation, (ii) provide cheaper software that could be customized for specific adopters, while following the regulation, (iii) provide automated tools for regulatory agencies to audit data. In that way, maybe the number of financial scandals could be reduced, as well as regulatory work in other areas, such as environment, could be simplified and streamlined.
Sunday, February 13, 2011
Enterprise Information Systems Patterns - Part III
After my last post, the code in Github changed many times, since I was experimenting with both the basic concepts and the testing stack we are using. We decided not to write features (BDD's Given-When-Then constructs) for the configuration of Resources, Nodes, and Movements, since it would be overkill (as you can see in the sketch branch, from December 12, 2010): by reading the specs anyone can understand and check the functioning of configuration.
The comments below are based on the last implementation (January 09, 2011), which doesn't implement Path nor the parsers for the configurators. We are considering writing features for Path configurations, because Paths represent business processes, and therefore, it make sense to have high level descriptions of requirements.
Currently, the proposal is based on two base classes, from which all others derive. The first one, Configurable (previously known as Maskable), is the base for building the abstract concepts, and the second one, Configurator, is used for providing a "friend" class for each concept which is responsible for managing the configuration of the concepts' instances.
Besides the basic attributes, Configurable abstract class holds a "To Do" method, which is simply a placeholder to remember that a method for doing "basic things" with instances should be implemented by subclasses. Configurators on the other hand must be able to parse, persist, and retrieve configurations to be applied to concept's instances. They are multitons, in other words, for a given key, composed by configuration's mask and version attributes, only one configurator object may exist. Therefore, every concept class, such as Movement, has a configurator class, such as Movement_Configurator.
Configurations supply the following attributes for instances (spec files provide complete configuration examples):
-Mask: defines a specific type for the instance. Example: payment (a payment movement).
-Version: holds the version of the mask, multiple versions can be in use at the same moment. Example: monthly (monthly payment).
-Category: associates a category to a specific concept. Example: financial (monthly payment is a movement of category financial).
-Description: provides a description for the key mask+version. Example: ' a monthly payment from a organization to a person'. Note that, in this example, organization and person can also be categories.
Right now the implementation is in a very simple state, the next step is to implement Path, which is expected to reveal more requirements for making this framework more usable. For testing, we are going to use some real-world workflow implemented in ERP5, applying the mappings presented in the technical report "Mapping Business Process Modeling constructs to Behavior Driven Development Ubiquitous Language" to describe the workflow using Given-When-Then.
The comments below are based on the last implementation (January 09, 2011), which doesn't implement Path nor the parsers for the configurators. We are considering writing features for Path configurations, because Paths represent business processes, and therefore, it make sense to have high level descriptions of requirements.
Currently, the proposal is based on two base classes, from which all others derive. The first one, Configurable (previously known as Maskable), is the base for building the abstract concepts, and the second one, Configurator, is used for providing a "friend" class for each concept which is responsible for managing the configuration of the concepts' instances.
Besides the basic attributes, Configurable abstract class holds a "To Do" method, which is simply a placeholder to remember that a method for doing "basic things" with instances should be implemented by subclasses. Configurators on the other hand must be able to parse, persist, and retrieve configurations to be applied to concept's instances. They are multitons, in other words, for a given key, composed by configuration's mask and version attributes, only one configurator object may exist. Therefore, every concept class, such as Movement, has a configurator class, such as Movement_Configurator.
Configurations supply the following attributes for instances (spec files provide complete configuration examples):
-Mask: defines a specific type for the instance. Example: payment (a payment movement).
-Version: holds the version of the mask, multiple versions can be in use at the same moment. Example: monthly (monthly payment).
-Category: associates a category to a specific concept. Example: financial (monthly payment is a movement of category financial).
-Description: provides a description for the key mask+version. Example: ' a monthly payment from a organization to a person'. Note that, in this example, organization and person can also be categories.
Right now the implementation is in a very simple state, the next step is to implement Path, which is expected to reveal more requirements for making this framework more usable. For testing, we are going to use some real-world workflow implemented in ERP5, applying the mappings presented in the technical report "Mapping Business Process Modeling constructs to Behavior Driven Development Ubiquitous Language" to describe the workflow using Given-When-Then.
Subscribe to:
Posts (Atom)