Ideas on Enterprise Information Systems Development

This blog is devoted to ideas on Enterprise Information Systems (EIS) development. It focuses on Lean Thinking, Agile Methods, and Free/Open Source Software, as means of improving EIS development and evolution, under a more practical than academical view. You may find here a lot of "thinking aloud" material, sometimes without scientific treatment... don't worry, this is a blog!
Every post is marked with at least one of Product or Process labels, meaning that they are related to execution techniques (programming and testing) or management techniques (planning and monitoring), respectively.

Sunday, October 31, 2010

Effort Metrics Sophisma - Part I

This a summarization and translation to English of a presentation prepared by me and an associated researcher in 2008. It was used to support our arguments against detailed one-year plans, a demand from the management of a big and interesting project we've been taking part since 2006.

The goal of this presentation was to show that detailed planning (months ahead) in software development either:
1) Force waterfall cycles.
2) Are inaccurate, leading to a lot of changes in the baselines, which in turn lead to high costs of control function.

Planning in the (hard) Industry

Traditionally, the industry follows waterfall cycles (in spite of Concurrent Engineering), as you can see, for instance, in Gera's lifecycle. In the other engineerings, product metrics are well established, in some cases for centuries. If you say that 10 meters of wall will use 500 bricks, it will use 500 bricks in Brazil, China, or Denmark. In other words, in "traditional" engineering, products are tangible - you can measure them, doesn't matter who made them or where you are. A brick is a brick, and that's it.

(before you say that someone can count lines of code, even this metric is not exact, since I can write the same code using more or less lines, or even write different algorithms with different numbers of lines, and even though, they do the same thing - so please forget that LOC can be exact or tangible)

First Logical Chaining

Detailed planning needs → detailed estimates
Detailed estimates needs → detailed product design (for defining activities in detail)
Detailed planning needs → detailed product's design
So we have
Detailed product's design (in the beginning of the project) → waterfall
Detailed planning → waterfall!

First Conclusions

The arguments could stop here since it would fall into a discussion of whether or not using a waterfall cycle, which is already known to be inefficient in software development. 

Still, it could be suggested that in some cases it is necessary to make detailed product design in the beginning, for example, for Government contracts. In the next post I will show that even in that cases - when Law forces Big Design Up Front (BDUF) - the planning won't be accurate.

Thursday, October 28, 2010

Free/Open Source ERP (FOS-ERP) - Part VI

In 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. This post is part of a series that revisits this paper. Please refer to the first post of this series to better understand the structure used in this comparison. 

Opportunities and Challenges
FOS-ERP offer a series of opportunities for actors that are currently out of (or ill inserted) the ERP market. Since there is nothing such as free lunch, these opportunities come together with a series of challenges, as listed below.

For smaller consulting firms:
a) Opportunities:
P-ERP vendors generally impose rigid procedures, associated to high costs, for firms that want to enter their partner network, raising the difficulties for smaller enterprises to become players in this market. In contrast, smaller consulting firms can enter the FOS-ERP market in an incremental way, increasing their commitment to a project as new business opportunities appear and bring more financial income. In other words, firms can start contributing with small improvements to the project as a way of gaining knowledge on the system platform and framework, and, as customers appear, more money can be invested on a growing commitment to the project.
b) Challenges:
If on one hand it is easier to enter the market, on the other it is harder to retain clients: a broader consultancy basis empowers the demand side, making customers more demanding and reducing profit margins.
Keeping the quality level among a heterogeneous network of consulting services providers is also a major challenge. FOS-ERP in general lack certification and quality assurance programs that guarantee service levels to clients. However, exactly those programs keep smaller consulting firms way from P-ERP, pushing them towards FOS-ERP. For a small consulting firm, a possible solution to this deadlock is to start with smaller, less demanding projects, and then go towards bigger ones, as its deployment processes and related activities gain maturity. This maturity will become the competitive advantage of the firm on a high competitive FOS-ERP market.

For smaller adopters:
a) Opportunities:
lower costs open new opportunities for Small and Medium Enterprises (SME) to become ERP adopters. With globalization, small firms suffer more and more with competition, and when they try to modernize their processes, they hit the wall of global players’ high costs, or have to adopt smaller off-the-shelf (and also proprietary) solutions that ties them to a single supplier that usually doesn’t have a partner network. In contrast, FOS-ERP are less expensive and support can be found in different ways, including individuals and small consulting firms.
This is also truth for local governments and countries in development in general, by reducing costs and technological dependency from global players. In fact, FOSS in general is an opportunity for countries in development to shift from buyers to players in the software industry.
b) Challenges:
lower costs can also mean that adopters have to deal with lower service levels, then stressing the necessity of carefully evaluating FOS-ERP options and the maturity of their supportive services. Actually, as said before, consulting certification is yet on the early stages for FOS-ERP, thus quality of service must be carefully addressed during contract negotiation.

For researchers:
a) Opportunities:
I've been contributing to ERP5 since its conception. During this time it was possible to observe, and sometimes take part, of all the process that compose an ERP solution, from conception and development, to business models, deployment, operation and maintenance, and evolution. This is a really good opportunity, since most research papers on ERP are related to deployment and operation, given that P-ERP companies don’t usually open their solution's internals for researchers. Smaller research groups can find their way in this area by getting associated to a FOS-ERP project, and contributing to specific parts of it.
b) Challenges:
If on one hand the openness of FOS-ERP may give researchers more information on their internal features and development processes, on the other hand it is harder to get information from a distributed set of partners that sometimes carry informal relationships. Social and economical aspects, like reward structures, must be taken into account to understand the dynamics of FOS-ERP, like in every FOSS, bringing more components to be analyzed.

For individuals
a) Opportunities:
FOS-ERP represent an unique opportunity for an individual to install an ERP framework and understand its internals. It is the chance of participating in a big software development project without being an employee of a big company. Also, the developer can incrementally gain knowledge of the system, and get free support from the community, without the necessary investment on the P-ERP expensive training and certification programs. In the future, these advantages can make more free-lance developers enter FOS-ERP communities, currently formed mostly by consulting companies employees.
b) Challenges:
learning the internals of a FOSS in general means to spend considerable time in understanding system architecture, design decisions, and specific features. Moreover, FOS-ERP still lack courseware in general*** to help accelerating the learning process, and many times the individual must count on Web sites, mailing lists, discussion forums, and the good will of community members to acquire deeper knowledge on the framework.

***When I originally wrote this in 2007, ERP5's OSOE Project didn't exist yet. My opinion on ERP5 can be biased, but the truth is that Nexedi agreed on this point and decided to start a web based training program to fill this gap.

Free/Open Source ERP (FOS-ERP) - Part V

In 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. This post is part of a series that revisits this paper. Please refer to the first post of this series to better understand the structure used in this comparison. 

During operation, environmental changes will lead to corresponding changes in the system. These changes can be conducted by the original vendor, other service providers, or even the adopter with help of the community. Operation costs are reduced by other technologies of which FOS-ERP usually rely on, such as Operating Systems, Office Suites, Middleware etc.  

A conclusive remark is that, although experience has shown that most of the times the adopter will not be active on tasks that involve coding, FOS-ERP is still a good choice, since it reduces vendor dependency. Moreover, the openness of code delivers much more opportunities for creating competitive differential by implementing innovative processes or algorithms and integrating to other solutions.

Until this point I talked about the differences between FOS-ERP and P-ERP on the adopter side. For understanding the differences on the vendor side, please refer to the original article. In the next post I will talk about the opportunities and challenges brought by FOS-ERP.

Wednesday, October 20, 2010

Free/Open Source ERP (FOS-ERP) - Part IV

In 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. This post is part of a series that revisits this paper. Please refer to the first post of this series to better understand the structure used in this comparison.

Detailed Design and Implementation

The detailed design phase focuses on refining models and parametrization. The implementation phase concentrates on programming/configuring, validating, integrating and releasing modules for initial use. Remember that if you are using an interactive and incremental process, you will do this module by module, or even by implementing intermediary versions of modules: modules can have partial implementations that will be refined by operation, in other words, we can go back to refine a module in a future iteration. The Figure 1 revisits Gera's life cycle by changing it from a Waterfall to an Iterative & Incremental perspective.

Figure 1: Iterative & Incremental Gera's Lifecycle

In Figure 1, the outer loop refers to the iterations, while the internal loop refers to the possibility of releasing incremental versions of the modules.

If the adopter opts to participate actively in the selected project, deeper design and implementation decisions are involved, as well as the necessity of investing more human and financial resources for understanding the FOS-ERP framework, developing and maintaining parts of it, and managing the relationship with the project's community.

If a FOS-ERP vendor is involved, customization and maintenance contracts must define the responsibilities of each part on the deployment process. For instance, what the vendor should do if it finds a bug injected by the customer? What is the priority, for the vendor, for correcting this bug? Actually, is the vendor  responsible for correcting this bug, since for this part of the system the adopter decided to take advantage of the solution’s free license, therefore exempting the vendor of responsibility for the bug?

Furthermore, the adopter has the option of assuming different grades of involvement for each module. For ordinary modules, like payroll, the adopter can let the vendor do the work. However, for strategic modules, for which the adopter believes that holds competitive advantage by following its own business processes, the adopter can take an active role from detailed design to implementation and maintenance, to assure that the business knowledge, or at least the more precious details that keep the competitive advantage, will be kept in the adopter company. In that situation the vendor is limited to act as a kind of advisor to the adopter.

A very interesting point is the openness of parts customized for and sponsored by a specific adopter. Maybe the adopter doesn’t want to become a developer at all – which is very likely to happen, but it still wants to keep some tailored parts of the system in secrecy. In these cases, the ERP's licensing terms must be adapted, so that general openness of the code is guaranteed, while some client-sponsored customized parts can be kept closed.

Although this last point can seem to be nonsense in FOSS terms, it is a common real-life situation in FOS-ERP. In fact, I know a case that an adopter company sponsored the whole development of an FOS-ERP during a three-year period, without becoming a prosumer and keeping only a specific algorithm in secret. The original license had to be changed to fit this customer demand.

Thursday, October 14, 2010

Free/Open Source ERP (FOS-ERP) - Part III

Free/Open Source ERP (FOS-ERP), Process, ProductIn 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. This post is part of a series that revisits this paper. Please refer to the first post of this series to better understand the structure used in this comparison.

Requirements and Preliminary Design

Given that most software development (and customization) today is done (or should be done) through interactive and incremental life cycles, the requirements, preliminary design, detailed design, and implementation phases are performed in a loop.

Following its list of priority requirements, the adopter can model its main business processes – as part of the Preliminary Design – in order to check how the different ERP systems fit to them. At this point, FOS-ERP need  to be evaluated using the criteria traditionally used to evaluate ERPs in general, as well as criteria related specifically to Free/Open Source Software (FOSS) in general, such as maturity of the community and the levels of support.

An interesting point regarding FOS-ERP is that, although it can produce a smaller financial impact, it may bring a bigger knowledge and innovation impact. The access to the source code in FOS-ERP can drive to a much better exploration of the ERP’s capabilities, thus allowing a better implementation of differentiated solutions. Of course, software development resources must be available to reach this, what means that for smaller organizations this is usually not possible.

From this standpoint, the strategic positioning of an adopter in relation to a FOS-ERP seems to be of greatest importance, given the possibility of deriving competitive advantage from the source code. Therefore, the adopter must decide to behave as a simple consumer, only obtaining the solution from a vendor or the community, or become a prosumer, by mixing passively acquiring commodity parts of the system with actively developing strategic ones by itself. Thus it is clear that when an adopter considers FOS-ERP as an alternative, it should also consider developing parts of the system to fit its requirements – taking into account that this kind of positioning represents allocating managerial and technical resources for development tasks in a FOSS environment.

Sunday, October 10, 2010

Free/Open Source ERP (FOS-ERP) - Part II

In 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. This post is part of a series that revisits this paper. Please refer to the first post of this series to better understand the structure used in this comparison.

During this phase, high-level objectives are established, such as the acquisition strategy, preliminary time and cost baselines, as well as the expected impact of the ERP adoption in the organization. If besides customization, development is necessary, in the case of FOS-ERP, the level of involvement of the adopter in this development can be established. In other words, even from this initial point, the adopter can start considering the possibility of contributing to a FOS-ERP project, becoming a prosumer - a mixture of consumer and producer of the solution. The final decision on becoming or not a prosumer will be possible only during the following phases, when the adopter better understands the solution requirements and the solution alternatives.

Small and Micro enterprises hardly will become prosumers because of the lack of IT personal necessary to develop - I will talk about this latter.

Free/Open Source ERP (FOS-ERP) - Part I

In 2007 I wrote a book chapter on the differences between FOS-ERP and proprietary ERP (P-ERP), published as part of the Handbook on Research in Enterprise Systems in 2008. I will start now a series of posts that will revisit this paper.

This first part will quickly introduce the Generalized Enterprise Reference Architecture and Methodology (GERAM) as a framework for comparing FOS and P-ERP.

GERAM defines seven life-cycle phases for any enterprise entity and can be used as a template life cycle to analyze FOS-ERP selection, deployment, and evolution. These phases, represented on Figure 1, can be summarized as follows:
a)Identification: identifies the particular enterprise entity in terms of its domain and environment.
b)Concept: conceptualizes an entity’s mission, vision, values, strategies, and objectives.
c)Requirements: comprise a set of human, process, and technology oriented aspects and activities needed to describe the operational requirements of the enterprise.
d)Design: models the enterprise entity and helps to understand the system functionalities.
e)Implementation: the design is transformed into real components. After tested and approved the system is released into operation.
f)Operation: is the actual use of the system, and includes user feedback that can drive to a new entity life cycle.
g)Decommission: represents the disposal of parts of the whole entity, after its successful use.
Figure 1. GERAM Life Cycle Phases

Except for decommission and identification, which are not (directly) influenced by licensing models, these phases can be used to better understand how FOS-ERP differs from P-ERP, as next posts will explain.

Saturday, October 9, 2010

Proof of Concept for the BLDD Tool - Part V

Last week I bumped into what I think is one of the firsts academic papers on BDD to be published in a scientific journal. We have published three technical reports in, but they weren't peer reviewed: Filling the Gap between Business Process Modeling and Behavior Driven Development, Mapping Business Process Modeling constructs to Behavior Driven Development Ubiquitous Language and  A Tool Stack for BDD in Python.

This paper is on applying Model Driven Development (MDD) techniques to BDD, following the principles presented in the Agile for MDD white paper. The authors use Foundational UML (fUML), which defines a basic virtual machine for UML, to create bUML, a tool to support BDD activities that automatically updates the project status.

Our approach holds some differences in relation to theirs:
1) Although the examples in the paper are too "recursive" (given-when-then is used to describe the creation of stories and scenarios) to check details of their proposal, it is obvious that they follow MDD, in fact, at some point they say "The generated code intends to be complete, with no placeholders for the developer to fill out." We do the opposite: we leave placeholders to be filled out.
However, I still need to see some more practical example to affirm this.
2) Our proposed tool is meant to run step by step, I saw no reference to this kind of feature in bUML's description.
3) They use OCL as the basis for textual modeling, we propose to use a more natural should language. We envision the use of (some) should-dsl to insert and describe conditions (logical, acceptance) for the tests. OCL is a standard, however, should-dsl is closer to the user's language.

I need some more time to check bUML - I couldn't find the software or real world examples.  What I can say now is that they follow MDD, we follow Agile, so our proposals differ in the way we intend to implement and use the tools.

Friday, October 1, 2010

BLDD, BPEL, and BPMN - a first opinion

My good colleague Charles Moller asked me about in what BLDD would differ from BPEL and BPMN.

I will start by the easier part: BPMN. According to Wikipedia (from where I copied all phrases between quotes), it is "a standard for business process modeling, and provides a graphical notation for specifying business processes in a Business Process Diagram (BPD)." Since BLDD's proposal is to use any BPM notation to write specifications (possibly complemented by a textual language), BPMN is an option to BLDD.

Going further, "The primary goal of BPMN is to provide a standard notation that is readily understandable by all business stakeholders. These business stakeholders include the business analysts who create and refine the processes, the technical developers responsible for implementing the processes, and the business managers who monitor and manage the processes. Consequently, BPMN is intended to serve as common language to bridge the communication gap that frequently occurs between business process design and implementation."
In other words, as expected, BPMN can be used as an Ubiquitous Language (or a Shared Language, as I dare to call this concept).

Moreover, BPMN also provides a mapping between the graphics of the notation to the underlying constructs of execution languages, particularly Business Process Execution Language (BPEL). BPEL is an orchestration language, in other words, it is used to specify "an executable process that involves message exchanges with other systems, such that the message exchange sequences are controlled by the orchestration designer."  Now, it is interesting to read about the relation between BPMN and BPEL:

Relationship of BPEL to BPMN
There is no standard graphical notation for BPEL, as the OASIS technical committee decided this was out of scope. Some vendors have invented their own notations.
Others have proposed to use a substantially different business process modeling language, namely Business Process Modeling Notation (BPMN), as a graphical front-end to capture BPEL process descriptions.
However, the development of [tools that map from BPMN to BPEL] has exposed fundamental differences between BPMN and BPEL, which make it very difficult, and in some cases impossible, to generate human-readable BPEL code from BPMN models. Even more difficult is the problem of BPMN-to-BPEL round-trip engineering: generating BPEL code from BPMN diagrams and maintaining the original BPMN model and the generated BPEL code synchronized, in the sense that any modification to one is propagated to the other.
This final paragraph gave me some hope that BLDD is not an in vain proposal!

BLDD is based and inspired in BDD, which in turn is influenced by Domain Driven Design (DDD). In other words, for all reasons exposed by these techniques, having all the artifacts (models, requirements, source code, tests, documentation...) readable by humans is a must have feature. Also, round-trip is totally necessary. Besides following these principles, BLDD is simpler and independent of any execution engine - although we are investigating it in environments with workflow engines. I am not saying that BPEL isn't good, I am sure that some really complex systems can take a lot of advantage of it, however, we want to have a more generic proposal, which can be implemented in any language.

I know of BPEL Script, however, it seems to be a language to describe processes, it think is not a full fledged programming language. An idea to investigate is the use of automated tests with BPELScripts and make it another proposal for BLDD (something like BPELScripts + TDD or even BDD). However, what we propose now with BLDD is to use generic languages, such as Python, Ruby, and Java, to implement all the stuff, and more, in a way readable by humans.

So, in other words, I think that BLDD and BPEL has the same goal of implementing systems based on business processes. However:
a) BPEL (i) needs a heavier machinery, (ii) already provides a series of artifacts for dealing with highly complex systems, and (iii) is well supported by industry.
b) BLDD (i) is simpler in general, (ii) is heavily based in TDD, with all its advantages, and (iii) it works with human-readable artifacts from end to end.

Which one to use in your case? I think it depends on your demands, technical knowledge, and development culture. I don't have a final answer, mainly because we need to advance on BLDD proposal and also because I prefer to use the Keep it as Simple as Possible (KISP) principle. I think there is room for BPELScript + BLDD, however, more time is needed for analyzing this proposal (any volunteers?).

Anyway, after reading this quite interesting two-year old article (some of the problems expressed in it maybe are solved right now), I have a final remark: sometimes "enterprise class solutions" become so complex that programmers abandon them in the end, such as CORBA (and J2EE at some extent).