Dynamic Workflow with ]project-open[

This article presents the ]project-open[ "Petri-Net Dynamic Workflow" and explains the role that this component will play in future ]po[ releases. Target audiences are ]project-open[ consultants and partners, and enterprise software architects in general.

Why Use a Workflow?

A dynamic workflow represents a "variation point" in the ]po[ application: a place in the application where a ]po[ consultant can adapt the application behavior to a customer's needs. In particular, workflows represent variation points in the sequencing of actions and processes, as opposed to functional variations (Package Manager), data model variations (DynFields), parameters, security configurations, and functional variation points (user exits).

What is a ]po[ Workflow?

A ]po[ workflow is a graphically configurable combination of a "wizard" and a status engine. It represents the life cycle of an object by a number of statuses. The "transitions" between the statuses allow users to enter data or take actions.

With these features, the ]po[ workflow is basically comparable with the SAP or Oracle workflows, which says a lot. Here are several real-world examples:

• Expense Approval Workflow
• Vacation Approval Workflow
• Time Sheet Approval Workflow

The ]po[ Workflow Engine

The ]po[ workflow engine is actually part of the OpenACS platform underlying ]po[. It was written by Lars Pind in 2000 and has been fixed and extended since them, making it a proven and stable platform today. Quest uses this engine as the main component of its AIMS grant management system, distributing literally billions of pounds per year.

However, it is a rather complex technical structure, and the graphical workflow editor is currently not exactly user-friendly, to say the least.

The ]po[ WF Roadmap

Though the engine is not perfect, we believe it is a powerful foundation for the upcoming ]po[ projects in large companies and more complex organizational structures. We also hope that one day we'll find a customer willing to pay for an improved Workflow editor, which would allow us to cover up the most important disadvantages.

]po[ V3.2 and V3.3:

In V3.2 we used the engine for the first time in order to implement an interface to the Ophelia translation memory system. We've developed several display components for the workflow, including screens that allow users to assign team members to workflow transitions, for example. This code is still part of V3.2 and V3.3, but is kind of hidden. You can check the "Petri-Net" project in our V3.3 demo server (just search for "petri" using the ]po[ search engine).

In addition, during the IT management project at Basler Kantonalbank, we developed a ticket-management system based on the workflow and implemented several reusable components for workflow debugging and visualization.

]po[ V3.4:

V3.4 development basically started with the project at Cambridge Technology Partners (CTP, a success story under development). For this customer we developed the three existing workflows listed above in order to manage approval processes with financial implications. You can check out these workflows in our V3.4 demo server, for example when creating a new "Absence."

V3.4 ITSM and other future development:

For the future, we plan to use workflows everywhere. Here are some examples:

• Project budget approval (status: planning)
  
• Project customer self-service approval (status: planning)
• ITSM Ticket management (status: prototype)
  

The Downside

There are currently several issues with the WF that increase the learning time for new developers. We plan to resolve these issues, but it may take us several months or quarters to do so:

• Lack of Documentation:
  Lars Pind left us with some documentation, but it's nowhere near enough. We have started to write up some guides, but they're far from complete.
  
• Ugly WF Design GUI:
  The GUI looks good at the first glance, but you will soon find out that menus are located in inconsistent places and that the GUI doesn't really support any assignment logic, or anything of the sort.
  
• Written in PL/SQL:
  The main part of the workflow is written in PL/SQL. This means yet another language that you need to understand, and the communication from the PL/SQL to the TCL layer is clumsy.
  
• Difficult to Debug:
  Debugging in a workflow environment basically means postmortem analysis. There is already a workflow log to protocol all workflow "decisions," but this is still clumsy.
  

Summary

The ]po[ workflow is a proven, stable and extremely powerful component. We will use it for nearly all new ]po[ development in the future, and gradually improve its weak points. As it stands, most workflow development needs to be performed by ]po[ core members or certain OpenACS developers with experience in the workflow.

References

• Lars Pind's original documentation
• OpenACS Workflow Code
• PO-Workflow-Implementation-Manual.061115.DraftDraft.pdf
  
• PO-Workflow-Overview.080625.DraftDraft.ppt
  

The ERP as a Social Network

"ERP systems are all about some people putting information into a database and other people trusting this information." This is how a member of the "erp-select" mailing list recently defined ERP (Enterprise Resource Planning) systems. In this article we want to explore the inherently collaborative aspects of ERP systems and how to enhance them by using Web 2.0 techniques.

The ERP as a Web 2.0 Platform

According to this definition, an ERP is a place where people meet, collaborate and share information. Obviously, the type of information shared is different from MySpace, Facebook, LinkedIn, and all the other Web 2.0 social platforms. But there are still some important similarities — particularly if the ERP is based on Web technology.

So the questions for us (the developers of the ]project-open[ project-ERP system) is how we can leverage these new ideas in order to improve our customers' business processes.

The Essence of Web 2.0

So what are the lessons we can actually learn from Web 2.0?

Here are my personal favorites:

• User-generated content:
  Provide the users with incentives and they will produce the content themselves. This apparently works better then centrally controlled content generation.

• Using Wiki to reduce "write permissions":
  The main idea behind a Wiki is that basically everyone can create, modify, and extend existing content. Instead of creating permission hurdles to protect content, a Wiki focuses on social self-correction. In economic terms: The cost of not creating content is higher then the cost of eventual removal of contents. Even if certain users' content revisions cause heavy damage, a superuser can usually recover past versions of content, keeping the benefits higher than the costs.
  

• Tagging and folksonomies as lightweight semantic markup:
  The omnipresent tag clouds have managed to replace semantically precise domain models in many applications. Tagging is imprecise, and often nobody can exactly say what a tag means — but it works! No more questions asked.
  

• Interactive (AJAX) GUIs:
  The user-friendly GUI of Web 2.0 applications is an important condition to encourage users to contribute.
  

However, there is one important difference: Web 2.0 applications tend to host a homogeneous group of users, while ERP systems have to deal with widely varying user privileges. For this reason we have to add an important point:

• Powerful but easy-to-understand permissions:
  ERP systems need to provide graded access privileges to collaboration facilities based on the user's hierarchical position and membership in departments, offices or projects.
  

How to Implement the ERP V2.0

Once an ERP system provides a Web GUI, it isn't that difficult to implement some of the Web 2.0 techniques. The following few components make up the Web 2.0 features of ]project-open[:

e-Rooms:

• Everything is an e-Room:
  Projects, customers, users, and other important objects are implemented each as a kind of e-Room (virtual collaboration space)
  
• Access permissions on e-Rooms:
  The rich information in e-Rooms are protected by permissions
  
• Basic collaboration options per e-Room:
  Each e-Room provides a separate discussion forum, space for file storage, Wiki, etc.

Flexible surveys :

• User-definable surveys allow users to express their opinions about ERP objects such as providers, projects, etc.
  

The techniques presented above can flexibly be used in a variety of ERP modules:

Provider Management

By setting up an e-Room for each provider (with forum, file storage, and a survey to rate the provider) you can:

• Keep track with each provider's performance over time
• Engage in a mutual learning process
• Share information with the provider if appropriate (when necessary, negative information can be used to get better deals or spur provider performance increases)
  

Customer Management

By setting up an e-Room for each provider (with forum, file storage, and a survey to rate the provider) you can implement:

• CRM collaboration, by keeping all quotes and contracts used during the customer acquisition phase
• Customer satisfaction tracking, by using the survey facility to acquire customer satisfaction surveys.
  

HR Processes

• 360° evaluations
• Project "postmortem" reports
• Project progress reports
• Satisfaction survey for customers and project members

Knowledge Management Reloaded

The Web 2.0 techniques may actually revive Knowledge Management, because they resolve one of the most difficult aspects:

• KM does not require a separate application with different logins and permissions sets. Instead, KM is integrated directly into the ERP.
  
• The ERP provides a rich context for knowledge-carrying documents because documents are related to people and projects.
• The creation and use of knowledge resources can be directly tied to bonus payments.
• Tagging and other markup of customers and projects may be used to tag the respective documents.
• Clumsy domain models and ontologies are now replaced by "tag clouds" and folksonomies.
  

Summary

Web 2.0 and the ERP seem to be an excellent fit when an ERP is based on a Web GUI or a platform that allows for similar user interaction. ]project-open[ provides a sample implementation capable of demonstrating most of the techniques explained above.

Fun with Financial Controlling

"Any inconsistency in a large system will keep biting you until you fix it.
So don't start off with inconsistent stuff!"

That's an old developer's truth. However, there is sometimes a fine line between real inconsistencies and a complex reality.

Financial Project Controlling Basics

Today's discussion comes from the area of financial controlling, also called management accounting, analytical finance, or simply controlling. To me, controlling is the interesting part of financial management, with its goal of supporting management decisions; discussions of accounting, by contrast, tend to deal more with compliance issues and the sometimes Kafkaesque tax rules.

In particular, project controlling deals with calculating reasonable profit & loss (P&L) figures for each project in a company. This is very important, because it allows a company's managers to identify unprofitable customers and sometimes to detect budget overruns before these problem projects have had the chance to crash.

]project-open[ vs. Other ERP Applications

Most ERP applications are build on an "accounting" financial base, simply because these systems have been written for companies who are producing, buying and selling physical goods. Tracking the value of physical goods through a value chain is the domain of accounting.

However, ]project-open[ has been built exclusively for service companies. For service companies, accounting is nothing but a hassle twice or four times a year. Instead of looking at accounting measures such as circulating working capital, warehouse stock levels, and the like, service companies worry about things like keeping their knowledge workers in the company, perceived customer service levels, and detecting project cost overruns early. And yet these figures don't even appear in accounting balance sheets or income statements.

Controlling with SAP

Large ERP systems such as SAP, Oracle, and Dynamics-NAV also provide controlling functionality to companies' financial managers, but this controlling is built on top of the accounting base. In SAP, for example, controlling is based on reclassifying invoice items based on a number of business rules in order to come up with contribution margins per product class. This is why controlling is an optional module for SAP and the other ERP systems: The core is always accounting.

Controlling with ]project-open[

In ]project-open[ we have designed our financial system the other way around: Invoices and other financial documents produced in ]po[ are first created in and associated with a project. This way, we achieve two major advantages:

• Less information = Better usability:
  We don't need to ask the user to provide the system with the detailed and difficult to understand accounting information
  
• Real-time project controlling:
  By associating financial information directly with a project, all stakeholders get real-time financial information.
  

Export to Accounting

The missing accounting information — account numbers, tax codes, and such — are added in a second step when exporting the financial documents to a separate accounting system twice or four times a year. This step is usually supervised by an accountant who knows his job, so that none of the business managers have to spend their time crunching numbers.

Reassigning Controlling Documents

Up to this point, everything seems fine. But there is one area that causes trouble in ]project-open[: Financial documents that are related to more than one project.

This issue appears in particular in projects with many large sub-projects, where it's necessary to bill more then one sub-project in a single invoice, or when a provider writes a cumulative bill for services delivered for more than one project.

A simple solution could be to split a financial document in two and assign these parts to their respective projects. But by doing this, we would lose the 1:1 matching between ]po[ invoices (and invoice numbers) and accounting invoices.

The correct solution for this issue is to mark a re-distributed invoice as "redistributed" and to create separate financial document "pieces." However, ]po[ V3.3 and V3.4 don't yet support this relatively complex procedure.

The Need for Human Decisions

Instead, we have decided to require a "human decision" in these cases now. So we currently present accountants with an error message if an invoice relates to more then one project.

Inconsistency or Complex Reality?

To come back to the beginning of this article: Is this need for a "human decision" a fix for an inconsistency in the system, or is it part of a complex reality where only a responsible human has the necessary knowledge to make a decision?

We believe that the second option is correct, and that the ]po[ financial system suits its domain better then any of its competitors.

Please let us know your opinion!

Project vs. Product (1): The Underlying Conflict

One of the most persistent discussions in the ]project-open[ community is "Project" vs. "Product".

What is a Project?

There are many, many definitions of what projects are and what not. For us, the ]project-open[ guys there is an easy definition: "Get stuff done. Be fast! But make the customer happy!"

What is a Product?

A "product" in contrast (our ]project-open[ application) is something completely different. This has even a kind of esoteric dimension: "Do things right. Right from the beginning!". It is obviously very questionable waht "right" means. But it is clear that the "product" should not include quick & dirty hacks and similar constructs.

Instead, the "product" is something that we will push - earlier or later - to our ~2.500 customers around the world by means of a product update. So "product" is about responsibility, security, extensibility and many more.

Another Definition of "Product"

In a nice conversation with Dennis Riedel of Opus5 we came up with a nice alternative definition of "product": It's the least common code denominator of > 3 customers. Dennis' background is custom web site development for a number of high-profile customers. Their (Opus5) problem is that every customer has very special ideas on how their application should look like. So coding at Opus5 includes a code reuse strategy of "copy-paste-modify" then "productify".

Customer Variability

So here is a core characteristics of a "product": A single code base has to serve a number of customers. Specific customization should be reduced to a minimum. Instead, a "product" is supposed to include a number of parameters so that you adapt the product behavior to a range of customers.

But what happens if a customer doesn't fit into this corset?

Difficult question: Some companies would take on high-profile customers and mess up their product; these companies tend to be successfully in the short term. Other companies would reject such a customer and go for "scalability". These companies tend to be more successful on the long term. But there are always exceptions...

Turning down Customers???

So does that really mean turning down customers? Any sales guy may consider this to be heretic thinking. But yes, in order to "keep the product clean" you need to turn down customers who would pollute the "product" (remember? The beautiful piece of code that is the base of your business...) too much.

Conclusion? You will get trouble if you don't listen to your sales guys, but you will also get into trouble if you don't listen to them at all.

Conflict Lines

A "conflict line" in the context of "Project vs. Product" is the separation between:

• the "product guys", the "guardians of the pure code" and
  
• the "project guys", the people who create happy customers

From the text above we can already assume that none of the two can exist without the other one. However, each customer's project will come with a struggle between the two sides.

The Beauty of Conflict Lines

But here comes the beauty of a conflict line: If both projecties and producties take their job serious, they will - with some moderation by senior management - come up with a nice compromise, satisfy the customer and during that process enhance the product.

How?

Stay tuned, subscribe to this channel and ask for more! There will be more coming on conflict lines and project vs. product.

Frank

The "return_url" Web-Pattern

This "pattern" or programming style addresses a frequent issue in Web-applications.

Page Flows

One or more Web pages form a "page flow", i.e. a logical unit of pages that belongs together. The most simple flow is the combination of an "EditPage" (a page showing information about an object in a form) and a "SavePage" (a page saving the modified data to a database). More complex flows include "wizards", workflows etc.

The Anchor Page

Most of these flows start off from a certain page, which we want to call anchor page. The intuitive behavior of the application is return to this initial page once the flow has successfully terminated. However, a flow may be called from more then one anchor page. So the last page of a flow usually contains a quite ugly logic to calculate where to return.

The idea of the "return_url" pattern is to avoid this ugly logic and to replace it by a single parameter provided by the anchor page (or more general: page that initiates the flow).

Passing "return_url" Along

Nearly all pages in OpenACS and ]project-open[ contain this (in-) famous "return_url" in one or more places. There are two main places:

1. In the Link at the Anchor Page
   It is the responsibility of the anchor page to "call" a flow with the URL of the anchor pages as return_url.
   This is a kind of "inversion of control": Instead of the flow, it is the anchor page that tells the flow to return to the page.
   
   
2. In the "Page Headers" of the "flow" pages
   The code "{ return_url "" } is found in the "Page Header" of most ]project-open[ page. The "Page Header" is like the declaration of parameters of a procedure, only that it takes its variables via URL or HTTP POST commands. It is particularly nice that you can specify a default value for the return_url if it shouldn't have been specified explicitly by the user/application
   
   
3. In the forms & links of the flow pages
   Each flow page needs to pass the return_url to the next page. OpenACS/]po[ has several commands such as "export_form_vars" that automate this task, for both forms and links.
   
   
4. At the last flow page
   The last page of a flow executes a "ad_returnredirect $return_url" command as the very last action. This commands will redirect the browser back to the anchor page
   

Code Samples

For code samples please see any ]project-open[ package and search for "return_url". You can download the source code of ]po[ from SourceForge at the download section of http://sourceforge.net/projects/project-open/.

Summary

This "return_url" pattern is definitely no rocket science. However, this simple "inversion of control" frees the flow pages from ugly, obscure and difficult to maintain logic and allows page flows to be reusable for different anchor pages.

Or to put it in another way: Any rooky still coding "page flow controllers" should know that there are more elegant solutions out there...

How to Launch an Open-Source PR-Campaign

How to Launch an Open-Source PR-Campaign?

In the last week I have written about how to launch a new open-source project, based on an example project called TinyTM. In the last three days I've been working on getting the word out about that project. Here are some reflections.

Content

Obviously you won't be able to launch a very successful PR campaign if you don't have something newsworthy to get the media's attention. So first and foremost, make sure you're promoting a deliverable with some kind of impact on your target audience (see below).

Timing: Content vs. Lost Time

In the case of TinyTM we had three options:

1. Simply announce that we'd created the project,
2. wait until the first prototype for developers was available, or
3. wait until we'd finished the first first version that could be used by end users.

Basically, the longer you wait, the more content you can present. And content is what makes the press release "newsworthy."

On the other hand, the longer you wait, the higher the probability that you'll never publish anything. Also, possible collaborators will take more time to find your project.

Expectation Management

So it's tricky to find the optimum timing, and timing varies with the nature of your project. Traditional companies also need to consider the bad impression that an incomplete product might leave with customers -- no "banana software" that ripens with the customer. Open-source projects, by contrast, are known to release early and release often, so in our case it's OK to be on the earlier side. That said, it's critical to clearly state the maturity of your project, lest your readers feel deceived and lose interest in the next release.

At the end, we decided to release an early developer version plus a demo server. So even though user management and other functionality isn't yet available, non-technical end users can preview the GUI and get a better idea of the software's capabilities.

The Press Release Text

Writing a press release is a horror for non-native English speakers. In particular, we open-source guys are usually techies, and not so much into the humanities. So what can you do?

The bottom line is: Copy, paste and modify. Press releases are very similar to each other for some reasons. Don't try to understand or change that; just accept that it's how the world works. Then search for press releases about similar products and mix and match.

Further Information

It is essential to have a web site up with with detailed information about your project before launching the press release. If somebody is interested in your project, and he or she only finds the techy stuff on SourceForge, you won't exactly end up promoting your message. So make sure you have at least 2-5 pages to introduce non-hard-core-OSS people to your project, and include these pages in your press release.

Determine Targets

Let's assume you've got everything: The content, a web site with background information, and a a press release. Now: Where do you post your press release so that people actually read it?

There are several open-source publications that are obvious targets for your press release, such as SourceForge (project news might get on the first page and will be published as RSS) and Freshmeat.

Another channel to consider is related mailing lists and discussion forums. You may actually have to subscribe to these channels before you can post; that may fill up your inbox for some time, but it also gives you the opportunity to get feedback and to answer that feedback.

Similarly, you might have a list of "multipliers" who you could ask to spread the word for you. Don't think too poorly about your project and your press release (if you do have interesting content, that is). Multipliers maintain their reputation by providing others with information, so your press release might be a way to enhance their standing.

Extending the scope a bit more toward the mainstream is www.openpr.com and some similar PR distribution lists. We haven't seen many hits coming from these postings, but Google tends to index them and they might provide your web site with an increased Google ranking.

Finally, there are the big magazines and newspapers. Check their web sites -- most of them have a special email address for press releases. One hour of surfing your preferred magazines will provide you with 10 to 20 email addresses. Only 10% of them might publish your press release, but that could get you more visitors then all of the measures above combined. And you never know what's going on in the heads of the editors ...

Maximizing Impact

After publishing the press releases, try to gauge your success. How many links did you get? How many additional visitors? Then check where they came from and try to evaluate which of your PR measures were the most successful. You might also try asking people why they didn't publish your press release and what you could do to improve.

Beware of Negative PR

Just a word of warning about media people: They may not always publish what you would like them to publish. That's no problem in most cases -- as the saying goes, bad PR is better then no PR. However, keep in mind that negative news sells better in the media business than positive news, and that the friendly reporter may actually only write about the stuff that's missing from your release.

Cheers!

Frank

How to Start an Open-Source Project? And Why?

Why?

The last 10 days or so I've been working on TinyTM (http://tinytm.sourceforge.net/). We've decided to launch this new open-source project in order to cover an important shortcoming of ]project-open[ in the Translation sector. Most "competing" applications in this sector provide an integration with a "translation memory" -- the most important translation tool. We don't.

What?

Translation memories are a kind of "light" natural language processing tools. They record the sentences that a translator translates and are able to remember them later if a similar sentence appears. That is not as bad as, say, real machine translation, but it's sufficiently tricky to take a long time to work out.

The "Competition"

There is already one OSS translation memory, but it's written in Java; this makes it relatively difficult for linguists, many of whom are not technologically proficient, to install. It also doesn't support Microsoft Word, which is clearly a vital tool for working linguists. In short, it doesn't meet its current users' needs.

The "Plan"

So the idea is to offload most of the work to the OSS community. Will that work? There seems to be a big "market" for it (if that's the appropriate word in the OSS world); many users have requested a better translation tool, so it's certainly possible.

Allies

I've been in contact with FOLT, an industry association trying to set up an open-source TM system themselves. I was really keen to join their team until I heard that they've been discussing a TM system for about the last three years. Also, they didn't seem to grasp the open-source "publish early and frequently" approach. So that one didn't work out ...

I've already heard about other companies asking Common Sense Advisory about a TM system. They apparently even asked CSA to take the leadership. But I wasn't able to find out who that was. Maybe it's just easier to publish some code in oder to get their attention? Maybe that's the open-source answer to business development: Just publish your code.

Another group of potential allies are small TM vendors. These small companies currently have no chance to fight the market leader (SDL Trados), which has an 80%/90% market share. But if TinyTM gets successful, these small vendors might be able to "ride the open-source tiger." Since we OSS guys are usually not very good at creating user interfaces for non-techies, these guys might be able to sell their polished front ends as a kind of "enterprise version" for TinyTM if they can connect to the TinyTM back end. That's not 100% the open-source idea, but a nice commercial front end might convince the "early majority" to adopt TinyTM once it starts taking off. For this reason I've used the LGPL for the TinyTM protocol and the interface code.

The "Market"

So what are the "success criteria" for TinyTM to fly? There are some 900,000 people in the world spending their days translating texts, according to a post from Common Sense Advisory. But most of them are linguists, and linguists tend to be IT-averse, meaning they don't make very productive open-source community members.

Success Criteria

So we'd need to get the 1%-2% of linguists who are "open" to open-source and try to provide them with a fun environment where they try and extend the TM:

• "Recruit" enough skilled developers to extend/finish TinyTM
• Spread the word to reach as many of them as possible
• Try to lower the barrier to involvement.
  
• Think of ways to make involvement a fun experience
  
  
• Get some donations to push the project
  
  
• Reach to innovators and early adopters in translation companies who could start using TinyTM while it's still in its early phases.
  
  

Some Online Community Theory

Philip Greenspun, an impressive entrepreneur, once said: "Communities need killer content to attract users." So is a TM killer content? I believe that is actually true for a large number of people, because TMs still cost several hundred dollars per license.

Code!

So I've pulled my basic knowledge about natural language processing and written a first version of the software, trying to simplify the architecture as much as possible while keeping everything modular. Web services sounded nice, but I actually took the decision to start with a plain old relational database and Pl/SQL as the language to implement remote procedure calls.

This approach is very simple to start, it doesn't require all the web-application XML stuff, and it's easy to wrap the Pl/SQL calls in XML later. So: Quick wins without sacrificing anything about the future -- that seems to fit.

On the client side I decided to go with something very unconventional for an open-source project: VBA. Yes, Visual Basic for Applications. The advantage: MS Word, the #1 translation environment (reach the 900k users!), contains an ODBC driver, and already comes with the "editor."

Spread the Word

So I set up http://tinytm.sourceforge.net/ with the purpose:

• Attract the maximum number of developers
  
• Attract some companies as sponsors
• Convince the "suits" that the project is about something business
  

For Thursday (17th of April, 2008) I've prepared a press release and a list of magazines and other potential multipliers. I've send the press release to a few contacts for feedback, and it seems more or less to work out.

So let's see if it works out.
Please let me know!

Frank