Incubation Thing

This is the home page for all things related to what we are currently calling the "incubation thing".  It is the formalized process to be used by Pentaho and its community to convert ideas into product. Until a good motif is defined and names and terminology has been agreed upon, I will use "incubation thing" to represent an instance of the process. Check out Proposed Names for Incubation Area for a discussion on possible motifs.

Summary

The purpose for the "incubation thing" is to have a well known and repeatable process for creating software in the Pentaho open source environment. We want to take advantage of the transparency, openness and early and often features of the open source development model and add some process check points. The process should allow us to effectively communicate where the product is in its infancy, how much interest there is, how fast is it moving and what tasks are required to keep it moving forward.

There is really nothing new here. These 5 steps are always used but are usually locked in someone's head. The review at the end of every phase is important to keep the project focused and the people working on it synchronized. During each phase, standard open source development happens.

Requirements

• Provide a mechanism for transparent collaboration between geographically dispersed participants throughout all phases
• Allow each "incubation thing" to specify entry and exit criteria for each phase in a way that makes sense for that "incubation thing".
• The phases of the "incubation thing" must be easily understandable with specific entry and exit criteria
• A "incubation thing" must be well supported in order to remain active
• The status, activity, traction etc. of the each "incubation thing" must be readily available and easy to understand
• An "incubation thing" can be private or public. Pentaho uses the "incubation thing" process internally for management services and strategic partnership opportunities.
• The visibility can change during the "incubation thing". For example - Conception and Seeding can be restricted, then "opened" to the community for the rest of the "incubation thing"
• "incubation thing" can be terminated in any phase due to lack of interest or progress
• Allow for different conclusions for difference processes.  Here are some of the common results:
  • New Pentaho open source product
  • Added functionality for an existing product
  • New independent open source project
  • Becomes part of a non Pentaho open source project
  • A third party takes over the project
  • The idea gets trashed
  • ...

The "incubation thing"

The following diagram illustrated the steps in a typical development "incubation thing". The bullet points under each phase is only to illustrate the things that could happen during that phase in a typical process. They are not meant to be the rules that absolutely define each phase. Each "incubation thing" that follows this process will define the entry/exit criteria for each step. This should make the process work for the "incubation thing" and not try to shoehorn every "incubation thing" into the same exact steps.

Initiating a new "incubation thing"

It is not Pentaho's goal to have lots and lots of "incubation thing"s in process just for the sake of hosting projects.  We do not want to be a mini-SourceForge. Each "incubation thing" that gets started requires Pentaho resources and will have to show a compelling value proposition for Pentaho and the community. To ensure that an "incubation thing" is successful, a high level of commitment through sponsorship is required. 

New "incubation thing"s can come from many different places.  The common scenarios are:

• Pentaho Product Management (PM) initiates and sponsors the "incubation thing" of a feature or product that is on the Pentaho road map.
• A third party company has a product that fits into the Pentaho suite and wants to contribute it to Pentaho.
• A community member as an idea for a feature, discusses it in the "incubation thing" forum (to be created) and generated enough interest and support to request an "incubation thing" be created.
• The CTO office wants to sponsor a prototype or proof of concept.

The idea can start with anyone and be cultivated by whatever means makes sense.  The the "incubation thing" forum is just one tool the community can use to initiate discussions, gauge interest and find sponsorship.  An "incubation thing" sponsor is responsible for moderating discussions, reporting status and moving the "incubation thing" forward. Anyone from the community, Pentaho employees, customers or partners, can sponsor an "incubation thing" as long as they agree to be committed.  In order to assure that there is enough interest,  sponsor will need to get support from two other people.  The three of them will become the working group.  There can be more people in the working group but three is the minimum.

After a sponsor has been identified, the task has been defined and there is commitment from at least 2 other people, the sponsor can submit a proposal to initiate an "incubation thing".

Submitting the proposal

This sounds formal but is really a matter giving the following information to Community Connection (communityconnection@pentaho.org): (This could be a simple form)

• Identity of the Project Sponsor. Their responsibilities include:
  • Communicate status to Community
  • Coordinate meetings with working group
  • Request resources from Community as needed to support initiative

• Identify two other people to be part of the "incubation thing" working group
  • They should be stake holders in the outcome of the project
  • Agree to be involved in at least conception phase

• A mission-statement like description of the initiative. Examples:
  • Create a thin client, HTML based, slicer/dicer for Mondrian that will replace JPivot.
  • Create a WYSIWYG tool for defining Pentaho charts

• A short name (code name maybe) for the initiative to be used as an easy way to talk about the project.  Examples:
  • Dashboard Designer
  • Report Management Server (RMS)
• State what the visibility of the project is - completely open or restricted.

Accepting the proposal

If the proposal is accepted, (at this time there is no formalized process just general consensus) Pentaho Community will create a new "incubation thing" and coordinate setting up resources required. The "incubation thing" will start in the conception phase and have a collaboration space consisting of:

• Entry in the "Conception Phase" listing in the "incubation thing" space on pentaho.org
• A forum within the "incubation thing" forum space
• Wiki page for documentation, requirements, meeting notes etc
• Subversion project for source code when needed

What about these things?

• A working group email group for communication
• Some kind of webex access for meetings
• FTP Space for downloads

During the acceptance process, Pentaho Project Management (PM) will decide if the "incubation thing" is destined to become supported Pentaho product.  The "incubation thing" will be identified as "Supported Product" or "Unsupported" and be specified on the "incubation thing" page.  It should be obvious to everyone what the end game is.  Pentaho engineering will also specify their level of support of the "incubation thing".

General statements about the phases

Each of the 5 phases has entry requirements that must be met before it can start and specific goals that must be achieved before it can complete. For all phases except conception, the exit criteria for that phase must be defined before the step can start. The purpose for this restriction is to avoid confusion, focus the effort and provide a concrete way to answer the question "are we done yet?" In the event that a process gets "stalled", the exit criteria provide a quick and easy way for someone to see what needs to be done in order to move the process along. This is the most important feature of the "incubation thing". It allows the work during the phases to happen in a free and open way but provides well known and well defined checkpoints during the process.

The tasks that happen during a specific step in the "incubation thing" may be different for each instance and are be determined by the working group. This is in recognition that every instance is unique and we are not trying to shoe horn every "incubation thing" into the same process. That being said, when defining the exit criteria, it is important to match the goals of the phase with the intent of the phase. For example, no one should be coding features during the conception phase.

A phase is not complete until there is a satisfactory review. During the phase review, the working group must agree that the exit requirements for that phase have been satisfied and that the entry requirements for the next phase have been met.

During the review process, the working group may determine that the project is not viable or that there is little interest in continuing. In this case the project can be canceled or put on hold.

The working group leader will report the status of the review to Pentaho Community.

Conception

The conception phase about the communication of ideas: brainstorm requirements, discuss use cases, look for existing software, generate interest, etc. By the end of the conception phase, there should be an initial set of high level requirements with enough detail for someone to generate a prototype or start on seed code.

If someone already has a similar product that they want to contribute to open source, this would be the phase where it is evaluated for feasibility.

Most Active Community Participation

Anyone with an interest in defining the capabilities of the feature or product. Usually related to the end user, product management or system integrator.

• Product Management
• CTO
• End Users
• Integrators

Entry Requirements

No requirements beyond the Requirements for initiating a new "incubation thing".

Typical Tasks

• Define the process - Define the unique attributes for this specific "incubation thing" process. What is the end game? Will we need new seed code or does it already exist?
• Identify stake holders - Who are the interested parties? Users, contributors, resellers etc. Recruit people for the working group. Build interest in the project.
• High level requirements - By the end of this phase, there should be enough requirements to start prototyping and costing. This usually results in a Product Requirements Document (PRD)
• Storyboards - Create storyboards and use-cases
• Architecture design - The high level framework will be needed in order to define the seeding requirements for the next phase.

Seeding

The seeding phase is where the initial coding begins. Seed code in the form of a prototype or proof of concept is typically generated by a small group or single developer. Builds are made available for people to try, comment on and modify. It is usually easier to discuss features and requirements when there is code to try out and comment on. This is where the strength of early and often come in. By the end of seeding, most or all requirements have been defined, tasks have been identified and big chunks of work can be assigned. Seeding usually ends with a Milestone 1 (M1) release.

Most Active Community Participation

During this phase, a small group of developers get involved with prototypes and POCs. Product managers and system integrators will be verifying the prototypes against storyboards to make sure the requirements are being interpreted correctly.

• Product Management
• CTO
• Integrators
• Developers

Entry Requirements

• Defined Seeding Requirements - The things that need to be proven or prototyped are identified
• Define Exit Criteria - What must be accomplished before this phase can be considered complete.

Typical Tasks

• Code the framework - Initial architecture framework provides straw-man for evaluating requirements. It may morph into the implementation.
• Proof of concept (POC) - Usually throw-away code to demonstrate a feature or concept. Determine the feesability of a technology or help visualize a UI.
• Prototype - Similar to a POC, usually more complete and likely that some or all of the code will be used
• Define Exit Criteria - What must be accomplished before this phase can be considered complete.

Incubation

The incubation phase is where the bulk of the coding occurs. Requirements should be complete and JIRA cases should be created and assigned to developers. The incubation phase generally lasts through the Milestone (M2-M4) releases and ends with a feature complete (FC) release.

Most Active Community Participation

Many more developers get involved during this phase. Users can start looking at features as milestone builds are completed.

• Developers
• Users

Entry Requirements

• Completed Requirements or PRD - Product Requirements Document complete and agreed to by working group.
• Completed seed code - Prototypes or POCs should be complete. Framework code, if required, should be in a state where other developers can contribute
• Define Exit Criteria - What must be accomplished before this phase can be considered complete.

Typical Tasks

• Code major features - Code and document the features and develop JUnit tests
• JIRA Cases created - Requirements should be documented in JIRA so tasks can be assigned to developers and tracked
• Document design - The architecture and design documentation should be completed during this phase, preferably at the beginning
• Milestone Releases - This phase usually accompanies M2 - M4

Hatching

The hatching phase comes after the main features are complete and the project needs to be stabilized, debugged and completed. If it is going to be supported product, the go-to-market tasks can start.

Most Active Community Participation

With the major development complete, the bulk of community activity turns to feature testing and integration testing.

• Testers
• Developers

Entry Requirements

• Design docs complete - After all we are almost feature complete
• Remaining work is documented in JIRA - If it's not in JIRA it's not likely to get done
• Define Exit Criteria - What must be accomplished before this phase can be considered complete.

Typical Tasks

• Complete features
• Stabilize - It should become stable enough for people to start trying it out and deploying
• Bug Fixing - Find and fix major bugs, design defects etc.
• Create Samples - Samples, task oriented documentation and tech tips
• Go To Market Plan - If this is going to be supported product.
• Feature Complete - This phase usually ends with FC
• Migration plan - If applicable

Product

The product phase is where software is turned into whole product. If the project is going to be supported by Pentaho, all the normal GTM stuff happens here. If the project is going to be spun off as its own open source project, those preparations will happen here.

Most Active Community Participation

This is where the end users get heavily involved

• Testers
• Developers

Entry Requirements

• Feature Complete -
• Define Exit Criteria - What must be accomplished before this phase can be considered complete.

Typical Tasks

• ** -
• ** -
• ** -
• ** -
• ** -