KEMBAR78
Open Source at the Apache Software Foundation | PDF
Open Source and the Apache Software
            Foundation
Introduction
 Founding member & two term director of the Apache Software
  Foundation

 IBM’s first official ‘commercial’ open source developer (         ?)

 Committer on Apache HTTP Server & Apache Portable Runtime projects,
  Apache Incubator Project Management Committee member

 Electrical Engineer by training, Software developer by trade, Physicist at
  heart, Sinophile by choice

 Advisor to founding of Linux Technology Center and Eclipse Foundation

 IBM Senior Software Development Manager, Apache Geronimo and
  WebSphere Community Edition and WebSphere Open Source Strategy.
Agenda

 IBM and the Apache Group: Forging a Partnership

 Apache Software Foundation: The Organization

 Development at the ASF: The Apache Way
The Apache Group

1995 - 8 members organize around a website, mailing list and source
   code repository…

Objectives:
 Resume development on the abandoned NCSA httpd server
 Core infrastructure required to build out internet ecosystem
 Permissive licensing (lower barriers to use and participation)

Origin of the name “Apache”
 The name 'Apache' was chosen from respect for the Native American Apache
   Nation, well-known for their superior skills in warfare strategy and their
   inexhaustible endurance.
IBM Partners with Apache Group

1998 - IBM stops development of their proprietary Web Server, chooses
   Apache HTTP Server for their business needs

 Inside IBM, the decision was controversial & difficult
    – Fear of ‘loss of control’
    – Fear of legal liabilities & litigation


 Apache Group’s decision to embrace IBM was equally difficult
    – Apache Group was open to corporate participation but …
    – Didn’t expect a big company like IBM to show up


Could a partnership succeed? If so, it would be groundbreaking.
Forging a Partnership
The Contentious Issues…

IBM - large, impersonal, bureaucratic/process driven
 How to build business plans around ‘chaotic’ & ‘unpredictable’ open source?
 How do we know who really owns this code?
 How much will we pay in lawsuits for using this code?

Apache Group - small, highly individualistic, agile
 Would IBM dominate the community?
 Do corporate developers know how to write good code?


             How to Bridge the Cultural Impedance Mismatch?
Bridging the Impedance Mismatch

Some lessons learned by IBM…
 Open source development is not ‘chaotic’ or ‘unpredictable’ ..
     – Just different from the corporate ‘command and control’ style
     – Very predictable, but requires careful observation and understanding

 Control (of a community) is not necessary … or even desirable ...
     – ‘Influence’, which is earned, is sufficiently effective
     – Diversity of ideas creates fertile ground for innovation

 It’s okay to ‘scratch your own itch’ and look out after your own interests ..
     – Generalized solutions preferable to overly specialized solutions
     – Developing directly in the community preferable to “power-planting”

 Trust is really important - best achieved by being transparent & open
     – About goals and plans
     – Discussing, designing and implementing code
Bridging the Impedance Mismatch

Transformational changes occurred inside IBM…
 Institutionalized developer open source training…
    – How to be good open source community members
    – Legal issues related to open source
        • Licenses, Copyrights, Patents, Code Provenance
    – Use of open source code in IBM products to achieve business objectives


 Fundamental change in how IBM Legal views open source…
    – Transitioned from ‘zero tolerance’ for open source to acknowledging valid
      business reasons to be involved in open source
    – Developed business processes to manage risk
Bridging the Impedance Mismatch

Apache Group Key Insights ..
 Recognized an emerging opportunity to create a corporate friendly open
  source ecosystem

 Demand for corporate friendly OS would require more formal
  organization

 Developed new awareness of legal issues..
    – Source code provenance is really important to ‘deep pocket’ users of OS
      technology
    – More nuanced understanding of patents and copyrights
Observations

 Apache Group’s early interactions with IBM provided catalyst for creation
  of the Apache Software Foundation

 Began to articulate rules for running effective corporate friendly open
  source communities (a.k.a. ‘The Apache Way’)

 Insights gained drove IBM internal transformations regarding how open
  source was viewed

 Experience influenced creation of the Linux Technology Center and the
  Eclipse Foundations
Agenda

 IBM and the Apache Group: Forging a Partnership

 Apache Software Foundation: The Organization

 Development at the ASF: The Apache Way
The Apache Software Foundation

So let’s talk a bit about the Foundation…

 Founding Principles & Mission

 Some Interesting Statistics

 Corporate Organization & Structure

 Contributions, Funding and the Apache Sponsor program
The Apache Software Foundation

       1999 - Apache Software Foundation Incorporated

                              Founding Principles

 The Foundation exists to serve the development community

 “Let developers focus on what they do best: Code. The foundation exists to do
  the rest” Justin Erenkrantz

 Not-for-Profit

 Not affiliated with or controlled by any government or commercial entity
The Apache Software Foundation

                       Mission Statement
The Apache Software Foundation provides support for the
  Apache community of open-source software projects. The
  Apache projects are characterized by a collaborative,
  consensus based development process, an open and
  pragmatic software license, and a desire to create high quality
  software that leads the way in its field.
The Apache Software Foundation

                          Some Statistics
 21 founding members (1999)

 291 members (June 2010 - CAGR > 26%)

 ~2300 committers (June 2010 - CAGR > 50%)

 75 Top Level Projects (TLP) (June 2010 - CAGR > 48%)

 Over 30 projects in Incubation (transition state to TLP)
The Apache Software Foundation
                                             Organization
Foundation Governance                                           Board of Directors - Greg Stein
• 9 member Board of Directors                                     • Shane Curcuru    • Justin Erenkrantz
• Executive officers (Appointed by the Board)                     • Doug Cutting     • Geir Magnusson Jr
                                                                  • Roy Fielding     • Brian McCallister
Chairman, President, Treasurer, Secretary,
                                                                  • Jim Jagielski    • Brett Porter
Executive Vice President


                                     ~75 Apache Top Level Projects (TLP)



     Apache HTTP                Apache Tomcat           Apache Geronimo                               Top Level Project
R. Fielding (PMC Chair)         M. Turk (PMC Chair)     K. Miller (PMC Chair)       …..               < > (PMC Chair)
  httpd.apache.org             tomcat.apache.org       geronimo.apache.org                            <…>.apache.org



       Each TLP is governed by a Project Management Committee (PMC)
              • Individual projects have different policies for deciding PMC membership

       The PMC Chairman is an Apache Vice President who reports to the Board
The Apache Software Foundation
                     Organization (cont)
Committees that report to the board
 Hardware and Infrastructure

 ConCom - Conferences, roadshows, etc

 Legal

 Public Relations

 Brand Management

 Fundraising
The Apache Software Foundation
                           More statistics

434 active apache mailing lists, recently accumulating 2,697 messages per day.




            Courtesy of http://apache.markmail.com
The Apache Software Foundation
                 Even more statistics

Monthly http requests to apache.org domains by category (~8.5 M hits/day)




                     Courtesy of Vadim Gritsenko
The Apache Software Foundation
                   Funding and Contributions
The ASF is (mostly) a volunteer organization
   Apache does not pay developers
   Board of Directors and Officers are not compensated
   Most (but not all) developers are corporate funded ‘professional OS developers’
   2 paid system administrators

Apache Sponsor Program
 Primary source of cash contributions

Non-cash Contributions
 Code, design discussions, bug fixes, documentation, user support, testing, …
The Apache Software Foundation
                                       Sponsors
Platinum ($100K/year)


Gold ($40K/year)



Silver ($20K/year)
    – Basis Technology, Covalent/VMWare, Iona/Progress

Bronze ($5K/year)
    – AirPlus International, BlueNog, Intuit, Joost, Matt Mullenweg,
    – Two Sigma Investments
Agenda

 IBM and the Apache Group: Forging a Partnership

 Apache Software Foundation: The Organization

 Development at the ASF: The Apache Way
Development at Apache
“The Apache projects are characterized by a collaborative,
  consensus based development process…”


Goals of the Apache Development Process (a.k.a, “The Apache Way”)

 Reduce barriers to project participation

 Improve quality

 Achieve consensus and resolve conflict

 Balance needs of corporate interests with needs of individual contributors
Development at Apache
        Characteristic #1: Open & Transparent Decision Making

Give anyone interested an opportunity to participate in and contribute to all aspects
   of the project. Reduce barriers.

 All decisions should happen on the project mailing lists

 Discuss large changes, significant designs or other ‘big ideas’ on-list
     – If a change could be controversial, discuss it on-list
     – Okay to fix bugs w/o discussion (almost always…)


 Power of information sharing …
     – More opportunities to receive feedback on design and implementation
     – You might learn a key insight from an unanticipated source
Development at Apache
                   Characteristic #2: Scoped Contributions

Patch Contribution Guidelines to Reduce Barriers and Improve Quality.

 One bug one patch - do not fix multiple bugs in a single patch

 Avoid gratuitous source code changes
     – Reformatting changes mixed with functional changes

 Decompose large function changes into multiple logically related commits
     – Facilitate peer review
     – Manage complexity
Development at Apache
                       Characteristic #3: Voting Process

Consensus confirmed. Conflicts Resolved.

 Vote on releases, decisions/ideas, code … just the big stuff
    – +1, -1, -0, +0

    – ‘-1’ - ‘I’m really not in favor of this but if no one else has a problem, I’ll go along’

    – ‘-1’ - ‘I’m absolutely opposed to this’. This is a ‘veto’. Method of last resort to
      resolve a conflict.

 Rule of 3 and 72
    – Minimum of 3 voters - Encourage diversity and community

    – 72 hour voting period - Time to review and consider the vote
Development at Apache



“The Apache projects are characterized by a collaborative, consensus based
  development process, an open and pragmatic software
  license, …”
Development at Apache
         Characteristic #4: Permissive Apache License version 2.0


ASF Code can be used & modified for commercial use w/o restriction
 Just don’t call your product “Apache”. That’s it!

Legal Due Diligence Processes to verify provenance of inbound code
 Contributors must sign an Individual Contribution License Agreement (ICLA)

 Code and Patent grants required for donations of large amounts of pre-existing code

 Click through agreement for submitting patches via JIRA



          Reduces legal risk for ‘deep pocket’ users of Apache Software
Development at Apache


“The Apache projects are characterized by a collaborative, consensus based
  development process, an open and pragmatic software license, and a
   desire to create high quality software that leads the way
   in its field.”
Development at Apache
                         Becoming a Project Member

 Demonstrate Technical Competence

 Participate in discussions, make contributions, earn trust and respect

 Communicate effectively on project mailing lists

 Demonstrate understanding and appreciation for the community and Apache
  project style

 Take time to mentor new contributors

 In a sustained fashion for a period of time


        You will be invited to become a Committer and PMC member!
Development at Apache
                               Apache Incubator

 Apache Incubator: How new Apache Projects are started

 Incubator Objectives:
    – Build a sustainable, diverse community - no solo projects
    – Train new project developers in Apache style open source development
    – A successful Incubator project (a podling) graduates to an Apache Top Level Project

 Interesting projects welcomed!

 Want to start a Project at Apache? Make an incubator proposal.
    – http://incubator.apache.org/
Further Reading
How Big Blue fell for Linux
 http://www.salon.com/technology/fsp/2000/09/12/chapter_7_part_one

Rules for Revolutionaries
 http://incubator.apache.org/learn/rules-for-revolutionaries.html


Producing OSS by Karl Fogel
 http://producingoss.com/

How Open Source Projects Survive Poisonous People (And You Can Too) by Ben Collins-
    Sussman and Brian Fitzpatrick
 http://video.google.com/videoplay?docid=-4216011961522818645




China Vector outline from: www.presentationmagazine.com
Thank you!
:           2010     8   14       15
                                 :                                                    825
                                         :   http://wiki.apache.org/apachecon/CFPApacheAsiaRoadShow2010
                                         :   http://www.apachecon.com/2010/Asia/cfp

                                     :                tonywu@apache.org
                                                      caijunj@apache.org




• Apache              Justin Erenkrantz
• Apache          VP Tim Ellison
•
• More to come…

Open Source at the Apache Software Foundation

  • 1.
    Open Source andthe Apache Software Foundation
  • 2.
    Introduction  Founding member& two term director of the Apache Software Foundation  IBM’s first official ‘commercial’ open source developer ( ?)  Committer on Apache HTTP Server & Apache Portable Runtime projects, Apache Incubator Project Management Committee member  Electrical Engineer by training, Software developer by trade, Physicist at heart, Sinophile by choice  Advisor to founding of Linux Technology Center and Eclipse Foundation  IBM Senior Software Development Manager, Apache Geronimo and WebSphere Community Edition and WebSphere Open Source Strategy.
  • 3.
    Agenda  IBM andthe Apache Group: Forging a Partnership  Apache Software Foundation: The Organization  Development at the ASF: The Apache Way
  • 4.
    The Apache Group 1995- 8 members organize around a website, mailing list and source code repository… Objectives:  Resume development on the abandoned NCSA httpd server  Core infrastructure required to build out internet ecosystem  Permissive licensing (lower barriers to use and participation) Origin of the name “Apache”  The name 'Apache' was chosen from respect for the Native American Apache Nation, well-known for their superior skills in warfare strategy and their inexhaustible endurance.
  • 5.
    IBM Partners withApache Group 1998 - IBM stops development of their proprietary Web Server, chooses Apache HTTP Server for their business needs  Inside IBM, the decision was controversial & difficult – Fear of ‘loss of control’ – Fear of legal liabilities & litigation  Apache Group’s decision to embrace IBM was equally difficult – Apache Group was open to corporate participation but … – Didn’t expect a big company like IBM to show up Could a partnership succeed? If so, it would be groundbreaking.
  • 6.
    Forging a Partnership TheContentious Issues… IBM - large, impersonal, bureaucratic/process driven  How to build business plans around ‘chaotic’ & ‘unpredictable’ open source?  How do we know who really owns this code?  How much will we pay in lawsuits for using this code? Apache Group - small, highly individualistic, agile  Would IBM dominate the community?  Do corporate developers know how to write good code? How to Bridge the Cultural Impedance Mismatch?
  • 7.
    Bridging the ImpedanceMismatch Some lessons learned by IBM…  Open source development is not ‘chaotic’ or ‘unpredictable’ .. – Just different from the corporate ‘command and control’ style – Very predictable, but requires careful observation and understanding  Control (of a community) is not necessary … or even desirable ... – ‘Influence’, which is earned, is sufficiently effective – Diversity of ideas creates fertile ground for innovation  It’s okay to ‘scratch your own itch’ and look out after your own interests .. – Generalized solutions preferable to overly specialized solutions – Developing directly in the community preferable to “power-planting”  Trust is really important - best achieved by being transparent & open – About goals and plans – Discussing, designing and implementing code
  • 8.
    Bridging the ImpedanceMismatch Transformational changes occurred inside IBM…  Institutionalized developer open source training… – How to be good open source community members – Legal issues related to open source • Licenses, Copyrights, Patents, Code Provenance – Use of open source code in IBM products to achieve business objectives  Fundamental change in how IBM Legal views open source… – Transitioned from ‘zero tolerance’ for open source to acknowledging valid business reasons to be involved in open source – Developed business processes to manage risk
  • 9.
    Bridging the ImpedanceMismatch Apache Group Key Insights ..  Recognized an emerging opportunity to create a corporate friendly open source ecosystem  Demand for corporate friendly OS would require more formal organization  Developed new awareness of legal issues.. – Source code provenance is really important to ‘deep pocket’ users of OS technology – More nuanced understanding of patents and copyrights
  • 10.
    Observations  Apache Group’searly interactions with IBM provided catalyst for creation of the Apache Software Foundation  Began to articulate rules for running effective corporate friendly open source communities (a.k.a. ‘The Apache Way’)  Insights gained drove IBM internal transformations regarding how open source was viewed  Experience influenced creation of the Linux Technology Center and the Eclipse Foundations
  • 11.
    Agenda  IBM andthe Apache Group: Forging a Partnership  Apache Software Foundation: The Organization  Development at the ASF: The Apache Way
  • 12.
    The Apache SoftwareFoundation So let’s talk a bit about the Foundation…  Founding Principles & Mission  Some Interesting Statistics  Corporate Organization & Structure  Contributions, Funding and the Apache Sponsor program
  • 13.
    The Apache SoftwareFoundation 1999 - Apache Software Foundation Incorporated Founding Principles  The Foundation exists to serve the development community  “Let developers focus on what they do best: Code. The foundation exists to do the rest” Justin Erenkrantz  Not-for-Profit  Not affiliated with or controlled by any government or commercial entity
  • 14.
    The Apache SoftwareFoundation Mission Statement The Apache Software Foundation provides support for the Apache community of open-source software projects. The Apache projects are characterized by a collaborative, consensus based development process, an open and pragmatic software license, and a desire to create high quality software that leads the way in its field.
  • 15.
    The Apache SoftwareFoundation Some Statistics  21 founding members (1999)  291 members (June 2010 - CAGR > 26%)  ~2300 committers (June 2010 - CAGR > 50%)  75 Top Level Projects (TLP) (June 2010 - CAGR > 48%)  Over 30 projects in Incubation (transition state to TLP)
  • 16.
    The Apache SoftwareFoundation Organization Foundation Governance Board of Directors - Greg Stein • 9 member Board of Directors • Shane Curcuru • Justin Erenkrantz • Executive officers (Appointed by the Board) • Doug Cutting • Geir Magnusson Jr • Roy Fielding • Brian McCallister Chairman, President, Treasurer, Secretary, • Jim Jagielski • Brett Porter Executive Vice President ~75 Apache Top Level Projects (TLP) Apache HTTP Apache Tomcat Apache Geronimo Top Level Project R. Fielding (PMC Chair) M. Turk (PMC Chair) K. Miller (PMC Chair) ….. < > (PMC Chair) httpd.apache.org tomcat.apache.org geronimo.apache.org <…>.apache.org Each TLP is governed by a Project Management Committee (PMC) • Individual projects have different policies for deciding PMC membership The PMC Chairman is an Apache Vice President who reports to the Board
  • 17.
    The Apache SoftwareFoundation Organization (cont) Committees that report to the board  Hardware and Infrastructure  ConCom - Conferences, roadshows, etc  Legal  Public Relations  Brand Management  Fundraising
  • 18.
    The Apache SoftwareFoundation More statistics 434 active apache mailing lists, recently accumulating 2,697 messages per day. Courtesy of http://apache.markmail.com
  • 19.
    The Apache SoftwareFoundation Even more statistics Monthly http requests to apache.org domains by category (~8.5 M hits/day) Courtesy of Vadim Gritsenko
  • 20.
    The Apache SoftwareFoundation Funding and Contributions The ASF is (mostly) a volunteer organization  Apache does not pay developers  Board of Directors and Officers are not compensated  Most (but not all) developers are corporate funded ‘professional OS developers’  2 paid system administrators Apache Sponsor Program  Primary source of cash contributions Non-cash Contributions  Code, design discussions, bug fixes, documentation, user support, testing, …
  • 21.
    The Apache SoftwareFoundation Sponsors Platinum ($100K/year) Gold ($40K/year) Silver ($20K/year) – Basis Technology, Covalent/VMWare, Iona/Progress Bronze ($5K/year) – AirPlus International, BlueNog, Intuit, Joost, Matt Mullenweg, – Two Sigma Investments
  • 22.
    Agenda  IBM andthe Apache Group: Forging a Partnership  Apache Software Foundation: The Organization  Development at the ASF: The Apache Way
  • 23.
    Development at Apache “TheApache projects are characterized by a collaborative, consensus based development process…” Goals of the Apache Development Process (a.k.a, “The Apache Way”)  Reduce barriers to project participation  Improve quality  Achieve consensus and resolve conflict  Balance needs of corporate interests with needs of individual contributors
  • 24.
    Development at Apache Characteristic #1: Open & Transparent Decision Making Give anyone interested an opportunity to participate in and contribute to all aspects of the project. Reduce barriers.  All decisions should happen on the project mailing lists  Discuss large changes, significant designs or other ‘big ideas’ on-list – If a change could be controversial, discuss it on-list – Okay to fix bugs w/o discussion (almost always…)  Power of information sharing … – More opportunities to receive feedback on design and implementation – You might learn a key insight from an unanticipated source
  • 25.
    Development at Apache Characteristic #2: Scoped Contributions Patch Contribution Guidelines to Reduce Barriers and Improve Quality.  One bug one patch - do not fix multiple bugs in a single patch  Avoid gratuitous source code changes – Reformatting changes mixed with functional changes  Decompose large function changes into multiple logically related commits – Facilitate peer review – Manage complexity
  • 26.
    Development at Apache Characteristic #3: Voting Process Consensus confirmed. Conflicts Resolved.  Vote on releases, decisions/ideas, code … just the big stuff – +1, -1, -0, +0 – ‘-1’ - ‘I’m really not in favor of this but if no one else has a problem, I’ll go along’ – ‘-1’ - ‘I’m absolutely opposed to this’. This is a ‘veto’. Method of last resort to resolve a conflict.  Rule of 3 and 72 – Minimum of 3 voters - Encourage diversity and community – 72 hour voting period - Time to review and consider the vote
  • 27.
    Development at Apache “TheApache projects are characterized by a collaborative, consensus based development process, an open and pragmatic software license, …”
  • 28.
    Development at Apache Characteristic #4: Permissive Apache License version 2.0 ASF Code can be used & modified for commercial use w/o restriction  Just don’t call your product “Apache”. That’s it! Legal Due Diligence Processes to verify provenance of inbound code  Contributors must sign an Individual Contribution License Agreement (ICLA)  Code and Patent grants required for donations of large amounts of pre-existing code  Click through agreement for submitting patches via JIRA Reduces legal risk for ‘deep pocket’ users of Apache Software
  • 29.
    Development at Apache “TheApache projects are characterized by a collaborative, consensus based development process, an open and pragmatic software license, and a desire to create high quality software that leads the way in its field.”
  • 30.
    Development at Apache Becoming a Project Member  Demonstrate Technical Competence  Participate in discussions, make contributions, earn trust and respect  Communicate effectively on project mailing lists  Demonstrate understanding and appreciation for the community and Apache project style  Take time to mentor new contributors  In a sustained fashion for a period of time You will be invited to become a Committer and PMC member!
  • 31.
    Development at Apache Apache Incubator  Apache Incubator: How new Apache Projects are started  Incubator Objectives: – Build a sustainable, diverse community - no solo projects – Train new project developers in Apache style open source development – A successful Incubator project (a podling) graduates to an Apache Top Level Project  Interesting projects welcomed!  Want to start a Project at Apache? Make an incubator proposal. – http://incubator.apache.org/
  • 32.
    Further Reading How BigBlue fell for Linux  http://www.salon.com/technology/fsp/2000/09/12/chapter_7_part_one Rules for Revolutionaries  http://incubator.apache.org/learn/rules-for-revolutionaries.html Producing OSS by Karl Fogel  http://producingoss.com/ How Open Source Projects Survive Poisonous People (And You Can Too) by Ben Collins- Sussman and Brian Fitzpatrick  http://video.google.com/videoplay?docid=-4216011961522818645 China Vector outline from: www.presentationmagazine.com
  • 33.
  • 34.
    : 2010 8 14 15 : 825 : http://wiki.apache.org/apachecon/CFPApacheAsiaRoadShow2010 : http://www.apachecon.com/2010/Asia/cfp : tonywu@apache.org caijunj@apache.org • Apache Justin Erenkrantz • Apache VP Tim Ellison • • More to come…