Why you should open source your python eggs.

In the last 15 months, I have released 15 Plone add-on products as open-source python eggs, available for download on Plone.org and Pypi, the python package repository. In the preceding 4 years of Plone development, I released only a single product.

In this process, I have witnessed a personal metamorphosis. I can draw a relatively clear line between 'Ye Olde Worlde' web-developer who was set in his ways and narrow minded in his methods, and the new! improved! open source developer.

I've seen the transformative power of (the act of) open sourcing and although I'm sure that a lot of what I have learned is already old hat to many developers, I would like to share the insights I have gained from this experience. Many developers have been made aware of the inherent advantages of collaborating in an open-source community, but to take part in it yourself and to experience the inherent benefits first hand, is the only way of developing a true conviction for the appropriateness of open source collaboration.

The fact is, that you have a lot to gain by engaging with the community, through collaborating, sharing your ideas and making your code public.

Image by MrTopf

Open source everything that might be useful to others

It's relatively common practice in the Plone community to develop a so-called 'policy package' for a new project. The policy package should tie everything together, configure the Plone application to the client's needs, and install all the other add-ons.

Additionally, a theme egg for implementing a custom theme is usually also required. These two packages, together with a collection of third-party and in-house developed packages (each responsible for a specific feature set), on top of Plone itself, constitutes the software product presented to the client.

Before I was introduced to the idea of a policy egg by Martin Aspeli and others who championed the idea, I would usually when given an assignment, develop only one Plone add-on product for it. This add-on then contained all the custom software code that I wrote for this specific client and/or product, regardless of whether certain parts of it would also be beneficial to other people or projects.

This is of course not the proper way to develop a sustainable, reusable and configurable solution for a client. Instead, every piece of functionality that constitutes a logical grouping of features and that could theoretically be reused somewhere else, should be packaged separately in its own python egg.

And here's the catch: you should ideally open-source everything except the policy package. Even the theme egg should be considered for open sourcing.

The policy package serves as the puppetmaster, that ties these different pieces together to make the integrated unit (or puppet show). It is the proprietary configuration that contains the client specific implementation details, the client's copyrighted logo and images, the site copy and everything else that makes the site unique to the client. It's the secret sauce or the kernel's 11 herbs and spices, in a matter of speaking.

With regards to the rest of the custom developed eggs in your project, release them as open source eggs on Pypi! You have a lot to gain by doing so.

Let me elaborate on why...

Advantages

Open sourcing your code forces you to be a better programmer

The implicit mental adjustment coming from the knowledge that you are writing source code which will be read and modified by others, is already enough to considerably enhance the quality of the code that you will write.

Why? Because of the added social pressure.

Glossophobia, the fear of public speaking, rumoured to be the most common fear in the minds of people, is a consequence of the fear of public humiliation.

Nobody wants to be humiliated publicly. Not for being a poor speaker and not for writing bad source code. The fact is, that you will make more of an effort to write clear and legible code that follows best practices when you are writing for an audience (the community of developers) than when you are writing for personal or 'in-house' consumption.

Before I started writing code with the intention of open-sourcing it, I would be much more inclined to include quick and dirty hacks to "save time" or to "simplify" things. Ironically, quick hacks such as hardcoding a value or overriding a template instead of putting it in a viewlet will in the long run, waste time and overcomplicate things.

A quick real world example: Your boss or client wants a facebook style action panel on the bottom of a site.

Do you:

• Decide not to open source this functionality, and instead add the styles and templates to the theme egg while hardcoding the links on the panel because it's quick and easy?
• Decide from the getgo that you will open source this and therefore create a separate egg for this piece of functionality, that adheres to best practices to make it reusable, extendable and configurable?

The second option might take longer initially, but its benefits pay off in the long run, especially when the client requires technical support.

The community provides an additional level of quality control

I'm careful here to add the word 'additional' since you should not solely rely on the community to test your code for you. On the contrary, other people will be reluctant to try out your products if you don't put in the initial effort or write tests and doing quality control yourself.

That being said, granted that you have released a quality package, you will receive an additional layer of community powered quality control by virtue of the fact that the developers and integrators using your code will let you know when they run into bugs, and often even fix them for you! "With many eyes, all bugs are shallow."

The community provides new perspectives, ideas and enhancements.

This past year, various developers have contacted me with suggestions, ideas and improvements to some of the packages that I have released. Some have made a meaningful amount of effort to explain why a certain solution might be better, and in the process have provided me with a free education and broadened the scope of my Plone dev knowledge.

Overall, the quality of the packages that I have released, has been improved through the sustained contributions of other developers in the Plone community. This is after all one of the main advantages of open source, seen from a purely pragmatic point of view. To witness this process in action and with code that you yourself have contributed, is very satisfying.

Social currency (A social C.V or Résumé)

Open source contributions, being easily searchable and in the public domain, form an authorative record of the depth and quality of an individual or company's output.

Compared to the sections traditionally contained in a Curriculum Vitae (Résumé), your open source contributions form your work experience section, while their contributors and users serve as references. Sustained open source contributions coupled with helpful and courteous interaction with the community, provides a more authoritive record of ability and character than any CV ever could.

Photo by Superbez

Suggestions:

I'd like to round up this post by making some suggestions which I consider will better the chances of your product not only being used by the community, but also receiving code improvements and bugfixes.

Don't host in-house:

Use the collective repository to host your Plone source code. We're lucky to have it, and that's what it's there for. Alternatively, use Google's project hosting or github. Although, in my opinion, it doesn't make much sense to call something collective.* if it's not hosted on the collective repo (I know others disagree with this).

You want the barrier to entry for participation in your code to be as low as possible. That means making it as easy as possible to quickly provide improvements and bugfixes.

If you host your open source products privately, you might gain users, but you will have far fewer contributors willing to fix and enhance your code, simply because they're not willing to jump through hoops to be able to do so. Private or walled off hosting discourages contributions and promotes forking. Which would be a forking shame (sorry, I had to).

Start your project in the open:

Don't develop your packages privately, waiting for them to be "good enough" to be added the to a public repository.

Start from the beginning on developing publicly on a repository such as the collective. This will ensure that you stay aware of the fact that you are writing code for others and not just for yourself. As mentioned before, when you consider the fact that others will have to read and digest your code, you will more carefully consider your method, class and variable names. Do they make sense? Are they consistent? Is the code explicit and readable? etc.

Release your Plone eggs on both Pypi and plone.org:

Some users prefer to search for Plone add-ons via plone.org, while others use Pypi. Cater to both to maximise your visibility. Plone.org goes a step further by among other things allowing you to upload a screenshot and your company's logo.

Tools such as zest.releaser or jarn.mkrelease make releasing eggs a breeze. Use them! :)

Conclusion:

When it comes to open sourcing your code, the fear might arise, that by making your code available for others to use and adapt, you might in some way lower your chances of monetizing the code. For people living with a mindset of scarcity, in which ideas and creations must be guarded closely to safeguard their success and to prevent others from "stealing" or profiteering from them, it might be difficult to make the paradigm shift necessary, towards a mindset of abundance, opportunity, sharing and collaboration.

The fact is, that you have a lot to gain by engaging with the community, through collaborating, sharing your ideas and making your code public. ;)