Research is a significant component of development in any industry, be it the pharmaceutical or aerospace industries. In the computer industry the semi-conductor industry has been good at research as demonstrated by the Intel’s prowess in manufacturing as well as its continued efforts to sustain Morse’s law. Morse’s law has led to the increased power of hardware to such a point that the tables have been turned: it is now software that needs to catch up to the power that is available from the hardware people. Sadly software is still in a sorry state as it grapples with complexity in a world that is increasingly connected and hence need disparate devices of varying capabilities and purpose to communicate with each other.
It looks like research is being carried out in the industry generally; Microsoft, IBM, Sun Microsystem, Google, Yahoo etch all have research projects in one area of software or the other. However, my concern is with the research that is done by the open source movement in order to improve products and perhaps even come up with new developments. The Singularity research project at Microsoft is a fascinating example of how to push the boundaries of technology and reshape the order of things. The very nature of the open source movement does not seem to be a conducive environment to carry out long term research with a focus to solving current problems. According to the Cathedral vs Bazaar essay written by Eric Raymond, the main imperative that brings progress to the open source movement is the need of a developer to cure his/her itch. Looking at the history of most of the prominent open source projects, they started off as an effort by a single developer; while it is easy to start a project to develop a single and specific solution to a problem, it is a whole different ball game when the solution in question requires advances in multiple aspects of the solution. Take an example like the Singularity project over at Microsoft Research, they have had to develop the compilers, tools and runtimes to implement the project.
The genesis and perhaps the power of open source has been lone coders or hackers who take on challenges and share the result of their effort with the community and others (in the community) extend the code shared in such a manner that will add new features and improvements as time passes. The very nature of the open source community does not need neither does it encourage central control beyond that provided by project leaders though project leaders are largely stewards of their projects.
The movement has proved to be very fast in catching up with developments in the industry. The manner and speed in which Xgl (Xegl) and AIGLX were implemented remains a classic characteristic of open soruce in my opinion. Apple started the use of composition/window managers in the Mac OS X and Microsoft was touting a similar feature as a key feature of the often delayed Windows Vista. Before Microsoft debuted Windows Vista, Linux distros had integrated Xgl or AIGLX into their new versions. With a release cycles of two versions in a year for most of the distros, the improvements in Xgl and AIGLX were rapid. At the moment, other projects have popped up to provide improvements and/or plug holes that have not been covered by Xgl and/or AIGLX. This particular reaction from the movement was borne out of the need to cure the itch of a developer (at least in the case of Xgl).
Sun Microsystems’ recent foray into the open source world through various efforts such as open source java and open Solaris is provide of the use of open source as an integral part of a business strategy. Companies like IBM and Oracle have and maintained relations with various open source projects. These organizations have a long history in research and developing innovative technologies. As they take part in the open source movement, I hope they realize that their involvement in the open source movement needs to go beyond donating code, pledging not to assert pattern rights, defend the open source movement in the halls of justice (SCO comes to mind here) … etc. They should provide long term research direction that may not be possible under the norms of the movement. These giants should provide the incubation periods necessary to get at least a prototype proof of concept to work and then build a community around the project and let the magic of open source work its wonders.