For the past few days an ardent discussion on the OSAF design mailing list has revealed there is no consensus about the best way to collaboratively gather and sift design input about Chandler, a task which has become quite important.
I've learned since yesterday about the wiki, the twiki, the zwiki, and the wikipedia, all variants of an open source tool for collaborative editing. Each has its partisans and critics and each is used, in varying ways, by open source projects. We are investigating.
In a conventional corporation, there are a basic set of ground rules about how activity is organized. Corporations are still ultimately hierarchical. Some pyramids of power are steeper, others flatter, but they all point to the top. Anyone who works in the business world absorbs a set of default rules about how workplaces operate, e.g., your boss tells you what to do, but you don't tell your boss what to do.
As an open source project, we are experiment in progress. Like every other project we have to determine how decisions are made and who has what kind of power. Other projects have pioneered various methods of organizing their own activity, i.e., who can submit code for inclusion and who can commit it. There's a lot to learn from them and there is not yet any general consensus.
I had a fascinating conversation yesterday with Mitchell Baker, the general manager of mozilla.org, after which I began to appreciate the scope of process issues we are going to have to deal with. Active and successful community participation will require careful choice and use of the communications vehicles. I've already seen that the openness of mailing lists coupled with relatively wide membership and the willingness of participants to express themselves means that more than one side of most every issue is going to be brought up, no matter how small the issue.
Case in point: I received an email from the design list which I wanted to reply to. I use the reply command of Eudora. The new message is addressed by default to the person who posted the message I'm replying to, not to the list as a whole, which is what I want. I make a note to myself let Morgen, a developer who's currently doubling as system administrator to see whether the Mailman program we use can be reconfigured to fix what I regarded as a problem. Before I got around to sending it, I read perhaps a dozen postings to the list discussing what the correct behavior should be. Surprisingly, there were articulate and strong arguments on both sides. Not only that, but there are already lengthy essays on this posted in various places on the net which were cited.
Looking into it, I discovered there are substantive points about how replies to lists should form the To: address. One the one hand, since the previous message was sent by a poster to the list, a reply to the message ought to be sent by the default to that poster, as a reply is meant to go to the author, not the recipients, of the previous message. If I want to include the recipients, I should use the Reply to All command. On the other hand, what most people probably want to do is to reply to the list (and perhaps copy the author), in which case the "proper" solution doesn't do what is wanted and requires extra work. So why not munge the headers of the posted message such that when I do want to reply, the To: address of the new message is the list. For the rest of the gory details, you could start here
In a world in which all points can be discussed endlessly, it becomes important to have generally agreed-to processes by which decisions can actually be made and respected. Whether munging reply-to headers is really harmful or not isn't the point. It is that it's a Brave New World out there and we better get used to it. What's new here for me is not the endless discussions of mailing lists, but their impact on a project which is trying to work in an open and participatory fashion. And everything I discussed here is just the tip of the iceberg.
The design mailing list is proving to be very successful in my view as a way to gather input about Chandler. There are a lot of sympathetic, sharp-eyed, clear thinking folks who are making valuable contributions by asking questions, raising issues, making suggestions, and engaging in a critical thinking process. So, first of all, thank you.
It's clear there need to be additional forums and structures through which to coordinate the design process. No would argue that a mailing list is sufficient. As the pattern of recurring questions becomes clear, we will publish one or more periodically revised Design FAQs on the web site. Following usual net protocol, readers are advised to check there first to see whether a question has been answered or a suggestion made.
Using a Wiki to gather comment also sounds like a good idea if the details can be worked out. It would be possible, for instance, to start with a topically-organized outline covering the functionality of Chandler itself (e.g. it would have nodes for topics like replication and synchronization, serverless calendaring, Web and PDA access, etc.) in a then add comments. One aspect which has to be thought through are read/write/edit permissions. The Wiki we use internally is completely open to everyone. I think some kind of moderation and/or editorial control would be appropriate, but I'm not familiar with what the Wiki (or other tools) afford.
It also seems like a good idea to collate and summarize major feature requests and themes of the list. Ideally, each summary point could point back to the posts and./or threads where it was raised This project may have to wait until (1) there is someone who can do this, and (2) there is a tool which makes it easy to do.
I think it's important to maintain some distinction between the "what" and the "how" of Chandler functionality, with more the "what" on the design list and more of the "how" on the dev list. It's not possible or desirable to make a bright-line distinction between the two, but nonetheless, I am going to start suggesting certain conversation be moved from design to dev if they get immersed in the technical details of the best way to achieve a particular result.
Finally, you may notice this piece appears both in my weblog and on the design mailing list. I will occasionally cross-post. The weblog has a broader audience than the design mailing list, and it's a way of bringing aspects of the design discussion to a wider audience.
OSAF is covered in the New York Times of Monday, October 28. Said one expert "I haven't seen any evidence that there's a hole in the market here," he said. "But all the rational people have been completely wrong about most of these markets. So the fact that this sounds loony is probably a good thing."
I've been reading the postings to OSAF's design mailing list. It is incredibly heartening to be getting so much high-quality feedback in the form of feature suggestions and mentions of relevant products and projects, all of which is adding to the wealth of resources we have to tap into as we go forward.
I'm about halfway through everything posted since Sunday and am organizing a series of responses. Unfortunately, to do this well I need the very product we're trying to build. Lesser means must suffice for now.
A little more info about our approach:
Chandler will represent chunks of information as items, much as Agenda did. An item may consist of an email, an appointment, a contact. It can also be a document. An item can be thought of us having a body and a set of attributes (or meta-data).
Views are formed (logically) by specifying a query and running that query against the repository of all items. As in Agenda, an item can appear in more than one view. This is the underlying mechanism by which we will do the equivalent of "virtual folders".
Views can be of a single item type, e.g., email, or than can be of mixed types, e.g., all items relating to a single subject, regardless of whether they are emails, attachments, contacts, or appointments.
Every item in the system will have a unique URI, so it is referenceable, both from the user's own machine and remotely.
Items can be linked in arbitrary ways as well.
Whereas Agenda was limited to a single hierarchy of categories (equivalent to attributes), in Chandler we are using an RDF-compliant schema as the backbone. It will come with a basic schema for PIM's and it will be extensible, although we are still thinking about how extensible it will be, e.g., in terms of interoperability between different schemas.
I failed to mention Ed Belove as one of the original creators of Lotus Agenda, along with myself and Jerry Kaplan. Sorry, Ed.
I also was unaware of the existence of a Macintosh version of Outlook. See http://www.microsoft.com/mac/products/outlook/outlook_default.asp?navindex=
s. It's unclear if there is a significant user base.
I've already seen the propensity of the media to position OSAF's project as, "an Outlook Killer" (Slashdot). CNET's story yesterday opened with this: "Can a fledgling nonprofit organization with half a dozen employees challenge the largest software company in the world?" In the real world, matters are considerably more complicated.
One thing CNET did get right -- we're not aiming Chandler at the large enterprise market. As shipped, I'm certain it will flunk the checklist because we are not doing the work to make it scale to an organization of 1,000 or 10,000 people. Selling to large enterprises is where Microsoft rakes in the big money for Exchange server(s) and license fees . In that sense, as CNET reported accurately, we're not a threat to Microsoft's business.
We are trying to level the playing field by giving small & medium organizations collaborative tools which are as good as what large companies have had. We think we can do this in a way which doesn't have the administrative burden of Notes or Exchange. We're trying to be faithful to the original spirit of the personal computer -- empowerment through decentralization.
If Chandler gets initial traction, then perhaps with another turn of the wheel it will grow up, much as Linux did over the course of quite a few years to become an enterprise-class product. So, in this sense, it's a potential long-term threat, just as Linux emerged as competition for Microsoft in the server market. If I were Microsoft, I'd be worried about open source in general, not about losing Outlook/Exchange market share any time soon. With or without OSAF, I believe all of the applications in Office will be commoditized with equivalent free versions. I can see it happening
Another way in which headlines can mislead is that, from an internal vantage point, we're focused on creating an innovative, high quality product that users love. We're not thinking about how to beat Microsoft. Competition is a great motivator, but it's not the only one. Giving computer users more software options by making a great product is quite a motivator as well.
I've spent my time criticizing Microsoft in public and will do so again as circumstances dictate. A project of this scope, however, needs to be fueled by something positive, not by a desire to get even or show them a thing or two (not that my public criticism reduces to sour grapes!).
P.S. CNET should have egg on its face for including the following: "Kapor said he also has no plans to seek venture capital and has no visions of an initial public offering for his venture." OSAF is a non-profit -- I can't remember the time a non-profit raised VC or did an IPO.
To judge from the feedback, OSAF seems to have struck a responsive chord.
As of 10:45 A.M. PDT today, here are some statistics:
91,000 pages retrieved from www.osafoundation.org with 1.2 GB served. The Slashdot thread "Mitch Kapor's Outlook- Killer" has received 720 posts (203 rated 3 or higher). We now know what it's like to be Slashdotted. There have been approximately 85 posts to the design mailing list at osafoundation.org. I've received about 150 individual emails, including a number from old colleagues. And two donations via PayPal. Hello, all.
We've already fixed some small bugs on the web site and changed the stylesheet so this weblog should now be displayed in easier-to-read black, not gray.
Here's what I said in the design list: "First of all, thank you to all of you for your wonderful input. I am enormously impressed with the quality of the postings here. It is going to take some time (to put it mildly) to digest all of them, especially given that there are some other projects mentioned here which we were not aware of. While I'm not going to be able to acknowledge each contribution individually, I will be responsive in terms of how the various issues raised here have (or haven't yet) entered into our thinking."
Many old Agenda fanatics have put in a good word. Other PIM's like ECCO have their advocates as well. Many references to related open source projects to be checked out.
The OSAF staff and I will be digging out from under all this. Keep those cards and letters coming and thanks for all of your support.
The product, which is central to the whole undertaking, is a new take on the Personal Information Manager. It will handle email, appointments, contacts and tasks, as well as be used to exchange information with other people, and do it all in the spirit of Lotus Agenda, about which more here and here. Agenda, for those who aren't familiar with it, was a DOS product I designed (along with Jerry Kaplan) in the late 1980's which introduced a new kind of database optimized for entering small items of information in a free-form manner, and then adding organizational categories on-the-fly. It was much beloved by a few, despite (or perhaps because) being abandoned by Lotus.
We are trying to make a PIM which is substantive enough and enticing enough to make people want to move to it from whatever they are currently using, which statistically is probably Microsoft Outlook. I'm not going to bash Outlook here. Suffice it to say that while feature-rich, it is very complex, which renders most of its functionality moot. Its information sharing features require use of Microsoft Exchange, a server-based product, which is both expensive and complex to administer. Exchange is overkill for small-to-medium organizations, which we think creates on opportunity we intend to pursue (as well of course as serving individual users)
Have I mentioned it's going to run on Macintosh, Linux, and Windows and will not require a server? This is an ambitious goal, but we are convinced is possible to achieve using a cross-platform tool kit. (We are working with wxWindows/wxPython).
Also, everything is going to be fully open sourced.
Users in large enterprises will have to wait beyond the first release to get a version they can fully use, as we're deferring the substantial engineering required to scale the product. Once we have an initial release, we think it would be an excellent entrepreneurial opportunity to build a full-out large enterprise version on top of our code base. This is an opportunity OSAF is not going to pursue itself, but one we think could be very attractive to commercial developers. That door won't open for quite some time, but it's worth contemplating now.
This entails making sure we dot our I's and cross out T's with respect to all the features a product must have to be best-of-class, without sacrificing ease of use. We need to worry about migration paths from existing products, synchronization with PDA's and a whole host of details beyond core functionality which are required to make a truly first-rate product. On top of that, we have to have perhaps half a dozen killer features elsewhere unavailable, which I will be writing about in future entries. (Don't mean to tease; there's just too much to say all at once). One area which I will mention is that we have a lot of faith that the general and powerful information-sharing technology (built on top of Jabber) we are embedding, will make it trivially simple
A couple of months ago, it became clear that we could not do all of the above while at the same time fully realizing all of the new ideas we've developed about helping people manage information better. This gave rise to an important idea, which is that we see this project as needing to go through multiple major releases to grow up and become fully realized. We felt it was important to start with something which could over time gain wide adoption, because then there would be a larger potential base of interest for future developments. All of which is to say we're going to wind up deferring working on certain cool features in order to get an initial product out the door.
At this point, a small team has spent the better part of a year thinking through the problem space and developing the fundamental of our approach and has just begun writing the production code. We've made a number of fundamental decisions about the architecture and have arrived at a preliminary set of features. Andy Hertzfeld has built a terrific prototype which enabled us to explore lots of new ideas.
To join a discussion on the design of the product see this.
Some things I intend to write about:
Time: when is this thing going to ship?
Money: where is the money coming from to do this?
Our candidates for killer features
What was so great about Lotus Agenda?
RDF (Resource Definition Framework) and what it brings to the party
The sorry state of most software design today: how we got here
In the era of the Web, are PC applications obsolete?
During the decline and fall of the New Economy in early 2001, I was winding up a highly educational but ultimately unfulfilling stint as a venture capitalist. As I sat amidst the ruins of the dot com world contemplating what to do, I was looking for a next thing that would be both personally meaningful and contribute something to the world of computing.
In doing this, I found myself on the horns of a dilemma. Over the past 20 years, I'd worn many hats in information technology: consultant, developer, software designer, entrepreneur, CEO, angel investor, and VC. Of all of these, I most loved designing software, particularly the kinds of applications a person uses in everyday life. Other roles were each compelling in their own way, but the one which I found most sustaining was as a software designer. My passion about making useful software artifacts remained, and I had accumulated a major backlog of innovative ideas.
For the longest time, I could not see how to test those ideas in reality's lab. The whole idea of founding a company to development of new productivity software was a complete non-starter. No sane VC would or should fund a venture to compete with the Microsoft monopoly. As a different option, I refused even to think about the self-indulgence of "vanity publishing". No, if I was going to invest much time, effort, and money to develop something new, I wanted it to have a chance to have an impact in the world. But how to accomplish this?
It came to me slowly that going the open source route might be the answer. The enormous impact of Linux hardly escaped me, yet I was in no way an open source zealot. It was truly impressive how Linux had grown organically without being owned or controlled by anyone to the point where it would become a preferred alternative for servers. But for the end-user, Linux was still messy and full of holes, and the culture of open source, as I saw it, was much more about developers making tools for developers than with making quality applications for end-users. Open source felt risky, but the idea of making something, which, if it caught on, would be freely shared and improved upon felt extremely compelling. I could see it having lots of impact.
In the spring of 2001, I funded an initial, limited experiment by hiring a consulting group to prototype a couple of the key ideas. I quickly saw that to get real traction, there would have to be a small corps of full-time developers, not just consultants. We need to form a group with a common mind-set. I was beginning to get very excited about the idea of working again with a small group.
In the summer of 2001, I took the plunge, committed to open source, and hired the first employee of a fledgling non-profit. Why non-profit? I had not, all of a sudden, ceased to believe in the virtues of capitalism. But I wanted to make a clear statement, that my intent was not to use this as a vehicle to make more money. I would be very happy for others to make money and intended to find a licensing scheme which would permit both non-commercial and commercial development on our code base.
Thus was born OSAF.
Welcome! Here is where I expect to be writing at least twice a week my personal thoughts about the Open Source Applications Foundation with occasional digressions into whatever else interests me. For more info on me, see my home page. For comments, send email to email@example.com. This is my first foray into blogging; please bear with me as I learn how to add the more sophisticated capabilities bloggers are now using.