David McCusker has recently joined the OSAF staff as a developer. We're late in getting his name and info up on the people page of the OSAF web site, but that will be remedied soon.
He keeps a very active weblog which describes many technical and design discussions at OSAF along with his reactions. If you're interested in more depth on Chandler this is another good place to turn.
Mitchell Baker has joined the OSAF staff. I'm thrilled. No one knows more about working with the open source community. Mitchell will work part-time for OSAF and will continue as mozilla.org's Chief Lizard Wrangler. Details here and here.
Robin Dunn, the principal author of wxPython, has begun a six-month contract for OSAF to improve wxPython/wxWindows. His first project will be working on the MacOS X version of wxPython. All work will be contributed back to wxPyhton/wxWindows.
More staff news soon.
Not surprisingly, one of the most common questions I get is when will Chandler be ready. Our original standard answer goes like this: We will first put out code for developers to look at by the end of the year. It will be an extremely partial alpha. Optimistically, we could have a 1.0 by the end of 2003. Pessimistically, it will be 2004. I've never been entirely comfortable with this response, so this is my chance to dissect the issues surrounding Chandler's schedule.
Software, like construction projects, is typically late, sometimes very, very late. It typically takes longer and is much harder than any estimates. There are, of course exceptions, like the original version of Lotus 1-2-3, which shipped exactly on time, and there tend to be exceptional reasons why. In the case of 1-2-3, it was implemented almost entirely by one person, Jonathan Sachs, and it was the fifth time Jon had implemented a spreadsheet. Not only did he know what he was doing, but he had enough experience to accurately estimate how long each part of the coding would take. This was one of the keys. The other was that we had a relatively precise idea of the feature set going into the product. In other words, we both knew what work had to be done, and how long each piece would take. It was possible to roll up the schedule from the bottom up.
With Chandler, it its current state, we don't yet know how fast we're going to be able to go, nor how far we have to get to the first release. This is not so much a matter of poor practice as it is reflective of the early stage we are in. The developers have just started to write production code, which is to say, enough decisions have been made about the fundamental building blocks of the product and how they will be put together, to start to build the actual product. Once we have perhaps three months of development under out belts, it will be possible for the first time to predict our rate of progress. To the extent that the specific tasks to be done have been identified at a fine enough granularity to estimate completion times for each task, it becomes possible to develop more or less reliable schedules using a bottom-up approach. That's the goal, and until then, the most accurate summary answer about a schedule is "we don't really know".
As to the absence of a spec, I think it turns out to be a virtue at present. The features are grounded by a set of commitments (e.g. the preliminary feature list on the web site) which should be thought of not as sketches, not blueprints. What is really thrilling is the depth of thoughtfulness and creativity on OSAF's design mailing list which has ensued. Thank you, participants. I've been reading the 30-50 daily comments and have been truly impressed. I can guarantee the design list will have an enormous impact on the product itself, and I'm hopeful we will have on board very shortly the new staff who will coordinate and distill the conversations there.
Of course, I haven't gone into all of this blindly. I know just how hard it is to ship a fully competent product. Fit and finish issues alone are always incredibly time-consuming. The last 10% of the product can take 50% of the total time. There will necessarily have to be a series of releases over a period of years in which the product will mature. One of the key tricks is in proper sequencing -- teams should build on a firm foundation before trying to erect the shell of the structure. However, if the foundation and shell are secure, some of the rooms do not need to be built out before the occupants move in.
Also, if Chandler is good enough to do what people do currently, and there is a painless migration path, it only takes a handful of killer features to make a great first release. The delay in some or even many of the advanced and specific features for a PIM will no doubt frustrate some people, but if can establish credibility as to the quality and rate of progress, I am guessing most people will give us a break on this.
Another schedule variable has to do with how soon and how effectively development by volunteers can take place. It's an old saw that no matter how many men you put on the job, it still takes nine months to make a baby (and let's not forget the hard work is done by woman). Originally I had not counted on more full-time or most full-time volunteer developers, but it is beginning to look like we'll have some on the team, which is fantastic news. It gives us some choice about moving more quickly vs. having a wider scope for the first release.
One area we are rethinking, based on input from the mailing lists and other sources, is whether also build Chandler from the outset to support large organizations, not just small-to-medium ones. There may be an architecture and protocols which reduce the amount of work to do both to less than the 2x effort I had imagined.
William Friedman
and less understanding of code and the move off the desktop makes the devices less and less powerful I wonder if there will be fewer and fewer folks who know how to look under the hood."
It's certainly true that the automobile has become much less of a target of tinkering than it was in previous generations. When cars get complex, there is a loss of opportunity, but the news is not all bad. Today's autos have gotten a lot more robust, lasting longer and going further with much less maintenance than the cars my father had when I was growing up. It would be nice if software became equally reliable. You know the joke, if your car were run by Windows, every hundred miles it would stop suddenly for no reason and you'd have to start the car again to get it to go (to be fair, less true with Win XP, than Win 95, which is when I think I first heard this).
At the same time, software or aspects of it are likely to remain tinkerable. For one thing, you can put a programming interface to almost anything in software, so as software evolves in functionality one can continue to invent new API's on it to let people fool around with the code. On the one hand, there are already large parts of open source code which tinkerers are not advised to fool with, e.g., the Linux kernel. But there are much larger territories where almost anything goes.
Large parts of Chandler will be written in Python, which is an accessible high-level scripting language. This will encourage participation and experimentation, which is one of our goals. We are architecting the product to be highly modular so that many, many kinds of components (rule sets, spam filters, new views, smart parsers, import/export filters, indexers, etc.) can simply be plugged-in. We will certainly maintain a version aimed at the vast majority of everyday users, but we hope the adventurous will have a field day.
Dave Winer asked (in a comment to yesterday's entry) about OSAF's position on patents.
Dave,
OSAF hasn't worked out a specific position yet on patents, but obviously we will be doing this. You know my historical position, which hasn't changed much. The patent system is broken. Most software patents should never have been issued, as there is either prior art or they are trivial. Nonetheless, we who are in the software world have to deal with patents and a broken system while working to fix the system.
One big goal is not to let patents get in the way of open source. This means that any patents associated with code OSAF distributes have to be licensed royalty-free to OSAF with a viral cluase.
An issue is whether OSAF should try to obtain patents, licensed royalty-free with its code, in order to protect ourselves better against claims of patent infringement by others. This is akin to saying, "we're peaceful, but in order to protect ourselves against attacks, we must arm ourselves." My idealism and pragmatism fight each other on this one.
Comments, anyone?
I edited the settings on past entries to enable comments. I will be responding to comments, albeit briefly, from time to time.
Committing to this weblog has made me more sympathetic to journalists who regularly work under a deadline. I'm falling short of my goal of posting every other day. Herewith, some catch up.
For the first time in longer than I can remember, I haven't been able to get through all my email by the end of the day. This has been pretty much of a personal religious observance, so it feels somewhat sinful to leave mail opened at the end of the day. I try to be fanatically organized (which may be why I like the idea of Chandler so much). For email this means reading everything the day it comes in (not counting non-OSAF mailing lists), replying to all of the mail which requires a brief answer, and developing an action plan to deal with anything requiring considered thinking.
It's been a pet peeve of mine that many people don't have the courtesy to reply, even briefly, to personal email from a professional colleague. I'm not talking about spam, other kinds of bulk mail, or email from strangers. For instance, I recently sent out a query to about 20 CEO's of venture-backed firms with whom I have at least a slight connection concerning their interest in participating in a research project on quality of worklife in startups. None of it came out of the blue to any of them. All the CEO's were aware of the project. I would estimate perhaps 2/3 of the CEO's never bothered to reply at all, even though I included pre-fab check boxes to make it easy to respond. I just don't get this behavior. (By the way, all of the CEO email addresses had been verified.)
For now, however, I'm not able to reply to all of the email I receive. If you have written and haven't gotten anything back, I ask that you please not take it as a sign of incivility. I may not be able to acknowledge messages from well-wishers (of whom there have been many) but all of you should know the good will expressed for OSAF has been much, much larger than any of us imagined, and we thank you for your support.
Many people have written to volunteer to test Chandler, work on adapting it to other languages, or help with other activities requiring a near-finished product. Please check the web site periodically or sign up on the announce mailing list to stay abreast of Chandler status.
There have been a lot of suggestions of other products and projects to look at because they bear on what we're up to. We're compiling all of these and will be working our way through them over time.
One piece of good news -- we are getting a better handle on inquiries to info@osafoundation.org. At this point, general inquiries about OSAF will probably be handled more quickly if sent to the info mailbox than to my personal address which is listed on the web site.
Jobs at OSAF
We've been interviewing candidates for the open positions listed on the web site. All told, about 300 people submitted resumes. The quality has been higher than any of us on staff have seen over the course of our careers. It's not just that it's a tough job market, though that certainly contributes to the volume. I think it's also that developers and others really want to work on a product that matters and in an environment that isn't mired in big company politics. In fact, the flow has been so prolific we have turned off the spigot for now. Expect to see some terrific additions to the OSAF staff over the next month as we work through the hiring process.
Communication & Community
Over the past week Andy Herzfeld has posted two important documents, one on the Vista prototype we created and the other on a strawman proposal automatic secure mail. Lively discussion ensued and continues on the design mailing list. We're preparing a number of other documents, both setting out the thinking to date as well as putting a stake in the ground on key issues. We will have something on RDF and Shimmer (the prototype database used by Vista) as well as on current thinking about the calendar. We understand that an important part of the process is to make visible what is going on with the core staff. You should also see an increase in OSAF developer postings to the dev mailing list over the next week.
Responding to a question posted on the design list about whether there is a formal consensus on what Chandler will and won't have in it, I wrote:
There have been a bunch of working decisions made by OSAF, but they have not yet been compiled into a single structured document. The web site, my weblog, and posts by me, Andy, and other OSAF staff in this list and in the dev list all contain many individual pieces. It's obviously much too unwieldy, incomplete, unworkable, and even incoherent as is. A working decision is just that -- something we think we're going to do and the reasons why. Some decisions will be easy to change, others not at all, depending on how central they are.
We have the beginning of a plan to address this and some related issues by creating a topically organized wiki as a collaborative document to represent the current state of thinking and discussion about Chandler design. The wiki will include both design commitments and summaries of major design issues being discussed on the list. In this way, someone who has an interest in a particular area can consult the wiki as a starting point and from there dive into discussion threads on the list relevant to that topic. We hope this will facilitate community participation in the design.
How will we do this? We'd like the wiki to be open to volunteer editors who want to take responsibility for tracking a certain area. OSAF staff will also actively participate in editing and keeping it up do date. I'm waving my hands here as this is a part we haven't yet worked out. Obviously, a lot depends on having a logical topical organization overall and editors who can succinctly summarize different points of view about particular issues as they are written up in the design list.
The details of the plan will be worked out on the process list once we can put up a strawman draft, which will be shortly. We're all going to learn together how to do this
In the time before we unveiled the OSAF web site, I worried about how to create a sense of urgency about the project. In the world of venture-backed high tech startups, urgency is created by the need to ship product before the money runs out. As Samuel Johnson said, "The prospect of being hung in the morning concentrates the mind wonderfully." Since the announcement, it has become clear that meeting the expectations we've created and not disappointing people has assumed a major role as motivator.
Announcing a project without having code to show, that is, committing an act of vaporware, is generally not regarded as a best practice in the software world. It lessens credibility and heightens cynicism. So why did we do just that? In an ideal world, we would have announced at a time when we had code to show. In the real world, I worried that the OSAF story would leak out and we'd lose the chance to have a "big bang" coming out.
I also thought that it might make things easier to recruit staff and get help if people knew what we were doing. We had pretty much all of the development preliminaries out of the way over the past year and had settled on an architecture and enough of a product definition to begin production coding, so it felt like an assumable risk to announce. It's turned out that we're clearly accelerating our progress as a result of the interest and early participation of many talented people and organizations. So far the bet is paying off.
We now face a series of challenges. One of the first is how to keep moving along with the product definition while still benefiting maximally from community involvement? On the one hand, the more high-quality input and debate about the product the better. To paraphrase Bill Joy, most smart people don't work in your own organization. On the other hand, decisions must be made, and voting, even if it is non-binding, isn't the way to go. Some successful open source projects have solved this by having a Benevolent Dictator for Life (Linus Torvalds for the Linux kernel, Guido Rossum for Python), who is respected, sets general directions, and makes final calls. I'm honored that I've been proposed for this on the design mailing list, and I accept. So now my head is really on the chopping block. Respect has to be maintained and earned through action. I hope I'm up to it.
In the product definition itself we must find a way to reconcile conflicting goals.
How does Chandler manage to be both an application and a platform? How do we build something which is an easily adoptable solution for end users, as individuals and in organizations, while also creating a platform on which developers can both extend the core application and build new ones?
How do we serve both the everyday user who wants an uncomplicated tool to meet basic needs and the power user with an appetite for advanced functionality?
How do we create a user experience which is sufficiently familiar to provide a smooth transition for new users yet is sufficiently different to support major new capabilities?
I'll have more to say about each of these in the coming days.
You may have noticed I've turned comments on. This is an experiment. Let's see what happens.