Taking things public

About two weeks ago I realized that I was creating this project as an open source project but not taking it out into the open source community.

Anybody see a small disconnect there?

So, I have started a project (called saltation) on SourceForge and I will be checking in code within 24 hours.

And, I have already run into my first roadblock (or at least a speed bump).

Specifically, it is the build system. I use a common build framework for all of my projects which I call appropriately enough, common-build. It uses Ant and Ivy for the build and dependency management. For a little more information on that see this posting in my other blog : Ant versus Maven.

Both Maven and Ivy use an external repository for storing dependencies such as Java libraries. SourceForge still hasn't worked out all the ways to interact with this. so the bottom line is one of my first tasks is going to be making the build system require a minimum of setup (. I.e. you should be able to build right out of the box).

That will probably take me a day or two counting testing.

Till then,

- Jim

Technology choices revisited

In one of my earlier posts Technology Choices I had looked at what user interface toolkit I should be using. What I've discovered in the intervening time is that this application is going to put a premium on flexibility in presentation. As a result, I am seriously reconsidering the user interface toolkit. The Eclipse RCP is a very powerful platform upon which to build an application. Unfortunately it does have a very strong set of metaphors upon which it is built (Views and Workspaces) which seem too rigid for what I am attempting.

So I am seriously looking at using QT with the Java bindings.

Any thoughts?

Going to and fro

The other part of the last few months has been looking at importing and exporting and synchronizing.

Spent a great deal of time looking at the iCalendar specification and all of the recurrence rules. And then I read all of the use cases for recurrence rules interoperability that have been put together by some incredibly diligent groups of people. The people that work on those specs definitely come under my heading of unsung heroes.

As often appears to be the case in the area of calendaring, there are really no clean answers. And, going back and forth between the icalender format and a more flexible internal model is going to require a great deal of detailed work. But, it is very clear that iCalendar is the de facto standard for now and I see nothing on the horizon yet that will replace it.

And personally, I think it gives the biggest bang for the buck. If I can support CalDAV and ICalendar to any significant degree I will be able to publish calendars to and from Google Calendar , as well as many others applications.

So my immediate attention is to support import/export for ICalendar , and then support using CalDAV to publish.

- Jim

When things don't fit into a neatly modeled the world

I am back after a long hiatus. A combination of work plus a course I'm teaching has kept me very busy.

But always in the background I am thinking about the PIM. That's how we referred to it in my household "The PIM". I would wonder if I'm a little obsessed except I'm having so much fun.

I've been coding some commandline utilities to test the data model and, as I expected, I have run into the limitations of my original model. I had originally started with calendars, and agendas and so on being first class objects. What I mean by that is that they have some distinct existence in the real world , and that is why I am modeling them. What I have discovered is that calendars, address books, task lists, journals and so on are all simply semantically themed collections of stuff we care about.

There are few assertions I can make about what a calendar is that are universal enough to be easily recognizable to everyone.

So I stepped back and thought about it for a bit and this is what I have come up with: we have an ocean of first-class objects that we would like to track (incense, tasks, promises, contact information, little notes) and so on. And then we spend our time trying to organize them into multiple collections they give us easy access to what we want to do when we want to do.

So I am experimenting with having any of these collections simply be taxonomies. Just like described in an earlier post. This would mean that many calendars have a relationship to each other. An example would be the calendar I maintain for my children's schedule all together which could have two related calendars ( one for Aidan and one for Trent ) . both Trent and Aidan's calendar would refer to events in the Boy Scout calendar.

The same would be true of address books. I could have an address book that references my friends and a related address book that only has the subset of my college friends.

The same appears to work well for journals, agendas, sets of conversations, and the collection of resources and notes that I refer to as Data Mines or just Mines.

Domain Model: Conversations

I spend many hours each week tracking conversations. Not the conversations like "hi how are you ....I am fine....", I am talking about conversations that call for actions. They can be actions on my part or actions that i am requesting of others. ..... Most of those conversations have the same pattern centered around making a request, or making a promise and tracking its completion. 

Ideally I would like to track the state of these conversations and many of these conversations occur in email and some occur in IM so any tool I write for managing conversations would reflect that common pattern.

For the academics out there the known model of interaction that encompasses my day to day conversations is based on something called "Speech Action" or "Language Action Perspective". A commercial tool (The Coordinator) has been built using it and seems to have a great deal to recommend it.  The basic ideas behind the Language/Action Perspective:

All human results are achieved through conversations (typically written and spoken)that invoke actions.  Most of us don't need to track conversations that don't lead to actions. Those conversations typically involve exchanges of information  .... 

 

Conversations that lead to actions usually require being tracked....and are effectively a dance...

Those conversations start with either a request for an action or an offer of an action. They can also start off with a "What If" type of conversation such as "Wouldn't it be cool if" and then result in requests or offers of action.

People can accept a request, decline a request, or make a counter offer).

Accepting a request typically involves making a promise. The promise can be completed or revoked.

At some point the conversation for action is complete.

So a tool for  tracking and managing  conversations would allow the following...

1. Opening a Conversation

1a.Generating a Request OR 
1b. Generating an Offer OR
1c. Starting a conversation for a Possibility.

2. Responding 

2a. Accept a request OR

2b. Decline a request OR

2c. Make a counteroffer to a request OR

2d. Comment on or annotate the conversation  OR

2e. Acknowledge someone

2f. Putting off a decision ( i.e. giving a promise by when you will respond)

2g. Make an interim report

2e. Report that the conversation and actions are complete

3. Track conversations (and their participants progress) by their status:

New conversations 

    Conversations missing my response

    Conversations missing other's response

My Promises 

My Requests

Commitments

This may seem incredibly structured , or even artificial, but if you look at the conversations that go into causing projects you will see that they all fall into this structure. Of course, most users would prefer not to use this format for all conversations (or even most of them).  So this should be something optional, where you can did note an e-mail thread as a conversation for action and then track the status of the conversation from that point forward.

What activities should a PIM support

When I first started this blog I naïvely thought that because I have been thinking about this for so long that I had it all firmly clear in my head, and it would pretty much come out in some sensible order.  

Instead I have discovered that along the way I will end up going back and documenting something that seems so self-evident that I have forgotten about it completely.

One example of that is really documenting what are the activities I expect to be able to do with the PIM. This is implicit in a lot of the previous posts, but it is definitely worth making it explicit here.

Basic activities that I expect a user to be able to do include the following:

Capturing and evaluating input: Being able to triage and classify events, tasks, contact information, and other references and notes as they come in. Being able to quickly decide what needs to be handled immediately, what can wait, and if something needs to be scheduled immediately the user should be able to do that quickly even if it means classifying it or scheduling it approximately. "oh look, a dinner invitation, I don't even need to look at it until next week,  so I will throw it into next week.". Being able to throw a note down into the system to be converted to an action or a task later.

Organizing: Going beyond the initial triage to the point where tasks and events are being scheduled in time (or at least the order is being set), as well as coordinating with others. Being able to overlay your own taxonomy/ontology/organizational structure onto the raw objects in the domain ( events, tasks, promises, actions, contacts, etc.). Being able to extend the raw objects of the domain so that they can be personalized to carry the information that the user needs.

Collaborating: Not only sharing calendars, but tracking conversations associated with shared activities.    Tracking the status and progress of projects from organizing your grandmothers birthday party all the way to coordinating the training of a large group of people.

Squirreling things away so the nuts can be found again:  one of the biggest battles for users of a PIM  is storing the data (documents and pictures and other information) associated with an event, a task,a person, or a group in such a way that it can be easily found when needed. 

Finding: Looking through information that you stored off to the side with the intention of finding it again.   Being able to reconstruct timelines based on e-mail conversations and scheduled events. 

Presenting: Visualizing and reporting on the data in a way that is natural for the user. Being able to generate reports on the status of projects from scheduling birthdays and tracking who is coming all the way to larger projects like the implementation of a training regimen for a group of trainers.

Playing with the data model

Hello folks, I am back.

I have been working on the PIM for a bit, but without the space to actually blog about it.

I started experimenting with actually code. I have been working on the basic data model classes and how well they model real-world situations. I am not yet at the point of publishing the code, but I'm starting to get to the point where I can actually think about it.

One of the things I've been working on was trying out the java persistence technology Known as DB 4O. 

My intention has been to find a database storage technology that handled hierarchical data well. Relational databases manage many situations very well but even the best ORM (Object Relational Mapping) solution tends to break down when handling deep, flexible and higly modifiable hierarchies.

In order to handle the kinds of events and extensibility that I am looking for.  I need to be able to store and quickly retrieve objects that are referenced from hierarchies/taxonomies where the objects themselvesstore general attributes.

To give you an example I want to be able to generate taxonomies that have a fairly deep structure ( friends-> college friends->the guys-> the wedding party) and be able to walk that taxonomy and quickly collect all the elements it points to.  Then, if necessary, I would like to be able to rearrange the hierarchy.  This is doable with a relational database and an ORM but no matter how carefully you balance the design, and inevitably there will be performance corner cases that are drastically inconsistent with the performance of the rest of the system. 

In addition, I want users to be able tocreate their own classes of events and tasks. For example, I would like users to be able to create specific events and add their own attributes. if a user would like to distinguish between kids birthday parties and general birthday parties, they should be able to simply create a kid's birthday party event with customized properties and then be able to search within those customized properties.

Generic attributes such as this often present headaches for ORM frameworks, though not nearly as bad as managing hierarchical data.

So I was pretty sold on using a decent object storage framework.  If I could find one! 

DB4O has pretty much answered my requirements.

Through a mixture of byte code instrumentation and good design the DB40 folks have achieved a minimally intrusive persistence framework that effectively speeds up development, unit testing, and general maintainability of the code.

To give you a code example:



// Open the DB

ObjectContainer db;

Configuration configuration = Db4o.newConfiguration();

configuration.generateUUIDs(ConfigScope.GLOBALLY);
configuration.generateVersionNumbers(ConfigScope.GLOBALLY);

db = Db4o.openFile( configuration, filename);

//
// Create a bunch of objects (no matter how deep the hierarchy
//

List items = methodThatCreatesAllTheObjects();

// Store the objects

db.store(items);

// Close the transaction

db.commit();



As you can see there is a minimum of explicit code by default. So I don't have to explicitly open a transaction, that is done when I attempt to store the items. 

There are of course, ways to explicitly do all of the actions necessary, but so far I haven't needed to go outside of the defaults.