THe postings here have moved.

All of the postings here have moved over to my other blog: http://biggerwrench.blogspot.com/

PIM Manfesto

Personal Information Manager Manifesto

A manifesto is a declaration of principles. It may sound grandiose, but this declaration of principles is important to me because it is the distillation of at least 7 years of thinking about what the future I want to create really is.

Principal: The PIM is an expression of how people live their lives.

The things users talk about in real life correspond to the things they manipulate in their PIM. For example if a person thinks in terms of anniversaries, holidays, meetings than those are the things they schedule rather than something called an event.

The actions users talk about in real life correspond to the actions they take in their PIM. If they think in terms of making and keeping promises than the PIM makes it possible to make and keep promises.

Principle: The PIM has a very minimalist design.

It displays what the user needs when it is needed rather than overwhelming them with all the information they might need.

Principal: The PIM is designed to use and make available connections between the things the user deals with.

The PIM makes available the connections between the data when the user needs them. It is easy for the user to lay out the communities in their lives the way they think of them. Adding and subtracting people from a community should be easy.

Principal: The PIM allows users to recognize what is available from where they
are rather than remembering.

Having the user be able to get the information they need when they need it and recognize its availability rather than remember where it would be.

I will probably add to or simplify this over time but I wanted to get it down in writing and out there.

Going Mobile

For years now I have had a love-hate relationship with mobile PDAs. My first Palm Pilot (I vaguely remember that it was sometime in the 90s) was a revelation! It was simple and straightforward and while it didn't do huge amounts; what it did, it did well.

Unfortunately, as my scheduling needs grew, the Palm's did not. Multiple calendars were just not in the cards and synchronization via WiFi never seemed to work out.

Eventually, I gave up on the Palm devices and my wife gifted me with a Dell Axim. And I started a different love-hate relationship. Well, mostly hate. It is a Microsoft-based device after all. This meant I dealt with crashes and the necessity of clicking many things many times to get to where I wanted to go.

And then there are the Dell specific gotchas. I especially like the fact that even while in the case it is easy for the Axim's sound record button to be bumped and for the device to record a nice long audio recording that fills up the disk space.

Overall, it really wasn't that much better a device then the Palm devices. Though it does have the cool Bubble Breaker game.

And then I tried the many different ways to sync up to a single repository of calendar and other information. To mention a few: GooSync, MultiSync, the various SyncML -based solutions like ScheduleWorld. During this period I spent more time recovering from synchronization failures of many different types than I did doing actual work.

Until recently when Google released a Windows Mobile OS 5.0 application that allows me to sync between my Google Calendar and the Axim. That, in combination with MilkSync, for syncing with Remember The Milk has allowed me to achieve a basic minimum level of keeping my mobile calendar and my desktop calendar in sync.

So what does that leave me with as a solution? I have my Dell Axim, which allows me to capture information while I'm on the go including scheduling information and sync it up with my desktop calendar. Of course, notes and tasks are not synced up by default and need special handling. I have my cell phone, which gets notifications from Google Calendar so I have reminders of what needs to be done without depending on the Axim's erratic notification methods. And of course, I have the Google calendar.

It works, but it is hardly ideal.

Of course, being an incredible optimist, I have hopes that the new android-based devices will provide for a more total solution. But I have seen very little in the ways of reviews of their PIM capabilities.

How many objects do we need?

One of the interesting, and potentially invaluable concepts that the Chandler project brought to light (i.e. to my attention ) was the idea of "stamping". The concept of stamping is the idea of defining incoming data by assigning it a type. When someone is simply typing in some information quickly, it by default goes into a note and then allows the user to stamp it as an event, or a task, or whatever else is needed.

A typical (though narrow) implementation of this concept is in the ability to take an e-mail that is an invitation and have the information be put into the calendar at the appropriate time. Microsoft Outlook allows e-mail's to be scheduled this way.

As far as I can tell, the Chandler project does this by displaying different facets of information in different contexts. This implies that the information is stored and "stamped" with several different types.

For example: someone may send you an evite to an event. You respond to the evite but keep the e-mail around as a reminder. If you liked the restaurant that the event occurred at you may decide to keep a note about the fact that you like that restaurant and you may also want to put the address of that restaurant into your contact list. In the Chandler world this is done by stamping the same piece of information twice. The first time it is stamped with the type "note" and the second time with the type "contact".

My apologies in advance if it turns out that I am misinterpreting the design notes for Chandler.

I think, the same thing could be achieved by creating new objects from the initial object and then maintaining associations between them. This would allow someone to create an event from an e-mail and the event would have a reference back to the original e-mail.

Incomplete thoughts on displaying PIM data

Modeling calendars, agendas, projects and so on as views into the data.

Examples:

A calendar could be considered a view into a timestream populated with events and tasks and anything else that can be located in time allows organizing chronological events in terms of days, weeks, months, years, etc..

A timeline could be considered a view into a timestream populated with the events, tasks and anything else that can be located in time.

A timetable could be considered a view into a timestream populated with the events, tasks and anything else that can be located in time in a tabular form.

The daily agenda could be considered a view into a timestream populated with events and tasks and anything else that can be located in time.

A Gantt chart could be considered a view into a set of actions that can be tracked and have a due date. A to do list could be considered another view into a set of actions that can be tracked and have a due date.

A contact list could be considered a view into a set of information about people, organizations, companies and so on.

So far, so good.

Taking on that there is a view into a otherwise undifferentiated set of data has the potential to give great results. That is effectively what ECCO allowed you to do on a limited scale.

And it seems to me that there are two parts to this. There are the constraints or criteria that gives the list of items to be displayed such as show me all the items that can be located in time and whose date occurs somewhere within the next month. And some of the criteria could be tags and or "all items referenced by the following taxonomy..."

So that gives us some common views that may be of use:

1. For sets of items that can be located in time => calendars, schedules
2. For sets of items that can be tracked and/or have a due date there are projects, to do lists, agendas, checklists.
3. For sets of items that are contact information of various types there are contact lists, mailing lists, and directories.
4. For communications and messages there are threaded conversations.
5. For events that have happened in the past there are journals and audit logs.

So if we now look at how those views could be created we start having to pull together all the previous notes and discussions.

Say for example we needed to create a mailing list for a specific community such

as the extended family. Of course the criteria would be something like: please

display contacts referenced by the "family" tag/taxonomy.

Context is decisive

Context is decisive. What ever context you have for piece of information governs

your understanding of that piece of information. So context is decisive.

I have had thoughts about context going around and around in the back of my head

for days now. And since they are getting in the way of my other work I am

putting them into the blog in the hopes that they will leave me alone.

One of the most common requests I have bumped into is for the PIM to take into

account the context in which you are running the application. So, in other

words, if you are at work it would only show you by default the tasks and events

pertinent to you being at work. If you are on the road it would only show you

those items that are pertinent to you being on the road.

The GTD methodology recommends setting up tasks list with tags such as @phone

and @office so that when you are on the road you can simply list those tasks

that you have the resources to perform. Is this something that makes sense to

model in a more sophisticated manner.

In other words, what does knowing the context make available in terms of work?

Should it be modeled and if so how?

It was Col. Mustard in the library with the candlestick

One of the big questions in any enterprise development project is who gets to muck with the database or data repository.

This is significant even in the case of the PIM.

When looking at how to deal with pushing data out to other repositories such as Google, polling data from other repositories such as Google or RSS feeds, and just the general headaches of synchronizing to and from other devices is clear that who gets to touch the database and how is a critical question.

For now, I am assuming that all of the tools that feed to and from other data sinks/sources will do so by operating against the database rather than having the PIM up and running and managing those operations.

Kitchen sink modeling

The more I work with user stories and scenarios for PIMs the more it is becoming clear that there is simply a sea of information that only the user has any sense of. Most of the things that we are modeling have specific meanings to the individual (i.e. must a promise have a due date?). For some people the answer is yes for others the answer is no.

It is clear that for most people there are clear distinctions between the different types of data they have. In other words most people have a consistent way they mentally model meetings versus tasks versus holidays. And the way they model it is quite closely tied to the way they work. So people work and live in a world of pieces of data that have clear types and clear behaviors.

The collections (i.e. projects, agendas, to do lists, checklists, etc..) people use to manage those different types of data ( i.e. to do list items, tasks, promises, calls, errands, chores, etc..) are just as individualized as the data items themselves.

For example: For some people a task may be committed to to the extent that somebody has said "I will do that sometime next week". And they put a sticky note in their calendar so that when they are looking at that week they know what things they promised to do that they haven't put into "space and time" yet.

Other people maintain tasks on lists that don't count them as real until they are scheduled on a calendar. Until that point they are on the "Not Doing Now" or "Unscheduled" list.

So clearly, the majority of everything to be modeled needs to be customizable. so, after walking through user story after user story this is what I see:

1) Every thing to be tracked in a PIM has one or more behaviors.

The behaviors are

• Locatable in time (LIT)
• Can Occupy time (0T)
• Locatable in space (LIS)
• Has a lifespan (SPN)
• Has trackable progress (TRK)
• Has a due date (DUE)
• Requires resources (RR)

The different types of items can have the behavior or not have the behavior. If they do have the behavior then they can either have the state of having a fixed value or not having a fixed value.

For example, a promise can have the behavior of being locatable in time, but it may not yet have been fixed in time.

2) Everything is versionable.

There are many areas where tracking the changes to a scheduled item (who generated them and when) is critical to fixing something. This is specially true when you're dealing with synchronizing schedules with multiple calendars.

3) All items tracked in the PIM are associated with a specific type and that type has associated properties and each type has default values for those properties and behaviors.

For example, an event can have the behavior of being locatable in time and may have the behavior of occupying time and the default time it occupies is 15 minutes.

4) Context is decisive: context such as home -- online or home -- off-line or work -- online etc. have resources available such as (phone, computer, Internet, e-mail, etc.)

Many methodologies such as GTD take into account the context in which you were working. For example there are some things that should only be done from home, there are some things that should only be done from work, and there are some things that can only be done when you have a phone available. The user's context governs both what they should be doing as well as what they are capable of doing given the resources available.

5) Taxonomies (hierarchies) go from wider towards narrower ( i.e. Extended Family => Immediate Family)

In looking at all the different ways people navigate through hierarchies and the way they create their own private taxonomies (using file folders, categories, tags, etc.) it is clear that most people navigate from wider and narrower.

Upgrading my existing PIMs and Systems: Take 2

As I said last week, Chandler appeared to be a bust.

Over the weekend, I tried out using Thunderbird 2.0 and Lightning and a few plug-ins.

It is definitely far more stable than Chandler. Installation was a breeze. The Thunderbird functionality was rock solid. The Lightning integration was a little less so. Many of my Google calendars did not display all of the events.

I then went on to try Thunderbird 3.0 beta and the corresponding experimental Lightning version. Integration is a little cleaner, some Google events still did not display (though not the same ones) and the application did crash under the 64-bit version of Ubuntu (Jaunty Jackelope) I am running.

So I ended up back using Evolution as my primary PIM. But I did make a few changes in how I used it.

I change the system so that my two Gmail accounts are accessed using IMAP. My Comcast account is accessed using POP , and I am experimenting with using rules tailored for each of the three accounts.

In addition, I have switched over to using "Remember The Milk." (aka RTM) as my primary task manager. Google tasks appears to be of no real use for me since I can't sync my tasks to anything useful.

I use Tasque ( Linux only) to edit and modify my task entries and I use Evolution's read only access of RTM to see the tasks on the fly. It is far from ideal. Far, far, far from ideal. But it is a step up. When Evolution has the ability to access Google Tasks then I may be able to switch to one online provider. Of course, for that to happen Google needs to publish an API that allows us to access the tasks.

- Jim

Upgrading my existing PIMs and Systems

Periodically, I reviewed the existing PIM solutions that are out there in the hopes of finding something that will please to handle my intermediate needs while I am writing something to handle all of my needs.

I am currently using Evolution in connection with Google calendar and sinking evolution to a Palm pilot. In my review. I discovered that Chandler is now at a greater than 1.0 version. Yay. Sort of.

I have often commented in this blog about how effectively the Chandler Project has modeled the domain of task management and scheduling. They have some brilliant concepts and have nailed many of the basic issues needed to take things to another level.

Unfortunately, stability is not one of them. I tried pulling down and installing Chandler on four different machines/OSes. Windows XP/Vista/Ubuntu 8 and Ubuntu 9.

Every single one of them crashed multiple times during normal operations. Whether it was importing from Google ics files, to creating a new collection, or creating a new calendar. Crashes were the rule rather than the exception.

This was also my experience over a year ago.

I don't know what else to say, but "Damn"

Over the weekend . I will look at both Thunderbird 2.0 and its corresponding Lightning plug-in as well as the Thunderbird 3.0 beta and its corresponding Lightning plug-in.

- Jim

Model Mismatch

Interoperability is a stone cold *expletive deleted*. When writing a PIM or any other calendar app, you will pick wrong. Just get used to it.

Here is what I mean:

In the calendar world their is only one viable standard for the exchanged info : the icalendar file format. It can be exchanged in multiple ways, though the
CalDAV protocol (Sits on Top of WebDAV which sits on HTTP) is becoming the exchange method of choice.

But whos should we use ? Lots of applications support it. But Microsoft's ICalendar format is not always in sync with Google's and so on AND it is not even anyone's fault. The areas where things fall down are often where the spec itself has issues such as recurring events and events that last all day.

When does an all day event start ? 00:00:01 or 00:00:00 ?

When does an all day event end ? 23:59:59 or 24:00:00 (oh wait, thats 00:00:00) oops.

Those questions just begin to touch on the issues.

Whichever way you handle all day events you will have to do accurate conversions to the other way or risk all sorts of "round trip translation errors". If you have synchronized your PIM with another and back again you have probably already run into this....

There are conferences and technical groups that spend days just battling with how to specify and manage recurring events. Look for "IIOP Recurring Events"

Most PIMs solve this by closely adhering to the icalendar specification for their domain model. Thats great where the spec really works but their are some key things that don''t appear to be covered or are incompletely specified. i.e. Tagging and taxonomies, the same events in multiple calendars, hierarchical tasks and events, and so on. I am not yet an expert on icalendar but I will become solidly familiar with it over the next few weeks. Luckily icalendar provides an extension mechanism so that the application specific information can be captured. Of course, other applications won't use the info and some of them won't even preserve it (Google's icalendar support seems to remove tasks from the calenar).

My project for next weekend is to try and do a full round trip between Google calendar and the Saltation domain model using Google's Calendar API and maybe using the icalendar file format.

6.21.09 I wrote this post on 6.19.09 and then posted it today. As I go back and look at what Google actually supports I am stunned. To try and push through the Google API looks like a decent amount of work for little initial return. I think I am going to work first on FULL icalendar file format support (import and export) next weekend and then look at publishing and reading from Google directly. At least debugging the process will be simpler.

Progress

I just realized that naming the PIM and the blog the same thing may have been less than optimal. Ah well.

I did a quick clean up of my build scripts for saltation.

I am still wrestling with how to present the information on SourceForge in a way that works. The gods know that I have getting annoyed often enough with OSS projects and navigating around trying to figure out what is where. I will spend some
serious time and thought to laying things out in away that works.

Taking things public

About two weeks ago I realized that I was creating this project as an open source project but not taking it out into the open source community.

Anybody see a small disconnect there?

So, I have started a project (called saltation) on SourceForge and I will be checking in code within 24 hours.

And, I have already run into my first roadblock (or at least a speed bump).

Specifically, it is the build system. I use a common build framework for all of my projects which I call appropriately enough, common-build. It uses Ant and Ivy for the build and dependency management. For a little more information on that see this posting in my other blog : Ant versus Maven.

Both Maven and Ivy use an external repository for storing dependencies such as Java libraries. SourceForge still hasn't worked out all the ways to interact with this. so the bottom line is one of my first tasks is going to be making the build system require a minimum of setup (. I.e. you should be able to build right out of the box).

That will probably take me a day or two counting testing.

Till then,

- Jim

Technology choices revisited

In one of my earlier posts Technology Choices I had looked at what user interface toolkit I should be using. What I've discovered in the intervening time is that this application is going to put a premium on flexibility in presentation. As a result, I am seriously reconsidering the user interface toolkit. The Eclipse RCP is a very powerful platform upon which to build an application. Unfortunately it does have a very strong set of metaphors upon which it is built (Views and Workspaces) which seem too rigid for what I am attempting.

So I am seriously looking at using QT with the Java bindings.

Any thoughts?

Going to and fro

The other part of the last few months has been looking at importing and exporting and synchronizing.

Spent a great deal of time looking at the iCalendar specification and all of the recurrence rules. And then I read all of the use cases for recurrence rules interoperability that have been put together by some incredibly diligent groups of people. The people that work on those specs definitely come under my heading of unsung heroes.

As often appears to be the case in the area of calendaring, there are really no clean answers. And, going back and forth between the icalender format and a more flexible internal model is going to require a great deal of detailed work. But, it is very clear that iCalendar is the de facto standard for now and I see nothing on the horizon yet that will replace it.

And personally, I think it gives the biggest bang for the buck. If I can support CalDAV and ICalendar to any significant degree I will be able to publish calendars to and from Google Calendar , as well as many others applications.

So my immediate attention is to support import/export for ICalendar , and then support using CalDAV to publish.

- Jim

When things don't fit into a neatly modeled the world

I am back after a long hiatus. A combination of work plus a course I'm teaching has kept me very busy.

But always in the background I am thinking about the PIM. That's how we referred to it in my household "The PIM". I would wonder if I'm a little obsessed except I'm having so much fun.

I've been coding some commandline utilities to test the data model and, as I expected, I have run into the limitations of my original model. I had originally started with calendars, and agendas and so on being first class objects. What I mean by that is that they have some distinct existence in the real world , and that is why I am modeling them. What I have discovered is that calendars, address books, task lists, journals and so on are all simply semantically themed collections of stuff we care about.

There are few assertions I can make about what a calendar is that are universal enough to be easily recognizable to everyone.

So I stepped back and thought about it for a bit and this is what I have come up with: we have an ocean of first-class objects that we would like to track (incense, tasks, promises, contact information, little notes) and so on. And then we spend our time trying to organize them into multiple collections they give us easy access to what we want to do when we want to do.

So I am experimenting with having any of these collections simply be taxonomies. Just like described in an earlier post. This would mean that many calendars have a relationship to each other. An example would be the calendar I maintain for my children's schedule all together which could have two related calendars ( one for Aidan and one for Trent ) . both Trent and Aidan's calendar would refer to events in the Boy Scout calendar.

The same would be true of address books. I could have an address book that references my friends and a related address book that only has the subset of my college friends.

The same appears to work well for journals, agendas, sets of conversations, and the collection of resources and notes that I refer to as Data Mines or just Mines.

Domain Model: Conversations

I spend many hours each week tracking conversations. Not the conversations like "hi how are you ....I am fine....", I am talking about conversations that call for actions. They can be actions on my part or actions that i am requesting of others. ..... Most of those conversations have the same pattern centered around making a request, or making a promise and tracking its completion. 

Ideally I would like to track the state of these conversations and many of these conversations occur in email and some occur in IM so any tool I write for managing conversations would reflect that common pattern.

For the academics out there the known model of interaction that encompasses my day to day conversations is based on something called "Speech Action" or "Language Action Perspective". A commercial tool (The Coordinator) has been built using it and seems to have a great deal to recommend it.  The basic ideas behind the Language/Action Perspective:

All human results are achieved through conversations (typically written and spoken)that invoke actions.  Most of us don't need to track conversations that don't lead to actions. Those conversations typically involve exchanges of information  .... 

 

Conversations that lead to actions usually require being tracked....and are effectively a dance...

Those conversations start with either a request for an action or an offer of an action. They can also start off with a "What If" type of conversation such as "Wouldn't it be cool if" and then result in requests or offers of action.

People can accept a request, decline a request, or make a counter offer).

Accepting a request typically involves making a promise. The promise can be completed or revoked.

At some point the conversation for action is complete.

So a tool for  tracking and managing  conversations would allow the following...

1. Opening a Conversation

1a.Generating a Request OR 
1b. Generating an Offer OR
1c. Starting a conversation for a Possibility.

2. Responding 

2a. Accept a request OR

2b. Decline a request OR

2c. Make a counteroffer to a request OR

2d. Comment on or annotate the conversation  OR

2e. Acknowledge someone

2f. Putting off a decision ( i.e. giving a promise by when you will respond)

2g. Make an interim report

2e. Report that the conversation and actions are complete

3. Track conversations (and their participants progress) by their status:

New conversations 

    Conversations missing my response

    Conversations missing other's response

My Promises 

My Requests

Commitments

This may seem incredibly structured , or even artificial, but if you look at the conversations that go into causing projects you will see that they all fall into this structure. Of course, most users would prefer not to use this format for all conversations (or even most of them).  So this should be something optional, where you can did note an e-mail thread as a conversation for action and then track the status of the conversation from that point forward.

What activities should a PIM support

When I first started this blog I naïvely thought that because I have been thinking about this for so long that I had it all firmly clear in my head, and it would pretty much come out in some sensible order.  

Instead I have discovered that along the way I will end up going back and documenting something that seems so self-evident that I have forgotten about it completely.

One example of that is really documenting what are the activities I expect to be able to do with the PIM. This is implicit in a lot of the previous posts, but it is definitely worth making it explicit here.

Basic activities that I expect a user to be able to do include the following:

Capturing and evaluating input: Being able to triage and classify events, tasks, contact information, and other references and notes as they come in. Being able to quickly decide what needs to be handled immediately, what can wait, and if something needs to be scheduled immediately the user should be able to do that quickly even if it means classifying it or scheduling it approximately. "oh look, a dinner invitation, I don't even need to look at it until next week,  so I will throw it into next week.". Being able to throw a note down into the system to be converted to an action or a task later.

Organizing: Going beyond the initial triage to the point where tasks and events are being scheduled in time (or at least the order is being set), as well as coordinating with others. Being able to overlay your own taxonomy/ontology/organizational structure onto the raw objects in the domain ( events, tasks, promises, actions, contacts, etc.). Being able to extend the raw objects of the domain so that they can be personalized to carry the information that the user needs.

Collaborating: Not only sharing calendars, but tracking conversations associated with shared activities.    Tracking the status and progress of projects from organizing your grandmothers birthday party all the way to coordinating the training of a large group of people.

Squirreling things away so the nuts can be found again:  one of the biggest battles for users of a PIM  is storing the data (documents and pictures and other information) associated with an event, a task,a person, or a group in such a way that it can be easily found when needed. 

Finding: Looking through information that you stored off to the side with the intention of finding it again.   Being able to reconstruct timelines based on e-mail conversations and scheduled events. 

Presenting: Visualizing and reporting on the data in a way that is natural for the user. Being able to generate reports on the status of projects from scheduling birthdays and tracking who is coming all the way to larger projects like the implementation of a training regimen for a group of trainers.

Playing with the data model

Hello folks, I am back.

I have been working on the PIM for a bit, but without the space to actually blog about it.

I started experimenting with actually code. I have been working on the basic data model classes and how well they model real-world situations. I am not yet at the point of publishing the code, but I'm starting to get to the point where I can actually think about it.

One of the things I've been working on was trying out the java persistence technology Known as DB 4O. 

My intention has been to find a database storage technology that handled hierarchical data well. Relational databases manage many situations very well but even the best ORM (Object Relational Mapping) solution tends to break down when handling deep, flexible and higly modifiable hierarchies.

In order to handle the kinds of events and extensibility that I am looking for.  I need to be able to store and quickly retrieve objects that are referenced from hierarchies/taxonomies where the objects themselvesstore general attributes.

To give you an example I want to be able to generate taxonomies that have a fairly deep structure ( friends-> college friends->the guys-> the wedding party) and be able to walk that taxonomy and quickly collect all the elements it points to.  Then, if necessary, I would like to be able to rearrange the hierarchy.  This is doable with a relational database and an ORM but no matter how carefully you balance the design, and inevitably there will be performance corner cases that are drastically inconsistent with the performance of the rest of the system. 

In addition, I want users to be able tocreate their own classes of events and tasks. For example, I would like users to be able to create specific events and add their own attributes. if a user would like to distinguish between kids birthday parties and general birthday parties, they should be able to simply create a kid's birthday party event with customized properties and then be able to search within those customized properties.

Generic attributes such as this often present headaches for ORM frameworks, though not nearly as bad as managing hierarchical data.

So I was pretty sold on using a decent object storage framework.  If I could find one! 

DB4O has pretty much answered my requirements.

Through a mixture of byte code instrumentation and good design the DB40 folks have achieved a minimally intrusive persistence framework that effectively speeds up development, unit testing, and general maintainability of the code.

To give you a code example:



// Open the DB

ObjectContainer db;

Configuration configuration = Db4o.newConfiguration();

configuration.generateUUIDs(ConfigScope.GLOBALLY);
configuration.generateVersionNumbers(ConfigScope.GLOBALLY);

db = Db4o.openFile( configuration, filename);

//
// Create a bunch of objects (no matter how deep the hierarchy
//

List items = methodThatCreatesAllTheObjects();

// Store the objects

db.store(items);

// Close the transaction

db.commit();



As you can see there is a minimum of explicit code by default. So I don't have to explicitly open a transaction, that is done when I attempt to store the items. 

There are of course, ways to explicitly do all of the actions necessary, but so far I haven't needed to go outside of the defaults. 

Metadata and Taxonomies (aka How do I find it my way?)

Some of the very best PIMs allow you to organize your data according to your needs. Usually that involves using tags or categories. And a great deal of discussion has gone into whether tagging or categorization is more effective for people who are tracking large amounts of information.

What is the difference? Tagging is like sewing little labels on your socks, categorizing is like picking the specific drawer to pick each type of sock in. I would suggest that the best overview document is Shirkey's document here. He very effectively details the strengths and weaknesses of both methods. I would also suggest Christian Ricci's discussion for very clean examples of solid uses of Taxonomies.

After reading Shirkey's notes you may be left with the idea that Tags are the "One True and Useful Way". To the extent that he talks about (i.e. the world of the web) that may be very true. But in the world of PIM the many weaknesses of Taxonomies become strengths for an individual user. The first person I found that seems to have firmly grasped that is Paul Chen here. And so what I am aiming for is the use of Taxonomies where the user finds it works for them and tags as necessary.

For now I want to talk about Taxonomies (I will expand on tags in another post). First are some definitions. A Taxonomy is a representation of concepts (called Taxa) that have distinct relationships.

For example a Taxonomy of the communities I track in my PIM would have Work, Church, Family, and Friends. Friends would have a distinct subset of "College Friends" and Family would have related subsets for each family grouping for my immediate family, my wife's family and my ex-wife's family (see graphic above).

A taxonomy encodes the relationships between the concepts. These include:

• is-a-type-of relationships like spaghetti is-a-type-of pasta.
• is-an-instance-of-a relationship like "my spaghetti dinner" is-an-instance-of-a "spaghetti dinner".
• is-part-of relationship like Rome is-a-part-of Italy.
• is-associated-with relationship like Spaghetti is-associated-with Italy.

These relationships could be the key to achieving smooth access to the information you need when you need it. For example, if I had a tree of communities associated with each other such as the picture above then when I went to reply to an email from my brother I could quickly be given the oppurtunity to email it to my immediate family or the next highest group up beyond that. If a PIM supported multiple user defined Taxonomies such as a Community Taxonomy or a Calendar Taxonomy then the user could view each level of the taxonomy using the views that made sense for them.

As an example, a taxonomy could have the following levels and relationships Calendar->Jim, Janice, Boys, Church, Work with additional subgroupings of Church->Board of Trustees, Worship Committee. Any type of object could be associated with any level such as Work but only those objects that have an Aspect of LIT would show up in a calendar view or a schedule view. Only those objects with the aspect of LIS would show up in a map view.

Of course, as a user my preference would be for being able to create and manipulate taxonomies quickly as well as associate searches with Taxa for the ability to create flexible taxonomies.