Camel.FolderSummary disksummary branch

This is the summary part of the 'disksummary' name of the branch. This is an abstract class which provides altered interfaces which map more efficiently to disk-based records, among other things.

Array and index based interfaces are replaced with iterator and search based interfaces, mem-pool based objects are replaced with isolated allocations, etc.

In addition to these changes, all of the folder change events are handled by this class explictly, to reduce duplication. The change events themselvs are made more useful as well. This document will try to explain the new and changed interfaces.

Base object

The base class now no longer directly contains the various status information about the folder. These are instead based on the views, including the root_view which is the base view.

In addition, several supporting classes are always required; Evolution/CamelDS.ViewSummary, and Evolution/CamelDS.View (via #Camel.FolderView.

CamelFolderSummary now has tigher binding with CamelFolder, neither can exist without the other. In effect it's a way to extend the functionality of the CamelFolder class without cluttering it up too much.

 struct _CamelFolderSummary {
        CamelObject parent;
        struct _CamelFolderSummaryPrivate *priv;
        struct _CamelViewSummary *view_summary;
        struct _CamelFolder *folder;
        int frozen;
        struct _CamelFolderSearch *search;
        struct _CamelFolderView *root_view;
        EDList views;
 CamelFolderSummary *camel_folder_summary_new(struct _CamelFolder *folder, struct _CamelViewSummary *);
  • view_summary: The view-summary object which manages all of the views. It exists as a context for managing multiple-folder data in a way which may be shared between folders and stores.
  • folder: The folder this summary belongs to. Must be set. Used primarily for events without needing to proxy them.
  • frozen: Event emission can be frozen.
  • search: An object used to build and execute searches.
  • root_view: The root view contains all messages. A root view MUST always exist, and be initialised by the implementaiton.
  • views: A list of all other views. The root view is not contained in this list.

Base class

The class itself consists of many virtual methods and some important data.

 struct _CamelFolderSummaryClass {
        CamelObjectClass parent_class;
        size_t messageinfo_sizeof;
        size_t folderview_sizeof;
        GCompareDataFunc uid_cmp;
        GCompareDataFunc info_cmp;
        .. virtual methods ..
  • messageinfo_sizeof, folderview_sizeof: The byte-size of each of the corresponding objects. These are used to allocate instances of these objects, using g_malloc. The previous implementation used EMemChunk.

  • uid_cmp, info_cmp: Functions which will compare a uid and a uid. These are used to sort lists of UIDs and MUST produce stable results (UIDs will always be unique so this shouldn't be hard). The data argument passed will be the CamelFolderSummary pointer. See the section on #message ordering for more information.

Apart from these changes, all of the content-info related functions were removed, view related functions added, and a few other cleanups to improve consistency.


Since the folder-summary emits events for each view, it makes sense to be able to freeze them, for efficiency and to simplify the code (for various reasons, the non-root view events cannot be easily frozen at the folder level). It is up to the implementation to check the frozen state if it is going to emit an event on a view. If the summary is frozen and subsequently thawed, the thawed virtual method will be invoked at thawing time, the default implementation of which emits all of the changed events for every view.

 void camel_folder_summary_freeze(CamelFolderSummary *);
 void camel_folder_summary_thaw(CamelFolderSummary *);
 int camel_folder_summary_frozen(CamelFolderSummary *);

The folder name might change, and the foldersummary has some references to the name, stored for example in view vids. This virtual method lets

 int camel_folder_summary_rename(CamelFolderSummary *, const char *newname);

The main summary interfaces are somewhat simpler. Add, remove and get. Add and remove will automatically create folder_changed events on the root view, which is different from the previous implementation.

 int camel_folder_summary_add(CamelFolderSummary *summary, void *info);
 int camel_folder_summary_remove(CamelFolderSummary *summary, void *info);
 void *camel_folder_summary_get(CamelFolderSummary *, const char *uid);

There are also array equivalents - these have default implementations that just call the above functions. How useful these really are is debatable, it may make sense to ditch them to simplify the api.

 int camel_folder_summary_add_array(CamelFolderSummary *summary, GPtrArray *infos);
 int camel_folder_summary_remove_array(CamelFolderSummary *summary, GPtrArray *infos);
 GPtrArray *camel_folder_summary_get_array(CamelFolderSummary *, const GPtrArray *uids);
 void camel_folder_summary_free_array(CamelFolderSummary *summary, GPtrArray *array);

Then comes the search interface. This is the only interface that can be used to iterate through all or some subset of the messages. It returns a CamelIterator, and can be used to iterator ovew a specific view, or even match specific messages in a specific view. Or even range over a specific subset of messages (although this isn't implement fully yet).

 CamelIterator *camel_folder_summary_search(CamelFolderSummary *summary, const char *viewid, const char *expr, CamelIterator *subset, CamelException *ex);

And a global clear function. not fully implemented

 void camel_folder_summary_clear(CamelFolderSummary *summary);


The CamelMessageInfo is basically the same as the one in Evolution/Camel.FolderSummary#Camel.MessageInfo, although the content-info information has been completely removed from the CamelMessageInfoBase implementation.

One additional flag exists, CAMEL_MESSAGE_RECENT which indicates the message has been newly seen.


The methods have been streamlined slightly or filled out for consistency, although most are the same as before.

The main difference are the factory methods. No longer are there separate 'summary' and 'non-summary' versions, they are all just 'summary' versions, but with the simpler 'non-summary' names. new_from_message now takes a CamelMessageInfo which supplies it with the flags, which are not stored on messages.

 void *camel_message_info_new(CamelFolderSummary *summary);
 void *camel_message_info_new_from_header(CamelFolderSummary *summary, struct _camel_header_raw *header);
 void *camel_message_info_new_from_parser(CamelFolderSummary *summary, struct _CamelMimeParser *parser);
 void *camel_message_info_new_from_message(CamelFolderSummary *summary, struct _CamelMimeMessage *message, const CamelMessageInfo *base);
 void *camel_message_info_clone(const void *info);
 void camel_message_info_ref(void *info);
 void camel_message_info_free(void *info);

The accessor methods are the same.


The CamelFolderChangeInfo structure from Evolution/Camel.Folder has been moved here, and renamed to CamelChangeInfo. It is still however, seen to be emitted on the folder via the folder_changed event.

It has been simplified slightly; now the changes are listed as CamelMessageInfo pointers directly, not just UID strings - which almost invariably had to be looked up and converted to CamelMessageInfo objects anyway. The recent list has also been removed - now recent is a flag on the message.

 struct _CamelChangeInfo {
        char *vid;
        GPtrArray *added;
        GPtrArray *removed;
        GPtrArray *changed;
        /* private */
        GHashTable *uid_stored;

The interfaces have likewise been streamlined and unused functions discarded:

 CamelChangeInfo *camel_change_info_new(const char *vid);
 CamelChangeInfo *camel_change_info_clone(CamelChangeInfo *info);
 void camel_change_info_clear(CamelChangeInfo *info);
 void camel_change_info_free(CamelChangeInfo *info);
 gboolean camel_change_info_changed(CamelChangeInfo *info);
 void camel_change_info_cat(CamelChangeInfo *info, CamelChangeInfo *src);
 void camel_change_info_add(CamelChangeInfo *info, const CamelMessageInfo *);
 void camel_change_info_remove(CamelChangeInfo *info, const CamelMessageInfo *);
 void camel_change_info_change(CamelChangeInfo *info, const CamelMessageInfo *);

The shorter name is somewhat more pleasant to type too.


A view is like an alternative base index for a folder. It will contain a strict sub-set of messages from the root-view.

 struct _CamelFolderView {
        struct _CamelFolderView *next;
        struct _CamelFolderView *prev;
        guint32 refcount:31;
        guint32 is_static:1;
        char *vid;
        struct _CamelFolderSummary *summary;
        struct _CamelView *view;
        struct _CamelFolderSearchIterator *iter;
        CamelChangeInfo *changes;

The vid is the view id itself. The root view will have a vid of NULL. This is a copy of the same information from the Evolution/CamelDS.View, although it has the folder name removed.

Note that each view maintains its own changes list; this can be used by a view when building change lists, and to store change events while they are frozen, on a per-view basis.

The iter object provides a pre-parsed expression that is used in a one-shot mode to calculate membership for the view for non-root views - although an implementation may have an alternative mechanism for this (e.g. an SQL VIEW). See Evolution/CamelDS.FolderSearch.

By default, in addition to the root view, each provider may setup two basic additional views, a view id of ".#trash" for deleted messages, and ".#junk" for junked messages.

There are methods which must be used to create and manage the views on a summary. How these work and interact with the various virtual methods is a little tricky - you have to check the source for details.

 CamelFolderView *camel_folder_view_new(CamelFolderSummary *s, struct _CamelView *);
 CamelFolderView *camel_folder_view_get(CamelFolderSummary *s, const char *vid);
 void camel_folder_view_unref(CamelFolderView *v);
 void camel_folder_view_add(CamelFolderSummary *s, CamelFolderView *, CamelException *ex);
 void camel_folder_view_remove(CamelFolderSummary *s, CamelFolderView *);

This function will basically call new and add, if the view doesn't already exist with the same expression, etc.

 const CamelFolderView *camel_folder_view_create(CamelFolderSummary *s, const char *vid, const char *expr, CamelException *ex);

Message ordering

In the original Evolution/Camel.FolderSummary, messages are ordered based on their arrival time, or they are only ever appended to the end of the summary array.

This has changed - now they are ordered based on the uid_cmp function. Where UIDs are unconditionally increasing integers the result will be the same.

This is an important requirement as it allows some very efficient set operations to be implemented using incremental algorithms. Incremental algorithms translate much better to disk-based record storage.

Message Iterators

Message iteratos are based on the new iterator interfaces in Camel.Object.

need to explain the basic iterator interface??

Each call to an iterator will return a CamelMessageInfo, until there are no more left, in which case it will return NULL. This pointer will remain valid until the next iterator function call; it must be referenced if the pointer is to be saved. This simplifies most working loops where you need to just query the CamelMessageInfo and then discard it.

Message iterators MUST conform to the strict #message ordering requirements outlined above; the various set operations can then be implemented based on this guarantee.

Factory methods

There are a couple of helper factory methods for creating appropriate messgae iterators for small sets of in-memory messages. Normally an implementation would need to have a backend-specific iterator implementation.

 void *camel_message_iterator_infos_new(GPtrArray *mis, int freeit);
 void *camel_message_iterator_uids_new(CamelFolder *source, GPtrArray *uids, int freeit);

In each case, the message uids or messageinfo's must already be in the correct message order. freeit means the array and contents should be unallocated once it is finished with.


Indexing was removed to make the implementation easier, and it hasn't been replaced. The need for indexing to speed up vFolders isn't as great anymore since search results are persistent. However indexing would still aid one-off user searches.

This object is mostly complete, although the view interfaces are a little messy. Apart from those, everything has been streamlined and reduced as much as possible, removing all of the duplicate functions performing the same tasks. This extends to Evolution/CamelDS.Folder, where any message status functions have been removed since they are entirely accessible through the #Camel.MessageInfo interfaces.

Apps/Evolution/CamelDS.FolderSummary (last edited 2013-08-08 22:50:05 by WilliamJonMcCann)