Super Data Daemon Thing

What is it?

A collaboration between various open source developers to provide a common data access layer for the free desktop.

Scope

Ideally, all useful data will be accessible through this layer. Desktop applications, webservice data and devices (N800, PDAs, Phones, etc). Notifications will be provided as dataproviders become available and go away.

Who is involved?

In alphabetical order:

Projects we would especially like to involve:

Use Cases

• gimmie wants to receive a notification when something changes in a webservice or application
• tom has 3 applications that poll his S3 account, he wishes they all shared a connection so less bandwidth is used
• joe wants to upload his photo using {f-spot, nautilus, eog, cheese, conduit} to {flickr, smugmug, box.net, etc}
• boyd wants Tasky to have multiple backends {rtm, hiveminder, eds}
• john wants to plug in his phone and have his desktop react to it (e.g. automatic sync)

Design

Terms

• We should agree on some
  • Dataprovider, Datasource, Backend
  • Dataitem, Datatype

Always up to date

• For some apps, the design of the getter should be similar to beagle. You ask for data and it gives you everything currently available. It will send you update events as data is added, changed or removed
• In the case of conduit (and opensync) we would only really want changes since the last request...

Caching

• Think caching should be at the dataprovider level, so it works for more than just HTTP based webservices (see bug)

• We can provide a nice cache by setting up sync between the webservice and a local object store (sqlite?) The interface is the same whether the local data is used, or the online data is used. This could even be two-way sync so that offline changes can be pushed back to the webservice the next time NetworkManager reports its availability.

Object Types

• Much like Conduit, we need to establish our 'first class' object types.
  • Photo
  • Media (Video / Music)
  • Contact
  • Event
  • Task
  • Bookmark
  • URI

Dataprovider Availability

• Each dataprovider could be in one of 3 states: AVAILABLE, OFFLINE, UNAVAILABLE.
• Signal for dataprovider becoming available or becoming unavailable.
• Needs a bit of thought :

  • GetAvailableDataproviders() returns availble Dataproviders

  • GetOfflineDataproviders() returns offline Dataproviders

  • GetUnavailbleDataproviders() returns unavailable Dataproviders - probably shouldn't exist. We can't know all dataproviders that aren't available. What if theres a phone that has never been plugged in before? Or do we want UNAVAILABLE to be a "we've seen this thing before and it might come back, but we can't promise anything" state?

  • GetDataproviders() returns all Dataproviders

• BUT, applications might know about a dataprovider that /was/ there and has now gone away - e.g. you unplugged the phone. do something smart in SDDT, or leave this to clients?

Remote Authentication

• The system should store authentication information so that all programs do not require reauthentication
  • However, we need some sort of prompt to user as to avoid a security risk
  • Use keyring?

Configuring Endpoints

• In Conduit, the dataproviders have to be configured. E.g. Which evolution address book do you want sync? Which tags do you want to query on? What is your username? How do we export this GUI desktop wide, especially if we want to be x-desktop.
• Utilize web-login-driver to auto-configure some of the webservice type endpoints?

Implementation

From Scratch

• GOOD: We only have what we need (starting from something else could leave us with project specific bits)
• BAD: We need to start moving - until we have a few working demos we are just vaporware

Fork Conduit

• GOOD: Conduit's dbus service could be close to what we need. Split it out and s/conduit/superdatadaemon/?
• GOOD: Just need to expose the get / put / delete methods and signals on the already exposed data objects and strip out all unneeded deps
  • The DBus part is is a 60 minute job. (define un-needed deps BTW?)
• BAD: Conduit dataproviders do have some sync specific glue in them

  • Like what? Im not sure I follow. I would agree that they have some configuration specific code in them, but all the sync specific (such as checking something exists before you overwrite it) is needed in non sync cases too (unless you want to eat your data by mistake)

• BAD: Conduit dataproviders have a synchronous API
  • Thats because most of the things they deal with only offer synchronous APIs, and its cleaner to hide the synchronous dataprovider APIs behind an asynchronous get_all/get/put wrapper.

Use Conduit

• GOOD & BAD: Has all (dis)advantages of forking conduit (vs from scratch)

• GOOD: Less work for conduit, and others?
• BAD: A scarier prospect for other projects to depend on - people might not want to drag in the sync parts?
  • Lets see how 2.24 proposal goes first.

Merge Gimmie Item representation and Beagle Indexing with Conduit Data retrieval

• Needs some thought

Demo

• Build a random desktop background that can fetch pictures from {f-spot, flickr, smugmug, etc} without needing to know anything about how/where each service stores its data

Other

• This would be something awesome for a GSOC 2008 project.

• Join the Mailinglist at http://groups.google.com/group/SuperDataDaemonThing