About

When an application is about to create a file and it knows beforehand the metadata, it can make that metadata available sooner than at the moment of the file's creation.

Typical use cases:

• Downloading/Transfering a file
• UPnP uploads: upnp

Solution abstract

The application can inform Tracker's RDF store (tracker-store) about the new resource, and then Tracker's filesystem miner (tracker-miner-fs) will complete the information later. This makes the information available immediately; the file might not be completely written in the filesystem, but this is also annotated (implicitly) in by the resource's rdf:type.

Interaction between the Application and Tracker

1. Application creates the resource in Tracker, making it an instance of  nie:DataObject  (Important:  nie:DataObject  and not  nfo:FileDataObject ) and using a anonymous blank node.

   • INSERT { 
      _:x a nmm:Image, a nie:DataObject ;
          nie:url "file:///a/b/c.jpeg" ;
          # Other properties known at this time 
     }       

2. By default Tracker's filesystem miner will try to extract metadata from the file when your application is finished writing the file. If the application wants to avoid that this takes place (they have already provided all needed metadata and don't want Tracker's filesystem miner to extract and overwrite anything), they need to call the IgnoreNextUpdate method on the filesystem miner each time in front of finishing writing. See lower for a definition of "finish writing the file".

3. Application starts writing the file in the filesystem. Note, if the file has a thumbnail, the thumbnail can be written at this point

4. Nothing else must be done by the application. When the file is ready, the filesystem miner will set the resource as  nfo:FileDataObject  and it becomes a regular file-resource for tracker.

Extra considerations

• As soon as it is inserted, Tracker will notify applications using the Signals mechanism. At this point is the new resource visible for other applications.

• "Finish writing the file" means either close() or rename().

Example for a download

An application called flickr-sucker is downloading a picture from Flickr, making a local copy of it. The image will be stored in  /home/user/MyDocs/pictures/a.jpeg . It has all the important metadata from the server before starting the download: width, height, title and so on. So it is not needed to extract them from the image at the end of the process.

1. Flickr-sucker writes to tracker the metadata of the file (for example using tracker-store's SparqlUpdateBlank() D-Bus method):

   • INSERT {
       _:x a nmm:Image , nie:DataObject;
          nie:mimeType "image/jpeg";
          nie:title "What a party" ;
          nfo:width 800;
          nfo:height 600;
          # Other stuff like flash, location, contacts, ...
          nie:url "file:///home/user/MyDocs/pictures/a.jpeg".
     }

2. Before closing the file, Flicker-sucker calls the  IgnoreNextUpdate (FS_url)  method in the miner.

3. Flickr-sucker opens and writes the file
4. Flickr-sucker can now close the file. Tracker's filesystem miner will receive the "Close" event, but it will be ignored.

Example for a UPnP Server

UPnP MediaServer is receiving a file (Upload request) that will be stored as  /home/user/MyDocs/music/a.mp3 . It is a song from  Bob Dylan  called  Everybody must get stoned . These metadata is important for other applications so they should be set in Tracker's RDF store as soon as possible. This case is different from the previous because the UPnP MediaServer is going to create a empty file before starting the actual writing and it wants the actual metadata extracted from the file.

1. UPnP MediaServer writes to tracker the basic information of the file (for example using tracker-store's SparqlUpdateBlank() D-Bus method):

   • INSERT {
       _:x a nmm:MusicPiece , nie:DataObject;
          nie:mimeType "audio";
          nie:title "Everybody must get stoned" ;
          nie:url "file:///home/user/MyDocs/music/a.mp3".
     }

2. The result of the previous operation is the generated URN of the resource (added as blank node _:x)

3. UPnP MediaServer calls IgnoreNextUpdate method in the miner and creates an empty file.

4. When/if UPnP client requests to push data to the file, MediaServer just forwards the incoming data to the file.

5. When data has been written & file is closed by MediaServer, Tracker's filesystem miner discovers the change and extracts the metadata.

Command line example preventing file from overwriting set application

Here we will use the  IgnoreNextUpdate  D-Bus method on Tracker's filesystem miner.

We insert file information with title test, ask Tracker's filesystem miner not to extract metadate out of it, copy the file in (i.e. creating a file), and check the title in tracker ('test') and in the actual file ('Sunset 2'). It hasn't been extracted. Application's metadata data is used, the file's isn't.

$ mv sunset.jpg ..
$ tracker-sparql -u -q "INSERT { _:x a nfo:Image, nie:DataObject ; nie:url 'file:///.../sunset.jpg' ; nie:title 'test' }"
x: urn:uuid:f81e512a-de02-ab3c-ca70-616199eb8c78
$ tracker-sparql -q "SELECT ?title WHERE { ?file a nie:DataObject ; nie:title ?title ; nie:url 'file:///.../sunset.jpg' }"
Result: 1
 test
$ dbus-send --session --print-reply --dest=org.freedesktop.Tracker1.Miner.Files        \
 /org/freedesktop/Tracker1/Miner/Files org.freedesktop.Tracker1.Miner.IgnoreNextUpdate \
 array:string:"file:///.../sunset.jpg"
 method return sender=:1.219 -> dest=:1.226 reply_serial=2
$ cp ../sunset.jpg ./sunset.jpg
$ tracker-sparql -q "SELECT ?title WHERE { ?file a nfo:FileDataObject ; nie:title ?title ; nie:url 'file:///.../sunset.jpg' }"
Result: 1
  test
$ exiftool sunset.jpg | grep Title
 Title                           : Sunset2
$

When you plan to use rename() after you are done writing the file, for atomic updating it, you can pass both urls to  IgnoreNextUpdate 

$ dbus-send --session --print-reply --dest=org.freedesktop.Tracker1.Miner.Files        \
 /org/freedesktop/Tracker1/Miner/Files org.freedesktop.Tracker1.Miner.IgnoreNextUpdate \
 array:string:"file:///.../sunset.jpg.tmp","file:///.../sunset.jpg"                    \
 method return sender=:1.219 -> dest=:1.226 reply_serial=2
$ cp ../sunset.jpg ./sunset.jpg.tmp
$ mv sunset.jpg.tmp sunset.jpg

Command line example with the file overwriting the data

We set the title 'test', copy the file in, and now the file is a FileDataObject and the title comes from the extraction

$ mv sunset.jpg ..
$ tracker-sparql -u -q "INSERT { _:x a nfo:Image, nie:DataObject ; nie:url 'file:///.../sunset.jpg' ; nie:title 'test' }"
x: urn:uuid:a276d682-97da-25cd-0072-536ad95ecd48
$ cp ../sunset.jpg .
$ tracker-sparql -q "SELECT ?title WHERE { ?file a nfo:FileDataObject ; nie:title ?title ; nie:url 'file:///.../sunset.jpg' }"
Result: 1
 Sunset2
$