<boyd> sandy: so i think we just need to decide which option for sync is gonna give us the most bang for the buck

• and then charge forward on it that's one of the reasons i've been hesitant i would start working on the avahi/mdns solution, but ... i don't think that's gonna solve all the problems/issues because as soon as someone doesn't have their two comptuers on the same subnet, that option is busted

<sandy> well, Alex's idea was to have a webdav server that was exposed by Avahi

• or something like that, right?

<boyd> yeah ...

• i considered writing a simple tomboy server and have it be hosted somewhere

<sandy> you mean for all Tomboy users to use? <boyd> i'd hate to see someone be forced to understand ftp/webdav/etc.

• sure ... in my opinion, we need something like that ... some server though it could be google notes or backpackit.com or something if we had something like that, there'd be no reason to implement the avahi stuff

<sandy> true <boyd> and we'd get 90% of the users happy <sandy> I guess if I'm doing note sync

• I don't want to lose any data that's really my biggest worry for example, does Google Notes or Backpackit support tagging?

<boyd> not sure <everaldo> google notes support <sandy> I have an account but I've never used it * everaldo watching discussion <boyd> the prob. with google notes ... they don't have an api

• so anything that's hacked together right now would be pretty vulnerable for breakage later

<sandy> yeah, bad idea

• onto backpackit which I haven't played with at all

<boyd> well <everaldo> humm... google notes dont have tag, just sessions <boyd> i think one of the issues there would be ... <sandy> backpackit has pages, and each page can have notes, files images, etc <boyd> backpackit.com has a bunch of things that tomboy notes don't

• like embedded images

<sandy> yup <boyd> checkboxes

• etc. so really... a webdav server or something similar would be best

* boyd wonders if there are free hosted webdav services around <boyd> or ftp/etc. <sandy> so we have people come in with various homegrown solutions

• like that guy yesterday

<boyd> what do you mean? <sandy> do we know if most people who are trying to use sync can do it on the same subnet? <sandy> people use unison to sync their .tomboy <sandy> for example

• but it doesn't work the way they want it to

<boyd> ah, right...and ifolder

• or xyz file sync app

<sandy> right <boyd> that would be nice to figure out a solution there

• the issue is that tomboy doesn't have a way to reload all the notes from the directory on the fly it has to be restarted

<sandy> well, but it's more than that

• well, maybe not

<boyd> yeah, i don't like the idea of people/other processes mucking with ~/.tomboy <sandy> but they could be trying to sync between different Tomboy versions, for example <boyd> it wasn't really designed for that

• good point abt. the versions that could cause bad problems

<sandy> we could handle that automatically

• if we had a sync service

<boyd> agreed <sandy> but techinically, if we version our note XML, we can handle it automatically in the other case, too <boyd> well, not as easily if they just wholesale replace all the *.note files * boyd has a new resolution to spend any available "tomboy time" on sync <boyd> <sandy> so I know we've talking about merging/conflicts

• talked

<boyd> well, one thing that i think would be nice ... if we could guarantee a master server ... <sandy> yes <boyd> we could have it manage revisions like SVN does, for example

• i actually really like the idea of having tomboy work similar to svn

  and you could ask NoteManager for what revision it's on etc. did you ever look at that guy's svn sync solution i think he sent it in as a patch in a bug/mailing list

<sandy> the patch we got last year? <boyd> it was kind of clever

• but ... it required someone to manually set up svn

<sandy> I did awhile ago <boyd> it was interesting

• if there was an easy way for people to get hooked up to an svn server ... but that would likely get pretty hairy

<sandy> in all honesty

• it would be easier for us to set up an SVN server then to set up a full blown web service but then they'

<sandy> d HAVE to use the same Tomboy version <boyd> not necessarily though <sandy> *and* Tomboy sync would require them to have the svn client installed <boyd> well...

• if we wrote a little tomboy server ... that used webservices they wouldn't really have to have anything but the tomboy client on there no svn or anything

<sandy> yes...it's just more work <boyd> oh yeah! <sandy> but it at least seperates us from the implementation a bit

• well, let's talk about that presumably we'd need to have an https server

<boyd> btw, one thing i like abt. ftp / webdav ... is that ...

• it wouldn't really require us to write server software just do clever stuff on the client

<sandy> yes <boyd> and all we have to do is find a good webdav/ftp service

• and recommend it and companies could set their own users up with something like that on their own if they chose

<sandy> do such things exist?

• free webdav services?

<boyd> you'd think so

• not sure it would also be possible that we could build-in a webdav server inside tomboy and do the avahi thing

<sandy> right <boyd> if someone wanted to have their desktop machine to be the "master" <sandy> requires user to open port, etc <boyd> i think having multi-masters would be a nightmare <sandy> if they want to sync outside the subnet <boyd> so i like the idea of one and only one master * boyd checks his os x server to see if a webdav server is built-in <sandy> so about versions...

• we release 0.8.x, then 0.10.x adds features more metadata of some sort somebody had 0.8.x on their n800, 0.10.x on their desktop what do we support? lowest common denominator data?

<sandy> "warning, you are syncing notes that come from an older version of Tomboy. This may cause data loss" <boyd> yeah something like that

• we'd try to build something like that in to the 0.8 and then try to have 0.10 conform as much as possible maybe we'd have some sort of control file that's maintained on the server that each client has to conform to ? i'm not exactly sure how webdav works i'm guessing it's just something similar to ftp

<sandy> yeah, I hardly even know what webdav *is* <boyd> well, i guess in my simplistic view, it's a file push/pull api that goes over http(s) <sandy> is that what you think our hypothetical hosted web service would be?

• so that all the work is done by the client?

<boyd> well, if we use webdav ... we wouldn't use webservices

• yeah, all the work would be done by the client

<sandy> right <boyd> so when a client sync'd ...

• it'd lay down some sort of "lock" file so that other clients wouldn't be able to sync at the same time and of course, there'd have to be a fallback in case the first client died mid-sync

<sandy> okay, now I'm seeing your picture

• and there's be some sort of file up there that recorded when syncs were happening

<boyd> right <sandy> so you could know if there might be notes with conflicting changes <boyd> well...

• i wonder if it's possible to never allow a conflict on the server a client wouldn't be able to ever push a conflict up

<sandy> oh <boyd> maybe on the client, auto-rename and then push it up

• so the client manually/automatically woudl resolve the conflict and then get the notes up there. ?

<sandy> actually, this sounds pretty good

• I normally dont' like trusting the client so much

<boyd> yeah. in this case the server is always the "trusted" copy

• and clients have to adjust before they can sync

* boyd wonders what sort of webdav library is available for c# <sandy> well, worst case scenario

• just write some networking code can't imagine webdav is a complicated protocol

<boyd> sweet ... looks like there's a standard dav_module for apache (which runs on osx server...which is what i have installed @ home)

• sandy: yeah, it should be pretty simple

  would it be advantageous for us to build in revisioning in NoteManager similar to how SVN does things?

<sandy> so we could have an "advanced" option where you setup up on Tomboy instance as your server <sandy> and it broadcasts a webdav service over Avahi? <boyd> sandy: exactly

• sandy: but that would come later as an advanced option

<sandy> that should be "one Tomboy instance" <sandy> hmm

• revisioning

<boyd> yeah, so ...

• at any point in time, if we had revisioning ... a client could query the webdav server and know if there's a new version available and any new notes would be added to an "add list"

<sandy> well, certainly there should be some sort of manifest file on the server <boyd> and previously existing notes that already had a version on them ...

• if they're modified, ... we'd know which notes to check on the server if their revision number on the server has changed... that's when we know there's gonna be a conflict (or magic merge someday)

<sandy> this sounds good

• it's not really analagous to svn

<boyd> no, but similar concept <sandy> since svn has revisions for the entire repository, not for individual files

• but I see what you're saying

<boyd> a note revision could be stored in a note file just like any other note metadata

• well...

<sandy> yeah, either a number or a "last commit date"

• could serve the same sort of purpose

<boyd> after notemanager reads in all the notes, it'd know it's overall "revision"

• one reason i prefer revision numbers instead of last commit date ... because it's possible that the times/dates could be not in sync w/ all the other clients/servers

<sandy> true

• okay, numbers are better

<boyd> i would think the first step in implementing this would be to build the revisioning into tomboy

• get that working solidly

  then start setting up some sort of SyncManager ... that keeps track of which notes are new ... which existing notes need to be updated on the server and does a regular poll/check w/ the server and performs the sync, of course thoughts?

* sandy processes it <boyd> ...at least this is how i've been thinking it ought to work ... there's likely many other good ways to do it <sandy> it just came to mind that svn has webdav access

• with auto-revisioning not sure if that really helps us much, though

<boyd> what does it do? i mean, how exactly does that work? <sandy> I think you mount a webdav share

• and you just make changes and the server automatically commits periodically it's for PHBs editing word docs, mostly that sort of thing so the user is a client to webdav, not svn but we'd lose a lot of control that way and anyone wanting to act as a server would have to have that all set up

<boyd> ah, i see. <sandy> but anyway, reviewing what you just said

• I think you're right about putting in revisioning except

  how can we work on it before having the SyncManager?

<boyd> oh, i see what you're saying ... <sandy> I was thinking we don't want to up revision numbers unless we differ from our last sync <boyd> the sync manager would be the one managing the revisions

• i.e., the client can't assign them on its own yeah, basically, the sync manager would be the only thing that could increment the revision numbers during an active sync

* sandy thinks <boyd> if it's not in the middle of the sync (i.e., connected to the server with a lock file in place), it couldn't make an increment <sandy> quick question

• are you thinking that sync happens automatically ?

<boyd> could work either way

• user could manually hit a sync button or run automatically periodically

<sandy> I imagine there are a lot of disconnected laptop/device scenarios that we want to optimize for

• and assuming those work well, autosync should be easy

<boyd> to start with, we'd leave the autosync out

• and just hit some button/command to debug through things

<sandy> good

• okay so I sync I know all the base revision numbers for my notes I modify a note I mark it as modified, but don't increment the number yet?

<boyd> right <sandy> and then incremement the number when sycning the modified note <boyd> it just gets added to the sync manager's "notes to be synchronized" queue

• if the sync is successful, sync manager increments the revision on the note

<sandy> that's one reason I like dates, but I understnad the inherent problems

• with dates, I can just compare last-sync-time to last-modififed-time

<boyd> what benefit would dates give?

• ah

<sandy> now we have to either maintain a mapping of rev numbers to dates

• or mark the note as modified-since-last-sync

<boyd> to show the user ? <sandy> for our own knowledge <boyd> that should be okay <sandy> so that we know what is modified without having to compare against the server <boyd> we could just mark a metadata tag to indicate that it's been modified

• that sync manager would remove upon successful sync

<sandy> although maybe all we need to do is remember (on the client side) when we last did a sync

• and compare modified time to that when we do the next sync

<boyd> true

• either of those ways would work

<sandy> okay

• umm, should we document this?

<boyd> yeah... <sandy> I mean, I have the IRC log <boyd> for starters we could just post the IRC log on our sync wiki

• i like the idea it doesn't seem too complicated

<sandy> yeah

• and we can totally revamp in the future without being too tied into a bunch of web service infrastructure

<boyd> i guess the hardest part for tomboy users would be to just get a webdav acct. somewhere <sandy> yup <boyd> ...which we could have someone step up and provide a free service somewhere <sandy> is Novell interested in providing that sort of thing? <boyd> for the sole purpose of tomboy note sync

• possibly i'd have to ask

<sandy> I hate to put more burden on the gnome sysadmins <boyd> i have a server i could donate ...

• but it's not fast and old it would be experimental

<sandy> quick question

• are UIDs expected to always be unique, even between different computers?

<boyd> note UIDs? <sandy> right <boyd> well, in theory, a UID that's generated on one computer should not ever be able to be created by any other computer <sandy> is it possible for client A and client B to generate two notes with the same UID, and then tryto sync....

• okay

<boyd> i believe in practice it works that way too <sandy> sounds good <boyd> the one issue though

• is that ... if someone took two machines and synced them up to the same webdav acct. we don't check UID necessarily we use note title

<sandy> oh? <boyd> so the 2nd computer in would have all their duplicate notes renamed

• right?

<sandy> but that might change

• if I rename a note...the other client would see a delete and add seems weird why not go based on UID?

<boyd> well, once two clients are synchronized ...

• they would use UID i mean, if two non-connected clients were to both import their separate notes instead of ... (preferred process) ... sync computer #1 with all notes then sync computer #2 (which has no notes)

<sandy> okay, I see <boyd> i.e., no potential of name conflicts <sandy> there could potentially be name conflicts <boyd> yep <sandy> so the client has to check note titles

• this could happen with NotD

<boyd> and i think in that case, on the clien that has the conflicts, we either rename the conflicting note to "NotD (2)" or prompt the user to rename it or take care of it <sandy> if you're not careful, you could try to add a new one twice

• right sounds good for now prompt them to rename

<boyd> i think as we get into it more, it'll make more since (i hope) and we'll likely discover other issues like this <sandy> and have checkbox "automatically rename for me next time"

• Let's get an experimental server set up ASAP for our testers the people who understand about backing up stuff

* sandy wonders if he can host webdav stuff on his dreamhost domains... <boyd> i think all you really need is an apache server

• with the webdav module enabled
• ..not sure how you set up users though that might be a pain but i imagine someone out there would be able to write some simple asps to take care of that

<sandy> apps? <boyd> ASP.NET <sandy> ASPs?

• ok

<boyd> so a user could go "register" for an account <-- Shingo-- has quit (Leaving.) <sandy> http://blog.dreamhosters.com/kbase/index.cgi?area=2882 <boyd> something else someone could eventually do is ... <sandy> webdav can do automatic locking <boyd> write an ASP.NET that accesses the notes in WebDav

• so that users have access to their tomboy notes online

<sandy> yeah, that would be great <boyd> which is why it would be nice to tie in with one of these existing online note services <sandy> we already have a simple xsl to do that for the export * boyd wonders if any of them expose their notes via webdav <sandy> the cool kids write fuse filesystems for that sort of thing <boyd> if we can capture this stuff down in a way that makes sense, we could mail it out to the mailing list and ask for support/suggestions

• i know we'll have some disappointment from conduit guys

<sandy> yeah

• I'm sure some people will prefer Conduit for backpackit support

<boyd> but this follows right along with the plan we laid out in our RoadMap <sandy> and that's fine

• yup once there's a solid Maemo port... that will be a huge sync use case will conduit work there?

<boyd> agreed

• not sure this woudl though

<sandy> exactly <boyd> since it just does everythign via http <sandy> I can write something up to send to the list if you're busy (you're better at writing this stuff than I am) <boyd> i'm modifying the Tomboy/Synchronization wikipage right now <sandy> okay, awesome