IM, Contacts & Social hackfest: Shell: Design: integration of more Telepathy events

  • Owner: GuillaumeDesmottes

  • Integration of calls
  • Integration of file transfer
  • Integration of room invitations
  • Integration of subscription request
  • Fail to connect to chat account
  • Integration of desktop sharing?

Notes

Nick Richards enters.

Now that we've agreed upon a design that allows us to be signed into IM without having Empathy running, we need to improve notifications for incoming calls and other events. At the moment, Empathy itself does the “do you want to answer the call?”, “do you want to accept this file?” etc. bubbles using libnotify.

We could either split this into a different process which can be service-activated; or we can move things into the shell.

First thing: these notifications need to persist for the length of the offer.

Nick asks: do we have information about whether the call is audio-only, video-only or audio/video? We do have this information. In the specific case of calls, it's going to be a lot like something on a phone (except maybe not filling the screen). But not as obnoxious as Skype.

What do we want to do when we're busy? Do we tell the user they've got an incoming call, or do we auto-reject it? Can we do something in between the full-on notification and hiding the call? People know that phone calls interrupt the other person, so they make them explicitly when they do want to interrupt the peer. Particularly now that we have IM and SMS, people only call when it's necessary; so we do always need to show the call. It's on a par with “your battery is dying”.

We should probably look up the individual for who is calling and show the avatar and alias, just like we do for messaging bubbles, says Travis. Nick adds that the contact card experience, as seen in GNOME Contacts, Empathy, etc. would be ideal.

Should in-progress calls be tied to the shell, or should they continue to be a window you can move around? If so, should the notification be the window, too? Could we have an “ongoing call” thing in the shell, just like you get one in the taskbar on iOS, Android, N900 etc?

Allan wonders whether we can include all ongoing IM/VoIP activity in the bubbles in the messaging tray, not just IM? So we would have the IM session as normal, but also have information on the call in progress, file transfer, etc. Nick wonders, would it be ongoing activities, possible activities, or both?

Stef notes a commonality: call notifications want “accept with video; accept without video”. But we also want these controls within the ongoing call. Nick suggests that we could remember whether the user turned on the camera in the last call, and turn it on again; then you tell the user what will happen if you accept, but don't provide them the option. Guillaume notes that this might be an issue for the “I'm naked” use case. What if the user is told that they will be sending video, but doesn't want to: are they supposed to hang up and call back?

Danni likes the idea of separating “no”, “yes, but don't send video”, “yes, and video too”.

Nick likes the idea of a small experience with a way to take you to the full experience, for text chats. Clicking the bubble is maybe not the best way to do it—you have no idea what clicking it will do—but the general pattern is nice, and we could do something similar for calls.

Danni notes that you don't really need much UI for audio-only calls: you only have “hang up” and “mute”. It might be nice to have simple controls in the shell, and have a full window to pop out for video calls etc. Nick notes that one other control we want is “switch microphone” and “switch camera”.

A bunch of conversations happened; none of which your scribe could follow because there were way too many happening at once.

Danni sketches a diagram. The call controls can be in the shell, and Empathy-Call can be hidden (but still doing the GStreamery bits behind the scenes). When you click the “video” button, or mash the controls at the bottom of the screen, the video window would show up. The controls are always visible, so that when you're on a conference call you have a constant reminder of it.

You could also show the window for more details about the ongoing audio-only call, get dialpads, etc. So the window could have no close box, and explicit buttons for [ Hide ] [ Hang up ] [ Turn video on/off ].

Nick wonders about a cool visualization to show in place of video in an audio-only call: perhaps call quality indications, now-speaking indicators, etc.

Bastien thinks we don't want the shell to be the be-all and end-all of IM/VoIP conversations. It's fine to have rich notifications in the shell, and it's fine to be able to reply quickly to an IM, but we should take care not to feed more things into the notification model than belong there.

Nick returns to considering just a little notification in the bubble for a contact:

  Call      7:45 
  ---------------
  Me: some stuff
  Nick: hello

banner.png

expanded.png

Nick mentions that the ongoing call bar is a concept that doesn't really exist in GNOME Shell right now, and maybe there are other cases besides ongoing calls that need an ongoing reminder concept in the Shell vocabulary: maybe for now-playing music? Sjoerd notes that being able to mute the call as quickly as possible is a nice reason to have a bar at the bottom; Nick points out that this applies to music as well. Perhaps we can use the "mute" key on (say) ThinkPad keyboards to mute both in and out on the call. Bastien says this is technically possible, but the behaviour needs to be clear and documented, particularly in the case of bluetooth headsets, etc. It's not just muting the microphone, says Sjoerd: the application needs to know what happens, so it can tell the other party (over XMPP, or whatever VoIP protocol) that the call's been muted.

Should we merge the full-on call window with the full-on conversation window? There are technical difficulties for doing this in any reasonable timeframe, and also it doesn't make sense on SIP or POTS or on a VoIP call to a phone where the user is pressing the phone to their ear and so can't read the IMs you're sending them anyway.

Allan: should the items in the messaging tray be centred on people (as they are currently) or do we need separate notifications in the tray for different messaging types/applications? Bastien: we can declare that the pop-out items are *only* for text IMs plus a tiny bit of information, and anything more significant needs to be in a separate app.

Jumping back to the checklist:

File transfer: will end up in the sharing settings, with options:

( ) Never
(o) Always ask me
( ) Always save them in [ Downloads ]

We can move the FT approver into the shell, and make it check that information. We will need the same UI in the shell for other types of file transfer—bluetooth, wifi direct, etc—with slightly different presentation but broadly the same kind of information.

Subscription requests: do all protocols allow you to defer requests across reconnections? In general, we should be looking to work a little more like Facebook: rather than present the user with questions, show a “foo has added you as a friend” notification, and have UI for pending requests that the user can access independently. This would be integrated with Contact List / GNOME Contacts. This lets us give the user a way to get more information.

For sharing/incoming transfers stuff, we have mockups for the system settings panel, and Bastien will probably be implementing very similar notifications to what was in the bluetooth file transfer stuff for GNOME 2.

Providing the same user experience for file offers, room invitations and subscription requests as we have now from within the Shell, as a starting point for future integration, would not take too long. But nor would splitting EmpathyEventManager out of Empathy itself take forever.

Desktop sharing! This is like a call in terms of ongoing notification. Bastien wonders what the use cases are: just technical support and demos? Nick points out that it's a lot like a call. Guillaume notes that our current model is backwards: we want to be able to ask a contact to share their desktop with me, as opposed to the contact having to find the “share my desktop” button. Bastien notes a social engineering attack: rename yourself to “Microsoft Support” and ask to see their desktop. Stef says that, in the proprietary world, you normally tack desktop sharing onto an ongoing phone call. This gives you much better protection against social engineering attacks. This is a way from where we need to be, but this should be where we're heading.

There's a bunch of stuff which Empathy does during connection, like the ugly cert dialog box and password prompting, which needs to go somewhere. Stef thinks that the certificate one should go into the advanced settings in your account dialog: it's configuration. bug #652814

For people who don't want to save their password: we can just have a notification, you mash it, it asks for your password. For other authentication/sign-in failures, we should have a GOA-esque “one account failed”/“some accounts failed”; clicking it could take you to the most-recently failed account in the accounts dialog.

We should show a notification when (say) the Google server dies, and it should not repeatedly notify you on every reconnection attempt: only the first time, or when you explicitly say “reconnect me plz”. Whether we notify on successful reconnection is an open question.

Sometimes, your IM connections fail before NM realises the network connection failed: what do we do then? We'd have one notification for “3 accounts got disconnected” followed immediately by “your network connection died”. How can we avoid this? We really want to replace the content of the original notification, rather than replace the whole thing.

A digression about where monitoring network connectivity lives, given that currently it's in the Empathy contact list process and the Empathy contact list process is going to go away. It probably makes sense to put it into the gutted Empathy approver process that we're going to need for the transition: given that the code already exists, we're gonna want to keep it rather than rewriting it all.

Bugs

Hackfests/IMContacts Social2011/Tasks/ShellDesignEvents (last edited 2011-08-05 13:51:50 by GuillaumeDesmottes)