This is just a place to write down thing that we were talking during the Hackfest. They are mostly a dump of ideas. It would be good to have some conclusions.

Contents

  1. Day 1
    1. Global events vs per-object events
    2. Key event emission
  2. Day 2
    1. Actions
    2. Convenience of ATK/ATK-bridge/other plattforms
  3. Day 3
    1. accessibility enabled by default
    2. Visibility vs Availability
  4. Day 4
    1. Actions redux
    2. Selection
    3. Versioning
    4. Coalesce signals
  5. Day 5
    1. More keybinding
    2. About descriptions on gdkpixbuf

Day 1

Global events vs per-object events

  • Benjamin started with some concerns about a lack of someone having a deep knowledge of all the accessibility stack + DBUS
    • Someone that could debate with a DBUS developer (that we are lacking) about the current situation (about why we have what we have and why this is the best solution)
    • This can affects this debate
  • leasing accessibility objects
    • notify somehow that the accessible object is not available
    • Improve the process to get the object under demand

  • Related with the previous: AtkImplementor

    • Benjamin mentions that it would be good the AtkImplementor exposing the dbus object path somehow (generic ID)

    • bridge could cache the implementor instead of caching the accessible object
    • when required they would ask for the implementor for the accessible object
    • that would make easier to treat more dynamically the accessible object (no problem if we remove it)
    • Piñeiro made some questions

      • accessibility notifications are only sent when the accessible object is available (bug about requiring the accessibility object)

      • ie: AT is using an accessible, how they would receive a DEFUNCT notification if the accessible object is not available anymore
      • in that sense it seems that the accessible needs to be available as far as the AT "requires" it

      • what means "require it" => with per-object connection, that would mean having the accessible as far as someone is listening

      • in the past, with reference count, the object is "required" as far as the AT maintains a reference to that object

    • Day 2 question this would make sense with a strict and tight 1-to-1 relation between the accessible objects and the base objects

      • What would happen if we want a representation of a accessible object managing more that one base object, or several accessible objects for a base object?

    • Day 2 question to Benjamin

      • Piñeiro commented the option of having AtkObject as a interface instead of a object (bug about that), and use that on GtkWidget

      • Benjamin doesn't like too much this option.

      • GtkWidget is already really big, and it already frightening people, so include more interface could make that worse

      • That would not solve the underlying issues.
  • Qt seems that only emit specific events if some AT is listening to that specific events
    • atk-bridge doesn't emit events if there isn't any AT listening but AFAIK, it doesn't have that event emission tuning
  • Discussion about what is a "relevant" object: focused object is not enough
    • ie: searching. Focused object is the entry, but the dynamic search changes, and need to be exposed
    • Introduce a high-level widget or a searchable interface?

    • Add a new kind of relationship? => the problem is that someone would require to manually set that relation, hard to do at the toolkit level

    • Also notifications
  • About stop to emit some events to D-BUS?
    • Some discussion about what events are relevant or not
    • ie: Orca doesn't care too much about "children-changed"
    • Straightforward way to decide which events are relevant: AT explicitly asking for those events
  • When asked about some kind of conclusion about maintaining global-event
    • Problem with the way global events are right now implemented with hooks
    • Benjamin thinks that if we provide a way to notify global events, the current implementation is bogus
    • Benjamin thinks that the way to get global events is to do a direct connection to each object
      • In the end that meand a per-object connection to each element
    • next question is about who would decide the object to connect to (at the server side, or at the client side as MacOSX API reference seems to suggest?)
    • About "global-global" events (events that only makes sense as global), we always have the possibility to emit events from some toplevel object (root)
  • Piñeiro random thoughts about the example with MacOSX:
    • Why define more than one observer?
    • Why the callback is defined when you create an observer?
      • It seems to make more sense to define a callback when you add a notificatio

Conclusions

  • For the moment it is not clear the ideal solution (global vs per-object) as there are too many questions and too many maybes
  • It is clear that current implementation (hooks) on ATK is not apropriate

  • Piñeiro day 2 comments: thinking on the short medium and long term

    • Short term: Taking into account that window stuff is now real ATK thing, as I said on that ATK bug about global events we can implement the listeners code on ATK

    • That would allow to remove this code (so that hook code) from the toolkits, so at least, we will have only one implementation, so only one place to modify things (look what Qt does would be good)

    • Medium term: Meanwhile we explore alternatives it would be good to implement some kind of fine tuning to the current solution.

    • Medium-long term: For a while (<crickets sounds here>) we will maintain that hook-based implementation on ATK, but we need to start to think/play with some alternatives to improve the situation

Key event emission

  • Benjamin deprecated key event snooping, and also remove the key event emission from gtk
    • Several issues (like reentrancy) with the current approach
    • Not the proper solution
    • Also silly to require to reimplement this in any toolkit, when in the end, X calls are used
    • Piñeiro will be also happy removing that from Clutter (he doesn't plan to do that until having that alternative)
  • How to solve that?
    • Use X related stuff on the client side (at-spi2 or Orca) to re-implement that
    • Should be possible and not really complex

    • Frederik suggested http://kaizer.se/wiki/keybinder/

      • Piñeiro reviewed it a little
      • Not enough features
      • Unlikely to be a option as code reference, it relies too much on gdk (and also on gtk)
    • But the fact is that is not available right now, and also unlikely to be finished during this hackfest week
  • Benjamin asked Frederik how this works on Qt:

    • FIXME no notes about that

  • Appendix: About what others do:

    • MacOSX global event monitoring

    • Funnily enough it is only possible to observe key-events if accessibility is activated
    • Benjamin point: you don't receive an event, but a copy of that
    • In the same way, it is asynchronous, so there is no possibility of blocking stuff

    • Day 2 doubt: does observe to asynchronous key events ensure the proper order?

Conclusions

  • Benjamin reverted that change
  • This feature still makes sense, but implemented on the client side (at-spi)
    • And probably we should raise the priority (after all, now gtk is using deprecating stuff)

Day 2

Actions

  • Voice Control applications (like Simon or the deceased GNOME Voice Control) are the poster boy consumers of Action
  • Orca mainly requires it because is the place keybindings are exposed

  • About "not implement an action if it is already implemented by other interface" ([[http://developer.gnome.org/atk/stable/AtkAction.html#AtkAction.description|AtkAction documentation])

    • Usual way to interact with Tabs is using the Selection interface
    • But probably this and others could be more easy with some actions
  • In general, for applications like Simon, it would be easier to use ATK/AT-SPI2 stuff with a list of available actions, instead of looking at actions and what he can do via other interfaces

  • AtkSelection redundant?

    • It seems that this functionality could be obtained implementing Table interface

    • This is more true with Table2 and TableCell

  • AtkEditableText redundant?

    • set the run_attributes are sometimes not possible, and should be something provided by the application
      • Or using the UI or using a shortcut (ie: Ctrl+B for bold)

    • AtkText already have all the relevant signals (update, insert, etc signals)

    • Benjamin only concern is related with insert-text
      • Used the example of Simon
      • Frederik thinks that it is not required, as Simon could translate that as key events
  • Local vs global bindings
    • Local: bindings that are only available in a specific (local) situation, like on a menu
    • Global: bindings that you could execute in any moment

    • FIXME no more notes

  • Idea: call the action using the name instead a index
    • If that index changes, that could be a problem (it doesn't happen right now, but could in the future)
    • We already have a method to get a localized name
  • Idea of moving all the possible action-like methods from other different interfaces to actions
    • But keep just one implementation (ie: atk_component_grab_focus being implemented by calling the action)

    • FIXME: Piñeiro question, but I guess that we could deprecate some of those "action-like" methods on those interfaces

  • Discussion about the focus action

    • FIXME: I don't have notes about this part

  • Some debate about corner cases
    • What would happen if a dialog appear and you want to click on a button
    • other cases

  • FIXME: I have the feeling that I'm missing a lot of stuff, ie about keybindings

  • Benjamin wondered about the possibility of an Action object.
    • Current Action interface exposes all the actions and allow to execute it
    • So he is talking about having a real Action object
    • Frederik seems to prefer to keep things simple as current stuff is working fine enough
  • Primary Key and what should ATs get from toolkits
    • Gtk+ now exposes 'Primary' rather than 'Control' for the GDK_MODIFIER_MASK in *nix
    • The question is what should we get? Should 'Control' be provided by Gtk+ to restore existing functionality, or should Orca start presenting 'Ctrl' to the user?

    • Joanie said that presenting 'Ctrl' is something doable (via Gtk.accelerator_parse() and Gtk.accelerator_get_label()). What she does not know is why we were not doing this before in Orca, was it by design to meet a particular use case, by need due to the fact that these Gtk+ methods did not exist at the time this functionality was added to Orca, or is there no good reason? ;)

  • Appendix: What others do

    • MacOSX available actions

    • They have a tight relation between actions and roles
    • It would be hard to implment that tight relation on ATK, but this can serve as a reference

Conclusions

  • FIXME most of the outcome are basically questions

  • Add the possibility to execute a action based on the name instead of the index
  • Start to provide more actions, including some of the stuff already available on other interfaces
  • Regarding the 'Primary' key: Joanie will research the history/rationale. Based on this, she will either modifier Orca to present the accelerator label, or she will ask Benjamin to resume providing 'Control'.

  • FIXME for sure I'm missing something

Convenience of ATK/ATK-bridge/other plattforms

  • Benjamin is still considering skipping ATK and make a direct GTK-DBUS talk
  • Long talk about how this works on QT
    • In fact that wouldn't be the same proposed by Benjamin, but is hard to compare both due all the C++ stuff involved there
    • Anyway, it still talks with abstracted interfaces
  • Piñeiro (perhaps too paranoid) concerned about any hypothetical but unlikely IPC change
    • ATK implementations by toolkit survived that CORBA-DBUS change
    • In that sense, Qt accessibility support would also survive a IPC change (changing the bridge)
  • Ranting about DBUS
    • Mike Gorse mentions that some people suggested that for accessibility requirements it would be better a different IPC technology
    • Benjamin doesn't agree with those people
    • Benjamin suggestion about how things should be defined, mentioning auto-generate all the bindings and implementation on DBUS
  • Benjamin concerned about what would happen if someone asks for gtk accessibility support on MacOSX and Windows
    • Piñeiro asks about the current GTK support on those plattforms
    • Benjamin makes a summary, also add that this is not a excuse for not think about that now
  • Frederik explains how IAccessible2 is used on windows, and also mentions UIA
    • For the case of IAccessible2 it seems feasible to make a kind of ATK-IA2 wrapper-bridge (Eitan idea)
    • Piñeiro asks what NVDA uses: both libraries
  • Linux Foundation provides the idl for IAccessible2, making possible to auto-generate binding

Conclusions

  • Benjamin is still considering about skipping ATK and use directly GDBUS on gtk
    • He is not really sure anyway

Day 3

accessibility enabled by default

  • Starting to define what that means
    • ie: user being able to execute/run orca withouth the old "logout first" procedure
  • Mike comments that right now the bridge doesn't emit anything if no AT is listening
  • Benjamin think that is should be a two-way thing
    • If you turn off again orca, apps should keep working as if Orca were not launch in the first time
  • Benjamin would like to stop using the bridge as a module, but as a library
    • That measn that people compiling gtk will compile also the bridge
    • Would allow to add API to the bridge to be used by gtk accessibility code
  • Piñeiro: other advantage, solve the problem with the placement of the module
  • Benjamin: other advantage, modules modifies the binary

  • Brief conversation about the root object, and what happens when you have more than a toolkit Comment about adding a atk toolkit object

  • About the problem with the key events, when you define a root object, telling him to be the one taking care of the key events listeners

    • Move AtkUtil code to this AtkToolkit object

    • BTW, another reason to remove the key events from the server side as this is the thing making complex this AtkToolkit objct

  • Doubts about dependencies
    • Will clutter, gtk accept a new "atk-bridge" dependency?

    • Add atk-bridge as a ATK dependency? => cycle denpendency

    • Obviously there are some issues and details to solve, and cant be solved tomorrow
    • Commenting DBUS dependencies
  • We are going to have a lot of people complaining and making questions
  • Too near to 3.4 to do that now, we should target 3.6, enabling by default a11y at the beginning of the cycle
  • Joanmarie has some concerns about the interaction with the accessibility-toolkit gsetting
    • in theory that gsettings key will dissappear
  • Discussion should start before 3.4
    • ie: post summary of the event
  • About how to do that atk-bridge thing
    • As a library
    • Integrating it directly on gtk
  • Benjamin thinks that all that stuff should be only done on gtk3
    • gtk2 on maintainance mode
  • Question about how to know if there is a consumer
    • Asking the bridge if the is someone listening
    • Fore more refinenment, that idead about about AT registration (with AT roles)

Conclusions

  • Having accessibility by default
  • at-bridge should stop to be a module
    • Details (independent library, integrate on ATK, integrate on GTK to be decided)
  • Discussion should start before 3.4
  • Change should be done at the beginning of 3.6 cycle, so it could be properly tested (and reverted if things go bad)

Visibility vs Availability

  • The purpose is discuss when an object should be exposed via DBUS

  • FIXME total brainstorming, not notes, add details if you want

  • Benjamin proposes to create/destroy objects with a map/unmap
    • Corner-cases with comboboxes, menuitems, ..
  • One idea could be create the accessible object when it is shown
  • Some comments about accerciser (but as we now, this is not a real case)

Conclusions

  • Benjamin will experiment about that on GTK, so probably some objects will "disappear"
    • If "too many objects" disapper, we should complain

Conclusion

  • Postponed

Day 4

Actions redux

  • Frederik suggested about a signal on add/removal actions
    • That could be useful for Simon
  • Benjamin mentions that right now add/remove an action can be dangerous
    • Because you execute action by index
  • Piñeiro mentions that as we already concluded to execute by name the plan would be:
    • Deprecate do_action (index)

    • Add do_action_by_name on AtkAction

    • Add that signal
  • This is related with the bug about adding add/removal methods on ATKAction
  • Piñeiro also asks what would happen at object creation, concerns about too many signals
    • Consensus about not emitting that signal at init case
  • Benjamin comments an alternative. Add a "available" (he didn't use that name) property for actions

  • Frederik comments about how this works in Window (FIXME)

    • Mentions that they have the concept of the default action
    • Benjamin mentions that gtk has a similar concept, as you have "activate"

Selection

  • Selection stuff
    • Discussion about caret-move, selection-change signals
    • Orca would like to know if a caret-move signal is caused due a selection change

Conclusions

  • Conclusion: continue working on improving notification of selection changes

Versioning

  • About problems updating different parts of the stack
    • Frederik about exposing the version exposed
    • About providing not only the name of the toolkit, but also the protocol version implemented
    • Mixing toolkits: webkitgtk, about problems gtk using a new function but webkit no

Coalesce signals

  • It is not required to just reduce/fusionate the events, but for example, know that a bunch of events are related somehow (ie: user press something => following events are related with that action)

  • Orca could have problems to relate events, but it is more or less the same problem at the toolkit level
  • Would be hard to find a generic way to do that
  • One idea would be send something like "starting to do something" - "ending to do something" signals
    • A lot of corner-cases: ie dialogs
  • Frederik comments that in general we are emitting too much information to the bus
  • The idea would be have more clear what it is required or not
    • That would reduce this problem
  • Invalidating vs compressions
    • Some discussion from Benjamin and Frederik regarding how they collapse certain signals (e.g. mouse position updates)
  • ref_counting
    • Similar to the conversation we had on day1
    • We had that on corba, not implemented on at-spi2
  • bounds-changed
    • Frederik doesn't emit it on QT, in his opinion doesn't make sense (ie: animations)
    • Benjamin comments that makes sense in specific scenarios
    • Frederik mentions that it is not clear when it should be emitted
    • Piñeiro mentions that clutter also don't emit it, more or less of the same reason Frederik said

Conclusions

  • In general, we already discussed at different moments of the hackfest the need of not sending so much information to the BUS
  • In general, having a better definition of what is relevant
  • For example: previous visibility vs availability discussion
  • So in that sense, if we solve that issues, coalesce signals could not be so relevant

Day 5

More keybinding

  • For the while update the documentation for something understandable
  • Pass a localized string
  • Right now that string and the format, for the future
    • Piñeiro mentions that IAccessible2 provides an array
    • Benjamin mentions that the problem with just one array is that we have key bindings and mnemonics
    • Add a method like get_mnemonic would make sense.

Conclusions

  • Short-term: update documentation to explain the format of that keybinding string
  • Medium-term : provide a more high-level API to get the keybindings

About descriptions on gdkpixbuf

  • Current mapping is not enough name, description ... where tooltip fits
    • Curious: Qt has a tooltip, a statustip and a whatsthis. Also provides a way to set accessiblename and accessibledescription (it would use fallback?)
    • If it is a text entry, it also has the place holder
  • Required to properly define the mapping
  • gtktreeview allows to set a tooltip per cell
    • Joanie think that in that case that tooltip is a good description
    • Joanie asks Benjamin if it could be possible to expoes that tooltip as de accessible description
    • B said that at this moment no, because he requires to do some research (and he will)

Accessibility/Hackfests/ATK2012/Brainstorming (last edited 2012-09-23 12:12:20 by AlejandroPiñeiro)