I think that user annotations are a must, and based on first comments, users will like the idea. Annotations can be anchored to paragraph, image, list item or a table, function or constant definition in API references and can contain text, lists, links and remotely-hosted images. (I am taking care of XSS). They are loaded in "Ajax" manner, stored on server side. (At first, I planned to store them in cookies on client side, but it will disable some other features) Also, they can be rendered anchored (displayed on mouse hover) or as a comments, at the end of a page or in a side pane.
There will be a annotation manager, from which users can "subscribe" to annotations. Some scenarios I am thinking about are:
Michele wants to write something down, like "Never try to do this again" or "This is changed in Ubuntu" or "Here is more info on this"
John wants to improve docs, so he write what he will add/replace in new annotation anchored to selected paragraph. (Quick feedback described in documentation flowchart in Paolo Borelli's comment on Live Documentation Editing.
Sarah, Peter and Paul, 3rd grade hi-schoolers wants to subscribe to theirs school teacher's annotations adding additional help to User Guide for a project they need to do.
Emilia is starting using Ubuntu so she subscribe to Ubuntu tag to load Ubuntu specific annotations. (latter)
Cesilia is doing modification to a program that is using GTK+ and she subscribe to annotations source from company she is working for to see code snipets about best practices while she is browsing API documentation. (latter)
So, all annotations will be public and shared in a read-only way. Documentation editors will be able to review (by subscribing, and latter by using special made interface) and remove all offending or inaccurate annotations. Remember, annotations will not be displayed on default, and user must subscribe to some other user's annotations to view them so there should not be many deletions to do.
Combining annotations in a group is done by concept of tags, with "tag masters", coordinators that can approve/deny inclusion of specific tag to annotation. One annotation can have language tag and tag defining category(ies), and other users can subscribe only to specific tags on specific language(s). Languages list also can be loaded from browser preferences, as a default value, but as annotation manager is server side script, it can have additional customization for overriding browser settings.