1. About internals
Documents are located using "locators". For general locators like gdu (gnome-doc-utils) or gtkdoc (gtk-doc API references) additional tweaking can be done using "decorators". Locator must be stored in locators directory, and defined as class named after its module with 'Locator' sufix. On loading, module is imported and class initialized. Dictionary containing locator arguments is passed to locator's constructor. There is a big note in file 'libgo' on how you can add new options to be passed to your locator.
Locator class must define getList() method without arguments that returns list of document objects located. "Document objects" (or docobj for short) does not have to be fully initialized (some of required properties can be empty) but it must be capable to finish initialization and do build process itself. For example of locator implementation you can look at gdu locator in locators/gdu.py and/or read a section about it in this file.
Of locators, gdu locator is implemented. It can use GNOME's CVS or SVN or CVS image obtained with jhbuild to access files and selection is done with command line arguments and access via SVN is default.
Locator can abstract IO operators using a driver for loading files into cache. Like locator, driver also must be a class, named after module in which it is located with a 'Driver' suffix. On loading, module is imported and class is initialized. Unlike locator, driver does not need to implement any specific interface but it need to be compatible with locator that is using it.
For example, CVS/SVN and dummy drivers defines same interface as they are used from gdu and gtkdoc locators:
1 class dummyDriver:
2 def getBranches(self, module)
3 Get a list of branches for a given module. For CVS this is
4 done by listing Tags and Branches for main ChangeLog file in
5 a module and searching for gnome-X-Y branches where X is >=2
6 and Y is even. SVN has native support for listing branches
7 implemented. List of branches can be tweaked with decorators
8 so GEDIT-2-8 can become gnome-2-8, etc.
9
10 def updateFiles(self, files, basedestdir, branch=None)
11 Update list of files located in cache in basedestdir for a
12 branch/tag specified in branch. If branch is None, unstable
13 HEAD/trunk branch is used. Return tuple of list of updated
14 files and list of files that dont exist on source.
15
16 def getLastUpdate(self, files, branch=None)
17 Return a Unix timestamp of last update of a files in a list
18 for a branch/tag specified in branch. If branch is None,
19 unstable HEAD/trunk branch is used. If list is empty, or
20 all files from it are missing on source, 0 is returned.
21
22 def getListing(self, path=None, branch=None)
23 Get list of directories and files in path for branch/tag in
24 branch. If branch is None, unstable HEAD/trunk branch is
25 used.
26
27 def getModules(self, branch=None)
28 Return list of modules for branch/tag in branch. If branch
29 is None, unstable HEAD/trunk branch is used.
Currently, only dummy driver is implemented. Dummy driver is using jhbuild source dir as a replacement for HEAD branch from cvs.gnome.org and can access only files for unstable release. You can notice that dummy driver can work offline, and CVS/SVN drivers will need access to GNOME's CVS/SVN server.
Decorators can be implemented for other locators as criteria and functions for applying decorators are locatator specific. Decorators are located in decorators directory, in modules named in a form of N-name.py where N is a number starting at 1. Number in a prefix enforce decorator loading sequence so cascade decorators can be written. Like locators and drivers, decorators are implemented as classes named after its module without N- prefix and having 'Decorator' suffix.
Decorators must provide interface for loading by implementing following methods:
libgo will load all decorators on initialization, and create instances. There is a function in utils module to get only decorators for a given locator. get_decorators(locator) function will return list of tuples of decorator name, its criteria and instance. Locator now can implement selection/filter function that will return instances of decorators to apply on some docobj and interface how docobj or locator interanal state can be modified. You can refer to section about gdu locator to see how gdu locator is using Makefile.am path as a criteria for selecting decorators and to see which interface is implemented by decorators for gdu locator.
And now something about "Document objects" (or docobjs for short). They are needed to provide abstraction for building documentation. In short, document object have a list of callable functions (builder functions) which is instanced by locators and decorators and that can be called in sequence to do all the dirty work of getting XHTML output in outputdir. One builder function can append another builder function to a list to be called and there is a way to pass some additional data to builder functions.
Document object also have some properties stored (which can be initialized with one of a builder functions) like list of languages, urlname (short ASCII name for urls), release,... Properties of docobj and it's interface are documented in docobj module.
For example, with gdu locator there are two functions for every docobj, one to fetch all files (xml, omf, po, figures,...) using driver and other to call xml2po and xsltproc utilities. First function take a dictionary with Makefile.am variables (DOC_MODULE, DOC_LINGUAS...) and populate docobj's properties and second takes no arguments.
2. gdu locator
(some parts are not implemented)
In short, gdu locators loads list of modules, for every module fetch list of branches (stable branches and unstable HEAD/trunk branch) and load main Makefile.am files for each branch of every module.
Recursively, it load sub directories from Makefile.am's SUBDIR variable and load Makefile.am from them. When all Makefile.am files are loaded, it looks for those using gnome-doc-utils and use info from Makefile to create docobj.
Criteria for decorators is list of globs and selection function load decorator if Makefile.am path matches any glob from a list. Decorators can implement following methods:
1 def modBranches(self, brachs, makefile_path):
2 Return new list of branches for a given main
3 module (special glob '/module/')
4
5 def modSubdirs(self, subdirs, makefile_path, branch):
6 Return new list of subdirs for given Makefile.am
7 Used also for modifing list of modules when
8 special glob '/' is used.
9
10 def modDocobj(self, obj, makefile_path, branch):
11 Return new or updated document object
12
13 def buildDocobj(self, makefile_path, branch):
14 Do not try to build docobj with gdu locator but
15 use object returned from decorator. Only lastest
16 decorator in sequence that is implementing this
17 method is called and only later decorators from
18 a sequence are used for modSubdirs/modDocobj calls
modDocobj and buildDocobj can return None, and if docobj is None after calling all decorators from a sequence docobj is ignored. You can look at implemented decorators to get better picture.
Decorators are implemented to remove modules that does not have main Makefile.am file (using special '/' path in glob criterita) and to remove po directory from list of subdirs (as it does not have main Makefile.am file). Locator can work without theese decorators but will display many unnecesary warnings about missing Makefile.am files.
Decorators that are needed are the one to remove undefined variable from SUBDIR value in Makefile.am (where undefined means that it is not defined in same file or it is defined in a conditional statement) and one for ekiga to replace @PACKAGE_NAME@ with ekiga as DOC_MODULE value. There will be a need for a few more decorators to handle some other special cases.
gdu locator can handle cases when some required file is missing (when it will skip over incorrect document) or when PO file is missing (when it will remove that language from DOC_LINGUAS and continue).
It will rebuilt only docs that are updated or everything if XSLT files or templates for static content are changed from last rebuilt.
3. Releases
As library.gnome.org presents documentation for both lastest and past releases of GNOME platform, docobjs continain information about release for every documentation item. Locators, can implement their own internal representations for releases (gdu is using CVS/SVN tags/branches returned from driver's getReleases(module) method), but when filling general docobj's properites there are some rules:
'release' is tuple of uppercase string representing part of GNOME
platform and indetifier of release. For example valid releases
are ('GNOME','2.14'), ('GTK','2.10')... Archived documentation is
grouped by first value from a tuple and sorted by other. After
selecting one release only documentats and categories for that release
is presented to library.gnome.org users.
'is_lastest' is True if document is from lastest stable release
Documents for lastest releases are displayed by default, merged
together whatever part they belong.
'is_unstable' is True if document is from unstable (HEAD/trunk) branch
4. Localization
(not yet implemented)
Documentation localization is part of GNOME Documentation project and is not an issue to solve using this program. This section is about localization of static content in libgo output like "Annotation" or "Recently updated" strings.
4.1. Localizing stylesheets
Basic styles used are from GNOME Documentation Project and are localized as a part of gnome-doc-utils CVS module.
Additional stylesheets used are localized in a same way to produce l10n.xml file from l10n.xml.in by merging it with po/LANG.po file from libgo CVS module.
4.2. Localizing static content
4.2.1. JavaScript scripts
js/langcookie.js.in # save/delete langauge cookie, display a note
# if language prefs are loaded from cookie
js/annotations.js.in # support for annotations in UI (add new tab under
# table of content and define behaviors)Output is written as js/langcookie.js.LANG.js generated from *.in file merged with po/LANG.po
4.2.2. Templates
tmpl/index.in # main page -> /
# listing one update per channel
tmpl/user.in # main page for user channel -> /user/
tmpl/developer.in # and for developer channel -> /developer/
# listing updates and docs/categories in /
tmpl/user_list.in # cats/subcats/docs listings for users...
tmpl/developer_list.in # ...and for developers
tmpl/user_arch.in # past releases listing for users...
tmpl/developer_arch.in # ...and for developers
Templates are merged with po/LANG.po file to produce localized
templates and are used for creating index files.
5. Creating index files
(not yet implemented)
Index file are rebuilt on every execution of libgo program.
5.1. XML index files
As a part of XSLT transformation, index.LANG.xml files, containing localized name and abstract of a document are generated and located in same dir as XHTML files of a document.
/user/unstable/applets/accessories/clock/clock.sr.xml:
<?xml version="1.0"?>
<name>Упуство за програмче часовника</name>
<abstract>Програмче часовника приказује време и датум на панелу.</abstract>
5.2. List of recently updated documents
After rebuilding updated documents, index files for browsing documentation are rebuilt. First, list of documents to include as recently updated is generated for each language on which at least one document exist. For all languages except English, list of recently updated documents is divided in two parts, one for localized documents and other for not yet localized documents and one olf theese can be empty.
Unix timestamp of last change in a document (stored in docobj) is multiplied with importance factor and list of top 2 documents from each channel is created for English language. Next, for every other language, if there is no localized document in each lists, lastest localized document from a channel (if there is any) is founded and stored. Now, everything is ready for generating index XHTML files.
5.3. XHTML index files
Localized index templates are loaded and info on updated documents (from XML index files) is included and XHTML index files are written to output directory.