NetworkManager Proxy Handling

The Goal

Some networks do not automatically route to the outside world, and using the correct proxies is required in order to have connectivity at all. We need to facilitate this.

Normally, proxy configuration is a per-network thing. Proxy information can be picked up from DHCP, provided by VPN servers, discovered by WPAD or configured manually for a specific connection. (However, some users will instead want to always use a single local proxy regardless of the network; eg, Tor users.)

NetworkManager (and the surrounding environment) should handle this information automatically, and provide a simple way for applications (GNOME and otherwise) to ask the question "which proxy do I use for this URL?"

Often the proxy configuration is provided in the form of a Proxy Auto Config (PAC) file, which contains a JavaScript FindProxyForURL() function to answer precisely that question.

Existing approaches

There are a number of existing pieces to this puzzle, none of them ideal...

1. Environment variables

   There is a long-standing expectation that, especially for command-line tools, one can set environment variables such as http_proxy and https_proxy, to use a specific proxy. This has a number of disadvantages — it is a static configuration which cannot change as network connectivity changes (i.e. moving from one network to another, joining a VPN), and there is no way to reference a more complicated configuration which is expressed with a PAC file.

2. GNOME proxy configuration

   At org.gnome.system.proxy, GNOME keeps its own proxy settings. This can handle PAC files, but still has the problem that it is system-wide, and does not adjust according to the currently-connected network(s). Additionally, the user's proxy settings aren't visible to root processes, so things like yum won't pick up these settings.

3. libproxy

   Libproxy is designed to offer a simple API to applications to answer the what proxy? question. It has numerous back ends for obtaining information — including doing the WPAD lookup for itself, as well as looking in the GNOME configuration or honouring the environment variables, amongst other methods not mentioned here.

   One of the big disadvantages of libproxy is that it loads a JavaScript interpreter for itself behind the scenes, into every process which uses a PAC file. It also re-downloads the PAC file for each request. Many existing applications already have libproxy support, including glib-networking, which goes to the trouble of spawning its own 'glib-pacrunner' process to work around some of the problems mentioned above. (In addition to its implementation issues, libproxy is also not a great API; it has no way of signaling an error to the caller; if it can't determine the proxy settings for whatever reason, it just silently ignores the problem and assumes no proxy is needed.)

4. PacRunner

   PacRunner is a simple standalone dæmon which communicates via D-Bus. It is designed to receive its proxy configuration from a trusted source such as ConnMan or NetworkManager, then simply exposes a method on D-Bus for applications to ask the what proxy? question.

   There exists a back end for the original libproxy, which can be configured as the highest priority and do nothing except query PacRunner. In fact, PacRunner also offers a trivial replacement for the libproxy.so shared library, which has none of libproxy's complexity and which can only do that one thing. These approaches provide compatibility for existing applications which already have libproxy support. Alternatively, applications can query PacRunner directly via D-Bus instead of using the libproxy API.

The Plan

Service side

Firstly, let's look at the system-wide part of the solution...

The handling of proxies within NetworkManager can be modelled on its existing handling of DNS servers. For each network we know the domains, and IP address ranges, for which we use that network — for example we might know that the VPN connection is used for example.com, 172.16.0.0/12 and fec0:0:abcd::/48, so we configure dnsmasq to resolve requests in those ranges to the upstream nameservers provided by the VPN server, while other requests are resolved according to the DNS information received from the local physical network.

This extends to proxy resolution in precisely the same way. For each network, NetworkManager shall inform PacRunner of the domains and IP ranges associated with that network, along with the proxy configuration.

There are a few things we need to do here, to make this work:

1. PacRunner: direct requests to appropriate proxy config

   PacRunner already supports the existence of multiple proxy configurations for different interfaces/domains simultaneously. However, it only ever uses the most recently registered one. We need to fix it so that it will use the proxy config which is appropriate to the request — comparing the host or IP in the URL with the domain or IP range for the registered proxy configurations, and using the correct one.

   Note that although listed first, this does not have to be implemented first. It works for the single-network case already, and those are the users with the biggest problem. After all, if I use a split-tunnel VPN, I probably do that specifically to avoid having to route requests through the insane corporate network and its proxies.

2. NetworkManager: provide proxy information to PacRunner

   Just as we provide DNS information to dnsmasq, we need to provide proxy information to PacRunner. See bug #701824 and this thread on the mailing list.

3. NetworkManager: Add support for WPAD and explicit per-network configuration

   NetworkManager already supports receiving proxy information from DHCP, and it is fairly simple to also receive it from VPN servers which provide it. We will need to add support for doing the WPAD lookups, and also for an explicit manual per-network proxy configuration. The central proxy configuration in dconf should also probably die. (Note: look into security considerations of trusting WPAD; ISTR that it can be spoofed by anyone on the network. So that may have to be opt-in.)

4. gnome-control-center/nm-connection-editor: Add support for per-network proxy configuration
5. gnome-control-center: Figure out what to do with the existing proxy configuration...

6. PacRunner: Improve its libproxy.so replacement library In addition to providing libproxy-compat APIs, it should also provide some better APIs, with error reporting. Additionally, if proxying is not in use at all, or if there is a single proxy being used for all URIs, then it might be nice to have that fact cached in-process, rather than having to make a D-Bus call for every connect() operation.

Client side

Once the above is done, for clients it becomes simple — they can just trust that PacRunner will give the right answers. Applications which already have libproxy support can use that (and distributions should ensure they are shipping the libproxy-pacrunner plugin by default). In glib-networking we should probably query PacRunner directly, and we can deprecate all the old hacks to deal with the pain of the original libproxy.

Once the service side is done and we can trust that doing so will give correct results, distributions can also modify their packaging guidelines to strongly recommend that software shipped as part of the distribution SHOULD automatically use the information provided by PacRunner, or be considered buggy.