Camel.Stream

Camel.Stream is an abstract class which implements read/write streams. There are numerous sub-classes of this, including memory streams, file-based streams, TCP streams, efficient processing streams, etc.

The basic stream functions are pretty well self explanatory:

 ssize_t    camel_stream_read       (CamelStream *stream, char *buffer, size_t n);
 ssize_t    camel_stream_write      (CamelStream *stream, const char *buffer, size_t n);
 int        camel_stream_flush      (CamelStream *stream);
 int        camel_stream_close      (CamelStream *stream);
 gboolean   camel_stream_eos        (CamelStream *stream);
 int        camel_stream_reset      (CamelStream *stream);

Then there are some equally self-explantory utility functions:

 ssize_t camel_stream_write_string (CamelStream *stream, const char *string);
 ssize_t camel_stream_printf (CamelStream *stream, const char *fmt, ... ) G_GNUC_PRINTF (2, 3);
 ssize_t camel_stream_vprintf (CamelStream *stream, const char *fmt, va_list ap);
 ssize_t camel_stream_write_to_stream (CamelStream *stream, CamelStream *output_stream);

The base stream interface is not seekable, but certain subclasses are.

Camel.SeekableStream

This adds some more virtual methods, used for seeking. Again it is an abstract class. This may be a candidate for an ISeekable interface instead.

 off_t    camel_seekable_stream_seek            (CamelSeekableStream *stream, off_t offset,
                                                CamelStreamSeekPolicy policy);
 off_t   camel_seekable_stream_tell            (CamelSeekableStream *stream);
 int     camel_seekable_stream_set_bounds      (CamelSeekableStream *stream, off_t start, off_t end);

Seekable streams may be bounded, that is work over a smaller sub-set of a bigger file/whatever. There is an older class SeekableSubstream which should not be used anymore, which used to provide this functionality in yet another subclass.

Streams of Streams

Certain stream classes need other streams to work on, and provide additional processing.

Camel.StreamBuffer

This is a stream which provides buffering for another stream.

 CamelStream *camel_stream_buffer_new (CamelStream *stream,
                                      CamelStreamBufferMode mode);
 CamelStream *camel_stream_buffer_new_with_vbuf (CamelStream *stream,
                                                CamelStreamBufferMode mode,
                                                char *buf, guint32 size);
 
 int camel_stream_buffer_gets (CamelStreamBuffer *sbf, char *buf, unsigned int max);
 
 char *camel_stream_buffer_read_line (CamelStreamBuffer *sbf);

The gets function works like fgets.

read_line is simpler in that it always returns a full string up to the next newline, automatically converting \r\n into \n and stripping both. However, for some inane reason it returns a duplicated string rather than a pointer ot the internal buffer, which would be more efficient.

All other operations like <code>read</code> or <code>write</code> are automatically buffered through virtual method overrides.

Camel.StreamFilter

This is a sophisticated stream which can perform efficient, multi-stage, pipelined processing on a stream of data, in a read or write direction.

 CamelStreamFilter      *camel_stream_filter_new_with_stream    (CamelStream *stream);
 int camel_stream_filter_add    (CamelStreamFilter *stream, CamelMimeFilter *filter);
 void camel_stream_filter_remove        (CamelStreamFilter *stream, int id);

Each Camel.MimeFilter added to the stream will be called in turn on each block of data read/written to the stream, with the result returned to the caller or written to the output stream.

There are a multitude of Camel.MimeFilter classes implemented, that lets code stream-decode everything from base64 encoding, canonicalise line-endings, do character set conversion and more.

Physical Streams

These are concrete classes which implement physical streams.

Camel.StreamNull

This is a do-nothing stream, which is always at the end of stream, and successfully reads and writes any data sent to it. Apart from a sink for a stream processor, it can be used to count the number of bytes written to it.

Camel.StreamFs

This rather badly named stream is used to access files. Either by name or by file descriptor.

This is a seekable stream, with equally badly named constructors, whose named versions work using more or less posix open(2) semantics.

 CamelStream * camel_stream_fs_new_with_name            (const char *name, int flags, mode_t mode);
 CamelStream * camel_stream_fs_new_with_name_and_bounds (const char *name, int flags, mode_t mode,
                                                        off_t start, off_t end);
 
 CamelStream * camel_stream_fs_new_with_fd              (int fd);
 CamelStream * camel_stream_fs_new_with_fd_and_bounds   (int fd, off_t start, off_t end);

Camel.StreamMem

The well-abused memory based stream. All data is read from and written to a GByteArray memory buffer.

 CamelStream *camel_stream_mem_new(void);
 CamelStream *camel_stream_mem_new_with_byte_array(GByteArray *buffer);
 CamelStream *camel_stream_mem_new_with_buffer(const char *buffer, size_t len);

It is important to note, that if you just want to write to memory, access that memory then discard the lot, you don't need to use the new_with_byte_array function. It is also important to note that this function will own the GByteArray after it is called, so it should really only be used for pre-filling an existing stream from an existing GByteArray, the same way new_with_buffer is used, but without the copying overhead.

 void camel_stream_mem_set_secure(CamelStreamMem *mem);

Just clears the memory stream on finalise, rather than leaving the memory around. Some, but not much, extra security.

 void camel_stream_mem_set_byte_array(CamelStreamMem *mem, GByteArray *buffer);
 void camel_stream_mem_set_buffer(CamelStreamMem *mem, const char *buffer, size_t len);

These functions are only really for implementations. See the api documentation for caveats about using them and their strange behaviour.

Camel.StreamProcess

This is a popen() like stream, although it provides bi-directional communication with the sub-process. It will execute a command in a shell, and provide read/write access to it's stdin and stdout.

 int camel_stream_process_connect (CamelStreamProcess *stream, const char *command, const char **env);

This was a special-purpose stream created just for an IMAP feature. It would have been better had a more generic process object been created with the ability to access other file descriptors than stdin and stdout.

It should probably not be used in other code.

This is part of libcamel-provider.

TCP Streams

These streams are implemented in libcamel-provider.

Camel.TcpStream

Note: should be TCPStream

This is an abstract class which provides additional interfaces for working with TCP type sockets.

 int         camel_tcp_stream_connect    (CamelTcpStream *stream, struct addrinfo *host);
 int         camel_tcp_stream_getsockopt (CamelTcpStream *stream, CamelSockOptData *data);
 int         camel_tcp_stream_setsockopt (CamelTcpStream *stream, const CamelSockOptData *data);
 
 struct sockaddr *camel_tcp_stream_get_local_address  (CamelTcpStream *stream, socklen_t *len);
 struct sockaddr *camel_tcp_stream_get_remote_address (CamelTcpStream *stream, socklen_t *len);

Camel.TcpStreamRaw

Note: should be TCPStreamRaw

This is a sub-class of TcpStream which implements native posix, unencrypted socket connections.

Camel.TcpStreamSSL

Note: should be TCPStreamSSL

This is a sub-class of TcpStream that implements SSL streams, using Mozilla's NSS library.

 CamelStream *camel_tcp_stream_ssl_new(struct _CamelSession *session, const char *expected_host, guint32 flags);

A second interface is provided for accessing a normal stream, which may later be changed into an SSL stream. i.e. this is used to implement TLS.

 CamelStream *camel_tcp_stream_ssl_new_raw(struct _CamelSession *session, const char *expected_host, guint32 flags);
 int camel_tcp_stream_ssl_enable_ssl(CamelTcpStreamSSL *ssl);

Camel.HttpStream

This badly named (should be HTTPStream) stream implements a trivial HTTP/1.0 protocol stream, over a TCP or SSL stream. It is not a sub-class of TcpStream, but uses one as a source.

 CamelStream *camel_http_stream_new(CamelHttpMethod method, struct _CamelSession *session, CamelURL *url);
 
 void camel_http_stream_set_user_agent(CamelHttpStream *http_stream, const char *user_agent);
 void camel_http_stream_set_proxy(CamelHttpStream *http_stream, const char *proxy_url);
 void camel_http_stream_set_proxy_authrealm(CamelHttpStream *http_stream, const char *proxy_authrealm);
 void camel_http_stream_set_proxy_authpass(CamelHttpStream *http_stream, const char *proxy_authpass);
 
 CamelContentType *camel_http_stream_get_content_type(CamelHttpStream *http_stream);

Only the GET and HEAD HTTP methods are supported. It will automatically make the correct connection type based on the url definition, and the proxy settings.

Progress and Cancellation

Through Camel.Operation the progress of operations and the cancellation of operations is achieved. All of the above streams which may go into wait states implement cancellation processing. This involves waiting using a poll, or select, on both the socket or file descriptor and a cancellation file descriptor. Some may also implement progress reporting, where it makes sense, although often useful information is not available at the stream level.

This is important so that long-running or hung tasks are not cancellable from the application.

Exceptions

None of the stream interfaces take Camel.Exception pointers. This was because of the naive assumption that errno error codes would suffice and simplify error processing.

It doesn't, but for now these interfaces do not take exceptions. Perhaps that is something to consider in the future.