write docs, all in the VCC source. README.rst is now a symlink to

the generated .man.rst document.

README.rst

-============
-vmod_convert
-============
-
-----------------------
-Varnish Example Module
-----------------------
-
-:Date: 2015-03-03
-:Version: 1.0
-:Manual section: 3
-
-SYNOPSIS
-========
-
-import convert;
-
-DESCRIPTION
-===========
-
-Example Varnish vmod demonstrating how to write an out-of-tree Varnish vmod.
-
-Implements the traditional Hello World as a vmod.
-
-FUNCTIONS
-=========
-
-hello
------
-
-Prototype
-        ::
-
-                hello(STRING S)
-Return value
-	STRING
-Description
-	Returns "Hello, " prepended to S
-Example
-        ::
-
-                set resp.http.hello = convert.hello("World");
-
-INSTALLATION
-============
-
-The source tree is based on autotools to configure the building, and
-does also have the necessary bits in place to do functional unit tests
-using the ``varnishtest`` tool.
-
-Building requires the Varnish header files and uses pkg-config to find
-the necessary paths.
-
-Usage::
-
- ./autogen.sh
- ./configure
-
-If you have installed Varnish to a non-standard directory, call
-``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
-the appropriate path. For convert, when varnishd configure was called
-with ``--prefix=$PREFIX``, use
-
- PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
- export PKG_CONFIG_PATH
-
-Make targets:
-
-* make - builds the vmod.
-* make install - installs your vmod.
-* make check - runs the unit tests in ``src/tests/*.vtc``
-* make distcheck - run check and prepare a tarball of the vmod.
-
-In your VCL you could then use this vmod along the following lines::
-
-        import convert;
-
-        sub vcl_deliver {
-                # This sets resp.http.hello to "Hello, World"
-                set resp.http.hello = convert.hello("World");
-        }
-
-COMMON PROBLEMS
-===============
-
-* configure: error: Need varnish.m4 -- see README.rst
-
-  Check if ``PKG_CONFIG_PATH`` has been set correctly before calling
-  ``autogen.sh`` and ``configure``

README.rst

+src/vmod_blobcode.man.rst
\ No newline at end of file

src/vmod_blobcode.vcc

+#-
+# Copyright (c) 2015 UPLEX Nils Goroll Systemoptimierung
+# All rights reserved
 #
-# Copyright YOU (c) 1900.
+# Authors: Nils Goroll <nils.goroll@uplex.de>
+#          Geoffrey Simmons <geoffrey.simmons@uplex.de>
 #
-# (vmodtool requires this format.)
+# See LICENSE
 #
 
 $Module blobcode 3 binary-to-text encodings and decodings for the VCL blob type
 
+::
+
+  sub vcl_init {
+      # Create blob objects from encodings such as base64 or hex.
+      new myblob   = blobcode.blob(decoding=BASE64, encoded="Zm9vYmFy");
+      new yourblob = blobcode.blob(decoding=HEX, encoded="666F6F");
+  }
+
+  sub vcl_recv {
+      # The .get() method retrieves the BLOB from an object.
+      set req.http.MyBlob-As-Hex
+          = blobcode.encode(encoding=HEX, blob=myblob.get());
+
+      # The .encode() method efficiently retrieves an encoding.
+      set req.http.YourBlob-As-Base64 = yourblob.encode(BASE64);
+
+      # decode() and encode() functions convert blobs to text and
+      # vice versa at runtime.
+      set req.http.Base64-Encoded
+          = blobcode.encode(BASE64,
+                            blobcode.decode(HEX, req.http.Hex-Encoded));
+
+      # transcode() converts from one encoding to another.
+      set req.http.Hex-Encoded
+          = blobcode.transcode(decoding=BASE64, encoding=HEX, "YmF6");
+  }
+
+
 DESCRIPTION
 ===========
 
-XXX write doc
-XXX gen ENUMs
+This Varnish module (VMOD) supports binary-to-text encodings and
+decodings for the VCL data type BLOB, which may contain arbitrary data
+of any length. Currently (as of Varnish 4.1.1), BLOBs may only be used
+as arguments of VMOD functions; so this module is meant to facilitate
+the use of other VMODs.
+
+ENCODING SCHEMES
+================
+
+Encoding schemes are specified by ENUMs in the VMOD's constructor,
+methods and functions. Decodings convert a (possibly concatenated)
+string into a blob, while encodings convert a blob into a string.
+
+ENUM values for a decoding can be one of:
+
+* ``IDENTITY``
+* ``BASE64``
+* ``BASE64URL``
+* ``BASE64URLNOPAD``
+* ``HEX``
+
+An encoding can be one of:
+
+* ``IDENTITY``
+* ``BASE64``
+* ``BASE64URL``
+* ``BASE64URLNOPAD``
+* ``HEXUC``
+* ``HEXLC``
+
+Empty strings are decoded into a "null blob" (of length 0),
+and conversely a null blob is encoded as the empty string.
+
+IDENTITY
+--------
+
+The simplest encoding converts between the BLOB and STRING data types,
+leaving the contents byte-identical.
+
+Note that a BLOB may contain a null byte at any position before its
+end; if such a BLOB is decoded with IDENTITY, the resulting STRING
+will have a null byte at that position. Since VCL strings, like C
+strings, are represented with a terminating null byte, the string will
+be truncated, appearing to contain less data than the original
+blob. For example::
+
+  # Decode from the hex encoding for "foo\0bar".
+  # The header will be seen as "foo".
+  set req.http.Trunced-Foo
+    = blobcode.encode(encoding=IDENTITY,
+                      blobcode.decode(decoding=HEX, "666f6f00626172"));
+
+BASE64*
+-------
+
+The base64 encoding schemes use 4 characters to encode 3 bytes. There
+are no newlines or maximal line lengths -- whitespace is not
+permitted.
+
+The ``BASE64`` encoding uses the alphanumeric characters, ``+`` and
+``/``; and encoded strings are padded with the ``=`` character so that
+their length is always a multiple of four.
+
+The ``BASE64URL`` encoding also uses the alphanumeric characters, but
+``-`` and ``_`` instead of ``+`` and ``/``, so that an encoded string
+can be used safely in a URL. This scheme also uses the padding
+character ``=``.
+
+The ``BASE64URLNOPAD`` encoding uses the same alphabet as
+``BASE6URL``, but leaves out the padding. Thus the length of an
+encoding with this scheme is not necessarily a mutltiple of four.
+
+HEX*
+----
+
+The ``HEX`` decoding converts a hex string, which may contain upper-
+or lowercase characters for hex digits ``A`` through ``f``, into a
+blob. The ``HEXUC`` or ``HEXLC`` encodings convert a blob into a hex
+string with upper- and lowercase digits, respectively. A prefix such
+as ``0x`` is not used for any of these schemes.
+
+If a hex string to be decoded has an odd number of digits, it is
+decoded as if a ``0`` is prepended to it; that is, the first digit is
+interpreted as representing the least significant nibble of the first
+byte. For example::
+
+  # The concatenated string is "abcdef0", and is decoded as "0abcdef0".
+  set req.http.First = "abc";
+  set req.http.Second = "def0";
+  set req.http.Hex-Decoded
+      = blobcode.encode(IDENTITY, blobcode.decode(HEX,
+                        req.http.First + req.http.Second));
 
 $Event event
 
-$Object blob(ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX} decoding, STRING_LIST encoded)
+$Function BLOB decode(ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD,
+                            HEX} decoding, STRING_LIST encoded)
+
+Description
+	Returns the BLOB derived from the string ``encoded``
+        according to the scheme specified by ``decoding``.
+
+Example
+        ``blobcode.decode(decoding=BASE64, encoded="Zm9vYmFyYmF6")``
+
+$Function STRING encode(ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD,
+                              HEXUC, HEXLC} encoding, BLOB blob)
+
+Description
+	Returns a string representation of the BLOB ``blob`` as
+	specifed by ``encoding``.
+
+Example
+	``set req.http.Hex = blobcode.encode(encoding=HEXLC,``
+                                ``blob=blobcode.decode(BASE64, "Zm9vYmFyYmF6");``
+
+$Function STRING transcode(ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD,
+                                 HEX} decoding,
+                           ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD,
+                                 HEXUC, HEXLC} encoding,
+                           STRING_LIST encoded)
+
+Description
+	Translates from one encoding to another, by first decoding
+	the string ``encoded`` according to the scheme ``decoding``,
+	and then returning the encoding of the resulting blob
+        according to the scheme ``encoding``.
+
+Example
+	``set req.http.Hex2Base64 = blobcode.transcode(decoding=HEX,``
+                                    ``encoding=BASE64, encoded="666f6f");``
+
+$Object blob(ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX} decoding,
+             STRING_LIST encoded)
+
+Prototype
+	new OBJ = blobcode.blob(ENUM decoding, STRING_LIST encoded)
+
+Description
+	Creates an object that contains the BLOB derived from the
+	string ``encoded`` according to the scheme ``decoding``.
+
+Example
+	``new theblob = blobcode.blob(BASE64, "YmxvYg==");``
 
 $Method BLOB .get()
 
-$Method STRING .encode(ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEXUC, HEXLC} encoding)
+Description
+	Returns the BLOB created by the constructor.
+
+Example
+	``set resp.http.The-Blob``
+	    ``= blobcode.encode(IDENTITY, theblob.get());``
+
+$Method STRING .encode(ENUM {IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEXUC,
+                             HEXLC} encoding)
+
+Description
+	Returns an encoding of BLOB created by the constructor,
+	according to the scheme ``encoding``.
+
+Example
+	``set req.http.The-Blob = theblob.encode(IDENTITY);``
+
+For any ``blob`` object and encoding ``ENC``, encodings via the ``.encode()``
+method and the ``encode()`` function are equal::
+
+  # Always true:
+  blobcode.encode(ENC, blob.get()) == blob.encode(ENC)
+
+But the object method is more efficient -- the encoding is computed
+once and cached (with allocation in heap memory), and the cached
+encoding is retrieved on every subsequent call. The ``encode()``
+function computes the encoding on every call, allocating space for the
+string in Varnish workspaces.
+
+So if the data in a BLOB are fixed at VCL initialization time, so that
+its encodings will always be the same, it is better to create a
+``blob`` object. The VMOD's functions should be used for data that are
+not known until runtime.
+
+ERRORS
+======
 
-$Function BLOB decode(ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX} decoding, STRING_LIST encoded)
-XXX DOC
-$Function STRING encode(ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEXUC, HEXLC} encoding, BLOB blob)
-XXX DOC
-$Function STRING transcode(ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEX} decoding, ENUM { IDENTITY, BASE64, BASE64URL, BASE64URLNOPAD, HEXUC, HEXLC} encoding, STRING_LIST encoded)
-XXX DOC
+The encoders and decoders may fail if there is insufficient space to
+create the new blob or string. Decoders may also fail if the encoded
+string is an illegal format for the decoding scheme.
+
+If the ``blob`` object constructor fails, then the VCL program will
+fail to load, and the VCC compiler will emit an error message.
+
+The VMOD functions -- ``encode()``, ``decode()`` and ``transcode()``
+-- may not be called in ``vcl_init()`` or ``vcl_fini()`` (since a
+workspace must be available). Use in ``vcl_init()`` will also cause
+the VCL program to fail with a VCC compiler error.
+
+If any of the VMOD's methods or functions fail at runtime, then an
+error message will be written to the Varnish log using the tag
+``VCL_Error``. The encoders and decoders return ``NULL`` on failure;
+this means that, for example, if the return value was to be assigned
+to a header, then the header will not be set.
 
 REQUIREMENTS
 ============
-Requires commit 84a2903805580973c59635bacb99df5da901c02c in varnish (fix
-for trac issue #1844).
+
+This VMOD requires a Varnish build with commit 84a2903 (bugfix for
+issue #1844), which at the time of writing is not yet available in a
+released version of Varnish. So for the time being, it should be built
+against and used with the master branch.
+
+INSTALLATION
+============
+
+The VMOD is built against a Varnish installation, and the autotools
+use ``pkg-config(1)`` to locate the necessary header files and other
+resources. This sequence will install the VMOD::
+
+  > ./autogen.sh	# for builds from the git repo
+  > ./configure
+  > make
+  > make check		# to run unit tests in src/tests/*.vtc
+  > make distcheck	# run check and prepare a distribution tarball
+  > sudo make install
+
+If you have installed Varnish in a non-standard directory, call
+``autogen.sh`` and ``configure`` with the ``PKG_CONFIG_PATH``
+environment variable pointing to the appropriate path. For example,
+when varnishd configure was called with ``--prefix=$PREFIX``, use::
+
+  > PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
+  > export PKG_CONFIG_PATH
+
+By default, the vmod ``configure`` script installs the vmod in
+the same directory as Varnish, determined via ``pkg-config(1)``. The
+vmod installation directory can be overridden by passing the
+``VMOD_DIR`` variable to ``configure``.
+
+Other files such as this man-page are installed in the locations
+determined by ``configure``, which inherits its default ``--prefix``
+setting from Varnish.
+
+LIMITATIONS
+===========
+
+The VMOD allocates memory in various ways for new blobs and
+strings. The ``blob`` object and its methods allocate memory from the
+heap, and hence they are only limited by available virtual memory.
+
+The ``encode()`` and ``decode()`` functions allocate Varnish
+workspace.  If these functions are failing, as indicated by "out of
+space" messages in the Varnish log (with the ``VCL_Error`` tag), then
+you will need to increase the varnishd parameters ``workspace_client``
+and/or ``workspace_backend``.
+
+The ``transcode()`` function also allocates workspace for the string
+that it returns, and it allocates space on the stack for a temporary
+BLOB. If this function causes stack overflow, you may need to increase
+the stack size for the varnishd process, for example with ``ulimit
+-s``.
+
+AUTHORS
+=======
+
+* Geoffrey Simmons <geoff@uplex.de>
+* Nils Goroll <nils.goroll@uplex.de>
+
+UPLEX Nils Goroll Systemoptimierung
+
+This VMOD was originally adapted from, and is heavily indebted to, the
+digest VMOD by Kristian Lyngstøl.
+
+HISTORY
+=======
+
+* version 0.1: initial version
+
+SEE ALSO
+========
+
+* varnishd(1)
+* vcl(7)
+* VMOD digest: https://github.com/varnish/libvmod-digest
+
+COPYRIGHT
+=========
+
+This document is licensed under the same conditions as the
+libvmod-blobcode project. See LICENSE for details.
+
+* Copyright (c) 2015-2016 UPLEX Nils Goroll Systemoptimierung