Add info about namespace/api management

doc/sphinx/phk/apispaces.rst

+.. _phk_api_spaces:
+
+API spaces
+==========
+
+The reason you cant remember hearing about "API spaces" at university
+is that I just made that concept up, as a title for this piece.
+
+We need a name for the collision where APIs meet namespaces.
+
+At some point in their career, most C programmers learn that ``j0``,
+``j1``, ``y0`` and ``y1`` are names to avoid whereas ``j2`` and
+``y2`` up to, but not including ``jn`` and ``yn`` are OK.
+
+The reason is that somebody back when I was a child thought it would
+be really neat if the math library supported Bessel functions,
+without thinking about ``<math.h>`` as an API which had to coexist
+in the flat namespace of the C language along many other APIs.
+
+One of the big attractions of Object Oriented programming is that
+it solves exactly that problem:
+Nobody is confused about ``car->push()`` and ``stack->push()``.
+
+But Varnish is written in C which has a flat namespace and we must
+live with it.
+
+From the very start, we defined cadastral boundaries in the flat
+namespace by assigning VTLA prefixes to various chunks of code.
+
+``VSB_something`` has to do with the sbufs we adopted from FreeBSD,
+``VGC_something`` is Vcc Generated C-source and so on.
+
+Mostly we have stuck with the 'V' prefix, which for some reason
+is almost unused everywhere else, but we also have prominent
+exceptions.  ``WS_something`` for workspaces for instance.
+
+As long as all the C-code was in-project, inconsistencies and
+the precise location of function prototypes didn't matter much,
+it was "just something you had to know".
+
+Now that we have VMODs, and even more so, now that we want to provide
+some semblance of API stability for VMODs, we have a lot of sorting
+and some renaming to do, in order to clearly delineate APIs within
+our flat namespace and our include files.
+
+Frederick P. Brooks points out in his classic "The Mythical Man-Month",
+that is the difference between a program-product and a programming-product,
+and he makes the case that the effort required tripples, going from
+the former to the latter.
+
+Having spent some weeks on what I thought would be a three day task
+I suspect that his was an underestimate.
+
+I will now try to lay out what I think will be our policy on APIs
+and name-space sharing going forward, but please understand that
+this is mostly just an aspirational goal at this point.
+
+General namespace rules
+-----------------------
+
+1. Each API or otherwise isolated section of code gets a unique
+   prefix, ending with an underscore, (``VSB_``, ``V1F_`` etc.)
+
+2. Public symbols has upper case prefix.
+
+3. Private symbols use prefix in lower case, both as ``static``
+   symbols in source files, and when exposed to other source
+   files in the same section of code.
+
+4. Friends-With-Benefit symbols have an additional underscore
+   after the prefix:  ``FOO__bar()`` and are only to be used with
+   explicit permission, which should be clearly documented in
+   the relevant #include file.
+
+VMOD API/ABI levels
+-------------------
+
+Vmods can be written against one of three API/ABI levels, called
+respectively ``VRT``, ``PACKAGE`` and ``SOURCE``, defined in
+detail below.
+
+A VMOD which restricts itself to the ``VRT`` API/ABI gets maximum
+stability and will, we hope, work without recompilation across
+many major and minor releases of Varnish.
+
+A VMOD which uses the ``PACKAGE`` API, will likely keep working
+across minor releases of varnish releases, but will usually
+need to be recompiled for new major releases of varnish.
+
+A VMOD which uses the ``SOURCE`` API is compiled against one
+specific version of Varnish, and will not work with another
+version until recompiled.
+
+The VMOD VRT API/ABI
+--------------------
+
+This API space could also have been called 'inline', because it
+is basically what you see in the C-source generated by VCC:
+
+| Include files allowed:
+| 
+| 	``#include "vdef.h"``
+| 	``#include "vrt.h"``
+| 	``#include "vrt_obj.h"``
+| 	``#include "vcl.h"``
+
+Any private and Friends-With-Benefits symbols are off-limits
+to VMODs, (it is usually stuff VCC needs for the compiled
+code, and likely as not, you will crash if you mess with it.)
+
+The ``"vrt.h"`` contains two #defines which together defines
+the level of this API:
+
+| 	``#define VRT_MAJOR_VERSION       6U``
+| 	``#define VRT_MINOR_VERSION       2U``
+
+A snapshot of these will be automatically compiled into the
+VMOD shared library, and they will be checked for compatibility
+when the VMOD is imported by the VCL compiler.
+
+The VMOD PACKAGE API/ABI
+------------------------
+
+This API space provides access to everything in the ``VRT`` API
+space plus the other exposed and supported APIs in varnishd.
+
+| Include files allowed:
+| 
+|	``#include "cache.h"		// NB: includes vdef.h and vrt.h``
+|	``#include "cache_backend.h"``
+|	``#include "cache_director.h"``
+|	``#include "cache_filter.h"``
+|	``#include "waiter/waiter.h"``
+
+Any private and Friends-With-Benefits symbols are off-limits
+to VMODs.
+
+In addition to the two-part VRT version, ``"cache.h"`` will
+contain two #defines for levels of this API.
+
+| 	``#define PACKAGE_MAJOR_VERSION       1U``
+| 	``#define PACKAGE MINOR_VERSION       3U``
+
+Compile-time snapshots of these will be checked, along with
+their VRT cousins be checked for compatibility on VMOD import.
+
+The VMOD SOURCE API/ABI
+-----------------------
+
+This API space provides access to private parts of varnishd and its
+use is highly discouraged, unless you absolutely have to,
+
+You can #include any file from the varnish source tree and use
+anything you find in them - but don't come crying to us if it
+all ends in tears:  No refunds at this window.
+
+A hash value of all the .h files in the source tree will be
+compiled into the VMOD and will be checked to match exactly
+on VMOD import.
+
+*phk*

doc/sphinx/phk/index.rst

@@ -8,6 +8,7 @@ You may or may not want to know what Poul-Henning thinks.
 .. toctree::
 	:maxdepth: 1
 
+	apispaces.rst
 	VSV00001.rst
 	somethinghappened.rst
 	trialerror.rst