Check in generated man page RST for online consumption

.gitignore

@@ -30,7 +30,7 @@ stamp-h1
 # vmodtool
 
 vcc_*_if.[ch]
-vmod_*.rst
+vmod_j.rst
 
 # man
 

src/vmod_j.man.rst

+..
+.. NB:  This file is machine generated, DO NOT EDIT!
+..
+.. Edit ./vmod_j.vcc and run make instead
+..
+
+.. role:: ref(emphasis)
+
+======
+vmod_j
+======
+
+-----------------------------------------
+A JSON formatter for VCL which sucks less
+-----------------------------------------
+
+:Manual section: 3
+
+SYNOPSIS
+========
+
+.. parsed-literal::
+
+  import j [as name] [from "path"]
+  
+  STRING null()
+  
+  STRING true()
+  
+  STRING false()
+  
+  STRING nil()
+  
+  STRING number([INT integer], [REAL real], [BYTES bytes], [DURATION duration], [TIME time], [STRING string])
+  
+  STRING string(STRING)
+  
+  STRING array(STRING)
+  
+  STRING object(STRING)
+  
+
+INTRODUCTION
+============
+
+**BEFORE USING THIS VMOD, DO READ THIS DOCUMENTATION AND IN PARTICULAR
+THE** `WARNING`_.
+
+.. _JSON: https://www.json.org/json-en.html
+
+Formatting `JSON`_ in pure VCL is a PITA, because string processing in
+VCL was never made for it. VCL being a Domain Specific Language, it
+was made for processing HTTP headers.
+
+Consider this simple example of a JSON object with a single key
+``key``, for which the value ``value`` is to be taken from a VCL
+header variable::
+
+  {"key":"value"}
+
+in pure VCL, you have to write something like this::
+
+  {"{"key":""} + req.http.value + {"""} + "}"
+
+Applause if you do not lose your mental sanity trying to understand
+what this does.
+
+This vmod has been written because it drove the author crazy to
+maintain VCL code with constructs like the above (and that's an
+exceptionally trivial case).
+
+DESCRIPTION
+===========
+
+This VMOD is called *A JSON formatter for VCL which sucks less* in the
+hope to provide to VCL authors a tool which contributes to their
+mental well being. As we will see, for the time being, it still sucks,
+just *less*.
+
+With VMOD *j*, the example above looks like this::
+
+  j.object("key" + req.http.value)
+
+Notice how the concatenation operator ``+`` is "repurposed" for
+separating arguments in lack of support for variable arguments. This
+is one of the two cases of `creative` use of the Varnish VRT API, see
+`IMPLEMENTATION NOTES`_ for more details.
+
+Things get more interesting with more complex data
+structures. Consider this toy example::
+
+    j.object(
+	j.str("A") + j.null() +
+	"B" + j.object(
+	    "BB" + j.number(string="42.42e42") +
+	    "CC" + false +
+	    "DD" + true) +
+	"C" + j.array("A" + 2 + j.object(j.nil()))
+    )
+
+The JSON produced by this code looks like this when reformatted with
+:ref:`jq(1)`::
+
+	{
+	  "A": null,
+	  "B": {
+	    "BB": 4.242e+43,
+	    "CC": false,
+	    "DD": true
+	  },
+	  "C": [
+	    "A",
+	    2,
+	    {}
+	  ]
+	}
+
+This example should contain almost all of the JSON syntax there is:
+
+An outer object
+
+* with a key ``A`` and no value,
+
+* a key ``B`` whose value is another object
+
+  * with the keys ``AA``, ``BB`` and ``CC`` having as values a number
+    and the booleans ``false`` and ``true``,
+
+* and a key ``C``, whose value is an array containing as elements a
+  string ``A``, the number 2 and an empty object.
+
+.. _wrapper functions:
+
+Simple JSON types
+-----------------
+
+The safe way to create the simple JSON types is to always use the
+respective functions:
+
+* `j.null()`_ to create a *null* value
+
+* `j.true()`_ to create a *true* boolean
+
+* `j.false()`_ to create a *false* boolean
+
+* `j.number()`_ to create a number
+
+* `j.string()`_ to create a string
+
+As seen by VCL, all JSON types are of the STRING type, but to *j*
+functions, they have additional type information, see `IMPLEMENTATION
+NOTES`_ for details.
+
+Auto typing
+~~~~~~~~~~~
+
+Alternative to the type functions above, native VCL expressions can be
+used in many cases. As seen by VMOD *j*, all arguments to its
+functions are strings as the result of *string folding* from the
+original data types. VMOD *j* treats these strings as JSON types if
+they look like them:
+
+* ``"true"`` and ``"false"`` are considered boolean
+
+* anything which looks like a number is considered a number
+
+* anything else is considered a string.
+
+So, for example, if you want to use a *JSON string* ``"true"``, use
+``j.string(true)`` instead of ``"true"`` or ``true`` (which gets
+folded into ``"true"`` by varnish).
+
+Complex JSON types
+------------------
+
+`j.object()`_ creates objects. It always needs to be given an even
+number of "arguments" (concatenated strings). Each even numbered
+"argument" needs to be a JSON-string, so either be produced by
+`j.string()`_ or treated like one by the rules given above.
+
+Due to the VRT interface, the `j.nil()`_ argument needs to be used to
+create an empty object: ``j.object(j.nil())`` creates ``{}``.
+
+`j.array()`_ creates arrays. A `j.nil()`_ argument creates an empty
+array. See above for pitfalls.
+
+Unwrapped arguments to `j.object()`_ and `j.array()`_
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Arguments to `j.object()`_ and `j.array()`_ which are not passed
+through the `wrapper functions`_ (literal strings, VCL variables or
+other VMOD functions) are always interpreted as one of
+
+* *boolean*,
+
+* *number* or
+
+* *string*,
+
+but never as complex types.
+
+For example ``j.object("key" + "[1,2,3]")`` creates ``{"key":
+"[1,2,3]"}`` because the value is taken as a string.
+
+To create ``{"key": [1,2,3]}``, use ``j.object("key" +
+j.array(j.number(1) + j.number(2) + j.number(3)))``.
+
+Pitfalls due to repurpose of the ``+`` operator
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Our `creative` use of concatenated strings (STRANDS) as variable
+arguments fails when varnish combines strings at compile time or when
+the ``+`` operator is interpreted as arithmetic.
+
+For example, ``j.array(1 + 2 + 3)`` produces ``[6]`` and
+``j.array("1" + "2" + "3")`` produces ``[123]``.
+
+This VMOD might suck less, but it still sucks, until, maybe, the VRT
+interface gets more powerful.
+
+As mentioned above, to create an array of three numbers (``[1,2,3]``),
+use ``j.array(j.number(1) + j.number(2) + j.number(3))`` and to create
+an array of three strings (``["1","2","3"]``) use
+``j.array(j.string(1) + j.string(2) + j.string(3))``
+
+VMOD INTERFACE REFERENCE
+========================
+
+.. _j.null():
+
+STRING null()
+-------------
+
+Returns a JSON *null* value.
+
+.. _j.true():
+
+STRING true()
+-------------
+
+Returns a JSON *true* boolean.
+
+.. _j.false():
+
+STRING false()
+--------------
+
+Returns a JSON *false* boolean.
+
+.. _j.nil():
+
+STRING nil()
+------------
+
+Use this as an argument to create an empty `j.array()`_ or
+`j.object()`_.
+
+.. _j.number():
+
+STRING number([INT integer], [REAL real], [BYTES bytes], [DURATION duration], [TIME time], [STRING string])
+-----------------------------------------------------------------------------------------------------------
+
+::
+
+   STRING number(
+      [INT integer],
+      [REAL real],
+      [BYTES bytes],
+      [DURATION duration],
+      [TIME time],
+      [STRING string]
+   )
+
+Return a JSON number created from exactly one of the optional
+arguments. Calling this function with more than one argument triggers
+a VCL failure.
+
+* For the *integer* and *real* arguments, the returned number is the
+  same as the argument, formatted according to the JSON requirements,
+
+* for *bytes*, the number of bytes,
+
+* for *duration*, the number of seconds,
+
+* for *time*, the number of seconds since 00:00 UTC on January 1st,
+  1970 ("UNIX epoch"),
+
+* and for *string*, it is just that string, if it conforms to JSON
+  formatting.
+
+ALIAS num()
+-----------
+
+Deprecated alias for ``number()``.
+
+.. _j.string():
+
+STRING string(STRING)
+---------------------
+
+Return the argument as a JSON string with quotes added and special
+characters escaped appropriately.
+
+ALIAS str()
+-----------
+
+Deprecated alias for ``string()``.
+
+.. _j.array():
+
+STRING array(STRING)
+--------------------
+
+Return a JSON array with the constituents of a concatenation argument
+(strands) as elements.
+
+Examples:
+
+* to create ``[]``::
+
+    j.array(j.nil())
+
+* to create ``[1]``::
+
+    j.array(j.number(1))
+
+* to create ``["A", 42]``::
+
+    j.array(j.string("A") + j.number(42))
+
+ALIAS arr()
+-----------
+
+Deprecated alias for ``array()``.
+
+.. _j.object():
+
+STRING object(STRING)
+---------------------
+
+Return a JSON object with pairs of the constituents of a
+concatenation argument (strands) as key/value pairs.
+
+* to create ``{}``::
+
+    j.array(j.nil())
+
+* to create ``{"A":42}``::
+
+    j.object(j.string("A") + j.number(42))
+
+* to create ``{"A":42,"B":true}``::
+
+    j.object(
+      j.string("A") + j.number(42) +
+      j.string("B") + j.true
+      )
+
+ALIAS obj()
+-----------
+
+Deprecated alias for ``object()``.
+
+WARNING
+=======
+
+This vmod makes use of implementation details of the current Varnish
+VMOD interface, which might change any time. While the maintainer of
+this VMOD will try to keep it working, *be prepared for breaking
+changes*.
+
+IMPLEMENTATION NOTES
+====================
+
+STRANDS as variable arguments
+-----------------------------
+
+This vmod uses the STRANDS data type of the Varnish VMOD interface as
+a replacement for a variable arguments interface: When the VCC
+compiler sees multiple, concatenated strings to make up a STRANDS
+function argument, it tries to create a single string at compile time,
+which is only possible if all strings are literal. Otherwise, it
+creates a STRANDS with the values of the individual operands of the
+concatenation.
+
+This vmod makes use of this particular implementation: By wrapping the
+operands of the concatenation with functions, the VCC compiler can not
+create a literal string at compile time.
+
+Magic values of JSON STRINGs
+----------------------------
+
+Having understood the above, consider this example::
+
+  j.object(j.string("A") + j.object(j.nil()))
+
+produces ``{"A":{}}``, and::
+
+  j.object(j.string("A") + "{}")
+
+produces ``{"A":"{}"}``.
+
+If ``j.object(j.nil())`` produces the *VCL STRING* ``{}``, why does
+``j.object(j.string("A") + j.object(j.nil()))`` not produce
+``{"A":"{}"}``?
+
+The answer is that the STRINGs returned by *j* functions always start
+at an odd memory address and have a magic value in the preceding
+byte. In memory, the return value of ``j.object(j.nil())`` looks like
+this::
+
+	\x81{}
+	    ^
+	    +--- pointer
+
+This method relies on a couple of assumptions:
+
+* Inspection of a pointer addresses is possible
+
+* A pointer address can be decremented
+
+* If an odd memory address is valid, that address minus one is also
+  valid.
+
+These assumptions are true on modern hardware, but they might not be
+forever.
+
+For the magic check to do the right thing, we also rely on the fact
+that ordinary strings produced by VCC or VMODs begin at an even
+address (which most of them do) and, even if they begin at an odd
+address, are highly likely to not be preceded by a value of 0x81 to
+0x85.
+
+In particular, all this is only relevant if literals are used. If the
+`wrapper functions`_ are used always, the method is safe.
+
+Only if un-wrapped literals are used, the worst that could happen is
+that we would miss to auto-magically string-wrap an argument which
+would need string wrapping to produce correct JSON or that we would
+accept an invalid element as an object key.
+
+SEE ALSO
+========
+
+vcl\(7),
+varnishd\(1)
+
+COPYRIGHT
+=========
+
+::
+
+  Copyright 2023 UPLEX Nils Goroll Systemoptimierung
+  All rights reserved
+ 
+  Author: Nils Goroll <nils.goroll@uplex.de>
+ 
+  Redistribution and use in source and binary forms, with or without
+  modification, are permitted provided that the following conditions
+  are met:
+  1. Redistributions of source code must retain the above copyright
+     notice, this list of conditions and the following disclaimer.
+  2. Redistributions in binary form must reproduce the above copyright
+     notice, this list of conditions and the following disclaimer in the
+     documentation and/or other materials provided with the distribution.
+ 
+  THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+  ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+  ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+  OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+  HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+  LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+  OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+  SUCH DAMAGE.