Refs #3100

It uses a similar trick as VCC expressions to find where the symbol name
belongs.

Refs #3100

First, gather everything needed, then generate the C code. This avoids a
bit of code duplication and makes the error handling more natural, right
after the attempt at generating a matching expression.

Refs #3100

    Avoid VSB_printf for static strings

    Done with the following semantic patch for Coccinelle:

        @@
        expression vsb, fmt;
        @@

        - VSB_printf(vsb, fmt);
        + VSB_cat(vsb, fmt);

    This patch is available in the Varnish source tree.

    vtc_haproxy: enable mcli with a separate flag from worker mode

    Support for the master cli in worker mode was added with commit 86e65f1
    using the same flag as worker mode (-W). This is a problem with haproxy
    1.8 because it has worker mode but no mcli. This patch adds -S to enable
    mcli for haproxy in a vtc.

and:

    The two patches in attachment fix a bug in the haproxy CLI handling and add
    the support for the haproxy master CLI in VTest. Could you merge them? Thanks
    You can also find in attachment a vtc file which use this new feature.

    Submitted by:   William Lallemand <wlallemand@haproxy.com>

enum gethdr_e and VCL_HTTP have a 1:1 relation except for HDR_OBJ which
does not have a corresponding VCL_HTTP pointer.

As I am about to use a "reverse VRT_selecthttp()" to get from VCL_HTTP
to enum gethdr_e, I would want to guard against accidental introduction
of a regression.

Ref #3145 / 287dc4a6

Ref #3145 / 287dc4a6

Fixes typo in storage backends guide.

There's only one place where a symbol may be assigned the SYM_OBJECT
kind and it is ensured that the JSON descriptor is found and added to
the symbol.

Patch by @ehocdet, commit message edited by @nigoroll:

The root cause of #3131 was misdiagnosed to the extent that, while this
change had prevented it, the root cause was a bug in WS_ReserveSize()
fixed in 505b7bd9

The previous tlv_string() code was correct except for the
fact that error handling should have checked for WS_ReserveSize(ctx->ws,
len+1) <= len (also spotted by @ehocdet).

Someone had mentioned at some point that we would not want to VRT_fail(),
but I think this must have been related to the proxy transport code, not
the proxy vmod.

Ref #3131

Notes:

* for the acceptor, I think it makes sense to keep AN assertion (pun!)
  because varnish is not viable if the session workspace is too small
  to even hold the attributes initialized in the acceptor.

  If this was an issue, we should rather revisit the minimum values for
  the session workspace

* for h1 and h2 session setup, I have used XXXAN() because I am not sure
  how we should best handle allocation failures.

* The relevant bit, for now, is the proxy code which may allocate
  arbitrarily long TLV attributes, so this is the code for which we now
  actually handle errors and test that we do

On the vtc: I added the test to o00005.vtc because there existed a
previous overflow test from 267504b8,
but that only tested for the one case of a WS overflow which was already
handled.

Fixes #3145

This originates from a3d47c25, but
was overlooked in 4e333597:

When there is insufficient space to fulfil the reservation request, we
must not leave the workspace reserved.

Fixes #3131

It's only ever used by base64.c, so we might as well put it there.

Better diff with the --ignore-all-space option.

This is a mechanical change that hides the const and restrict details of
the codec contract in vmod_blob and makes the function signatures more
manageable.

Refs b0c0eaea

I started suspecting that we need this clarification during the review
of #3123 [1] and was able to verify it with a simple test case. First I
needed a function I put in vmod_debug:

    $Function STRANDS hoard_strands(PRIV_TASK, STRANDS s)

    Return the first value of s for the rest of the task.

The implementation is very straightforward:

    struct hoarder {
           VCL_STRANDS     s;
    };

    VCL_STRANDS
    xyzzy_hoard_strands(VRT_CTX, struct vmod_priv *priv, VCL_STRANDS s)
    {
           struct hoarder *h;

           CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
           AN(priv);

           if (priv->priv == NULL) {
                   h = malloc(sizeof *h);
                   AN(h);
                   h->s = s;
                   priv->priv = h;
                   priv->free = free;
           }

           return (priv->priv);
    }

And then the following test case results in a panic on my system, but I
suspect this is generally undefined behavior and other nasty results may
occur under different circumstances:

    varnishtest "Beware of STRANDS"

    varnish v1 -vcl {
            import debug;
            backend be none;
            sub vcl_recv {
                    debug.hoard_strands("recv: " + req.url);
            }
            sub vcl_miss {
                    debug.hoard_strands("miss: " + req.url);
                    return (synth(200));
            }
            sub vcl_synth {
                    set resp.body = debug.hoard_strands("synth: " + req.url);
                    return (deliver);
            }
    } -start

    client c1 {
            txreq
            rxresp
            expect resp.body ~ recv
    } -run

This also begs the following question: can it ever be safe to let a VMOD
function return a STRANDS? Maybe it should be banned from return types.

[1] https://github.com/varnishcache/varnish-cache/pull/3123#discussion_r345617108

Is it because German capitalizes so many words that his shift keys have
become hit or miss?

as suggested by @Dridi with reference to @bsdphk in
https://github.com/varnishcache/varnish-cache/pull/3130#pullrequestreview-316916213

we ensure that only backends on the director_list receive events.
Because events happen in the cli thread, we do not need another level of
serialization.

There is no explicit vtc, the test case has been added to vmod_debug and
runs with every invocation of that vmod.

Fixes #3094

we are going to need to check a temperature on its own

Ref #3094

Fix a small memory leak when failing to spawn a new thread.

When getidleworker signals the pool herder to spawn a thread, it increases
the dry counter, and the herder resets dry again when spawning a single
thread. This will in many cases only create a single thread even though
the herder was signaled dry multiple times, and may cause a situation
where the cache acceptor is queued and no new thread is created. Together
with long lasting tasks (ie endless pipelines), and all other tasks having
higher priority, this will prevent the cache acceptor from being
rescheduled.

c00096.vtc demonstrates how this can lock up.

To fix this, spawn threads if we have queued tasks and we are below the
thread maximum level.

Fixed the typos in the files mentioned in the corresponding issue.

Fixes #3135