I actually did consider this before, but thought we should keep it
because we are currently dicussing to add a CTX to vmod object
destructors. But this piece of code is unrelated and at any rate the
argument can be added back when we need it.

in short: @Dridi is right

VCC generates a call to VPI_vcl_select() before returning to the fsm. We
basically conducted half a rollback in VPI_vcl_slect and a rull rollback
in the fsm.

De-duplicate this dance and assert in the fsm that our request looks
rolled back.

... which is the vcl the topreq started on

It is unclear to me what the original intention was, but at any rate,
the privs are not vcl-specific and the vcl argument was not used.

The motivation for this change is the follow up commit
"cleanup implicit rollback for return(vcl(...))": For this commit, it
would be unclear which vcl to roll back with, but as it actually does
not matter, I think this change is warranted.

As the FSM will fail for this case, we should not run the code in the
first place.

Multiple calls to the synthetic() function result in the accumulation of
all body parts. The ability to set a [be]resp body accidentally resulted
in the same behavior.

In other words writing `set resp.body = "string";` in VCL behaves as if
`set resp.body += "string";` was written instead. This patch introduces
an enum to distinguish between the two desired actions and enables both
syntaxes without touching the behavior of the synthetic() function.

And avoid the indentation of characters in vcc_assign_expr().

Fixes #3079
Closes #3100

At this point we should probably consider renaming the arithmetic table
to something also more general.

Refs #3100

It uses a similar trick as VCC expressions to find where the symbol name
belongs.

Refs #3100

First, gather everything needed, then generate the C code. This avoids a
bit of code duplication and makes the error handling more natural, right
after the attempt at generating a matching expression.

Refs #3100

    Avoid VSB_printf for static strings

    Done with the following semantic patch for Coccinelle:

        @@
        expression vsb, fmt;
        @@

        - VSB_printf(vsb, fmt);
        + VSB_cat(vsb, fmt);

    This patch is available in the Varnish source tree.

    vtc_haproxy: enable mcli with a separate flag from worker mode

    Support for the master cli in worker mode was added with commit 86e65f1
    using the same flag as worker mode (-W). This is a problem with haproxy
    1.8 because it has worker mode but no mcli. This patch adds -S to enable
    mcli for haproxy in a vtc.

and:

    The two patches in attachment fix a bug in the haproxy CLI handling and add
    the support for the haproxy master CLI in VTest. Could you merge them? Thanks
    You can also find in attachment a vtc file which use this new feature.

    Submitted by:   William Lallemand <wlallemand@haproxy.com>

enum gethdr_e and VCL_HTTP have a 1:1 relation except for HDR_OBJ which
does not have a corresponding VCL_HTTP pointer.

As I am about to use a "reverse VRT_selecthttp()" to get from VCL_HTTP
to enum gethdr_e, I would want to guard against accidental introduction
of a regression.

Ref #3145 / 287dc4a6

Ref #3145 / 287dc4a6

Fixes typo in storage backends guide.

There's only one place where a symbol may be assigned the SYM_OBJECT
kind and it is ensured that the JSON descriptor is found and added to
the symbol.

Patch by @ehocdet, commit message edited by @nigoroll:

The root cause of #3131 was misdiagnosed to the extent that, while this
change had prevented it, the root cause was a bug in WS_ReserveSize()
fixed in 505b7bd9

The previous tlv_string() code was correct except for the
fact that error handling should have checked for WS_ReserveSize(ctx->ws,
len+1) <= len (also spotted by @ehocdet).

Someone had mentioned at some point that we would not want to VRT_fail(),
but I think this must have been related to the proxy transport code, not
the proxy vmod.

Ref #3131

Notes:

* for the acceptor, I think it makes sense to keep AN assertion (pun!)
  because varnish is not viable if the session workspace is too small
  to even hold the attributes initialized in the acceptor.

  If this was an issue, we should rather revisit the minimum values for
  the session workspace

* for h1 and h2 session setup, I have used XXXAN() because I am not sure
  how we should best handle allocation failures.

* The relevant bit, for now, is the proxy code which may allocate
  arbitrarily long TLV attributes, so this is the code for which we now
  actually handle errors and test that we do

On the vtc: I added the test to o00005.vtc because there existed a
previous overflow test from 267504b8,
but that only tested for the one case of a WS overflow which was already
handled.

Fixes #3145

This originates from a3d47c25, but
was overlooked in 4e333597:

When there is insufficient space to fulfil the reservation request, we
must not leave the workspace reserved.

Fixes #3131

It's only ever used by base64.c, so we might as well put it there.

Better diff with the --ignore-all-space option.

This is a mechanical change that hides the const and restrict details of
the codec contract in vmod_blob and makes the function signatures more
manageable.

Refs b0c0eaea

I started suspecting that we need this clarification during the review
of #3123 [1] and was able to verify it with a simple test case. First I
needed a function I put in vmod_debug:

    $Function STRANDS hoard_strands(PRIV_TASK, STRANDS s)

    Return the first value of s for the rest of the task.

The implementation is very straightforward:

    struct hoarder {
           VCL_STRANDS     s;
    };

    VCL_STRANDS
    xyzzy_hoard_strands(VRT_CTX, struct vmod_priv *priv, VCL_STRANDS s)
    {
           struct hoarder *h;

           CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
           AN(priv);

           if (priv->priv == NULL) {
                   h = malloc(sizeof *h);
                   AN(h);
                   h->s = s;
                   priv->priv = h;
                   priv->free = free;
           }

           return (priv->priv);
    }

And then the following test case results in a panic on my system, but I
suspect this is generally undefined behavior and other nasty results may
occur under different circumstances:

    varnishtest "Beware of STRANDS"

    varnish v1 -vcl {
            import debug;
            backend be none;
            sub vcl_recv {
                    debug.hoard_strands("recv: " + req.url);
            }
            sub vcl_miss {
                    debug.hoard_strands("miss: " + req.url);
                    return (synth(200));
            }
            sub vcl_synth {
                    set resp.body = debug.hoard_strands("synth: " + req.url);
                    return (deliver);
            }
    } -start

    client c1 {
            txreq
            rxresp
            expect resp.body ~ recv
    } -run

This also begs the following question: can it ever be safe to let a VMOD
function return a STRANDS? Maybe it should be banned from return types.

[1] https://github.com/varnishcache/varnish-cache/pull/3123#discussion_r345617108

Is it because German capitalizes so many words that his shift keys have
become hit or miss?

as suggested by @Dridi with reference to @bsdphk in
https://github.com/varnishcache/varnish-cache/pull/3130#pullrequestreview-316916213

we ensure that only backends on the director_list receive events.
Because events happen in the cli thread, we do not need another level of
serialization.

There is no explicit vtc, the test case has been added to vmod_debug and
runs with every invocation of that vmod.

Fixes #3094