Spotted via vtest.

now that we pass the bo's handling to the director method, in the case
of return(error) from v_b_f, we do no not want the deliberate
VDI_Finish() to change the handling from vcl.

Tests #2997

This is another requirement for use of VRT_fail() in director context.

For code checking for (struct vrt_ctx *)->vsl the logging order now may
change, and we might want to review our VSL() calls if the ctx vsl
buffer may now be used instead.

Ref #2997

This is option 'always add ctx->handling' from #2997:

we add the (struct vrt_ctx).handling pointer even where there is no VCL
method for consistency to enable use of VRT_fail() consistently.

When making other changes which require additional workspace, this test
may fail for too big a request in the first place.

This additional logging facilitates before/after analysis

we should not emit ReqUnset / ReqHeader if A-E is already correct

This was meant to go to a different remote before proposing it.
Sorry!

unmask/mask was inverted

remove superfluous comma

Fixes #3110

from both client and backend side.

When we breed threads without a delay, we should make an effort to let
them run as soon as possible and not have the cpu occupied with the
herder code.

This will increase the overhead due to additional context switching for
the benefit of reducing latencies until new threads get to run.

Effect seen with this vtc by @Dridi:

    varnishtest "over-breeding"

    varnish v1 -arg "-p thread_pools=1 -p thread_pool_min=10"
    varnish v1 -vcl {
            backend be none;
    } -start

    varnish v1 -expect MAIN.threads == 10

Tested with the following script:

    ./varnishtest -i -k -j40 -n1000 tests/as_above.vtc |
    awk '/^#/ { print $5; }'|
    sort |
    uniq -c

Multiple runs show a consistent drop of failed tests:

    before | after
    -------+------
        76 |    18
        67 |    24
        66 |    18

Fixes #3045

Found by @Dridi

I had previously overlooked that there exists a more complete regression
test by him in #3010

Integrate that vtc, polished and modified to account for the backend 503
for no restart after rollback, into m17, making r3009 obsolete again

Fixes #3083

to get there, also centralize req->resp setup for deliver and synth:

No semantic changes other than some reordering, which also fixes an odd
log line ordering as shown by the change to c00018.vtc

The same can be achieved with `varnishtest -p debug=+witness`, already
used by `make witness`.

This change introduces a top-level make witness target that builds a dot
graph and if graphviz is available, an SVG file as well. A shell script
replaces the previous python script that no longer works. Instead of
fixing witness.py, which is probably trivial, the shell script does an
intermediate pass and programmatically looks for cycles using tsort(1).

Checking lock dependencies becomes actionable in a CI context.

The script also takes explicit test directories on purpose, to have the
ability to aggregate test results from multiple executions. For example
when the test suite is run on various operating systems or with varying
privileges to cover feature-conditional tests.

The simple fact that Witness records might show up in the log might
break logexpect commands. There's no reason why we'd want to expect
Witness records since their purpose is to be checked after the test
finishes.

The gunzip vdp failed to handle junk after end of gzip data. This
basically mirrors #942 on the client side, also the fix is basically the
same as 41f7a356

The impact of this bug is likely to be low, because the built-in
beresp.filters logic will push the testgunzip VFP for gzip content
received from backends, so, unless VCL is forced to pass backend
responses unchecked or vmods generate body data, it can be considered
unlikely that this issue will be hit.

Fixes #3109

We use the first 64bit of a sha256 as our pool id, assuming that those
are safe enough against collisions. Ensure we do not fail on that
assumption.

This also makes it a caller error to deliberately use the same pool id
for different endpoints. As this was not possible before the id change,
I do not consider it a regression.

Ref: ec70dbc7

Ref: ec70dbc7

struct vtp_cs was only used for comparisons during pool lookup

If varnish closes the connection while a client is waiting for an
rxsomething command you may run into this kind of scenario:

    ***  c1   HTTP2 rx failed (fd:20 read: Connection reset by peer)
    ***  c1   rx: stream: 0, type: DATA (0), flags: 0x00, size: 0
    **** c1   s0 - no data
    ---- c1   Wrong frame type DATA (0) wanted SETTINGS

This was already done correctly for the frame body.

Instead of having to manually document them, we can now let the RST dump
figure everything out.

Refs #3099

Instead, they are added where they are needed: only in error messages
involving them.

Refs #3099

Refs #3108

* clarify defaults
* document Transient

Fixes #3108

Fixes: #3086

Refs #2872