We reported any PROXY connection failure as JUNK. Now we report the same
reasons as for failing HTTP1 connections.

Implmentation:

SES_DeleteHS() asserts that the hs argument be negative (one of
JUNK, CLOSE, TIMEOUT, OVERFLOW, EOF).

HTC_RxStuff() may return OVERFLOW, JUNK, COMPLETE, EOF or TIMEOUT (or,
instead of TIMEOUT, IDLE if the callback returned EMPTY).

Because vpx_complete() only returns JUNK, OVERFLOW or MORE, using
the HTC_RxStuff() return value for SES_DeleteHS() is safe if different
from COMPLETE.

adds handling of HTC_S_JUNK

Coverity CID 1099620

reported by coverity

This is to chase a new case of the watchdog off the leash

more re-use coming up in a PR

seen in vtest:

**   l1    Waiting for logexp
**** dT    1.477
**** l1    match|       1016 BogoHeader      c Missing header name: : Header
***  l1    expecting| expect * 1018 VCL_Error Bad header foo:

/9 overtook /8

seen in vtest:

**   top   === server s1 -repeat 5 {
**   s1    Starting server
---- s1    Server listen address (127.0.0.1 58150) cannot be resolved: bind(2)

NB: I am not sure about this attempt

false negatives reported by vtest:

**   top   === logexpect l1 -wait
**   l1    Waiting for logexp
**** l1    match|       1005 VCL_use         c vcl1
***  l1    expecting| expect 0 = ReqURL struct
**** l1    err  |       1005 ReqURL          c /encode
---- l1    bad| expectation failed

fixes #3485

they are revealed by vbe_dir_event() on a warm event

Fixes #3358

VBP_Control() asserts that, when the vcl is warm, probes are scheduled
to run (that is, exist on the probe scheduling binheap) and not so
when the VCL is cold. This is correct because any VCL events are
guaranteed to be serialized by mgt.

vbp_thread(), however, momentarily left probes removed from the
binheap while scheduling a probe not holding vbp_mtx, which left a
window for VBP_Control() to find an active probe not on the binheap.

Both vbp_thread() and vbp_task() re-schedule the probe at hand. The
former in case scheduling the task fails, and the latter to reschedule
it at the right interval relative to the probe having finished.

We now re-schedule the probe in vbp_thread() before adding a task to
run it, which does not change anything about the above, and, in
particular, keeps vbp_mtx free while adding the probe task.

Yet it closes the race and thus

fixes #3362

survived varnishtest -t 90 -n 10000 -j 200 -i tests/v00003.vtc

it does not need many worker threads and runs for >30s. This helps being
able to run this test with more concurrency to test #3362

... thinks this statement is confusing.

Ref: 15f94d33

VRT_new_backend_clustered() is the only caller of VBP_Insert().

There, VBP_Update_Backend() needs to be called after the director is
created in order to update the director status for a cold VCL.

For a warm VCL, VBP_Update_Backend() is called via
VRT_AddDirector() -> VDI_Event() -> vbe_dir_event() -> VBP_Control()

Thus, the additonal call before the director is added to the backend,
does not do anything but update the poll bits.

Somehow related to #3362

We don't need to assert the integrity of the VSB after every single
byte, it can become very expensive depending on the workload.

Ref 001485e6

Going back to lowercase, and aligning code blocks to 4 spaces to match
the rest of the code snippets.

Ref #3470

Ref #3470

Fixes #3478

When resolve requests race, we were not guaranteed to consider all
backends because we updated a shared nxt variable.

Fixes #3474

... also (C)ompleted bans.

The previous code made a lot of sense in light of the performance issue
due to too many bans, but would not work to prune a long ban list due to
(R)equest bans at the tail, as in this example:

Present bans:
1607346873.538879 354304 C
1607346873.532313     0 C
... lots of (C)ompleted bans
1607083561.980118     0 -  req.http.Host ~ foo
1607083561.972629    15 C

The documentation does not mention the previous behavior (that only
"active" bans are being counted), so this changes aligns code to
documentation.

This introduces a `PAN_dump_struct` function to do the
standard up-front checks for NULL, bad magic & already dumped.

    vtc_haproxy: VTCP_listen_on 127.0.0.1 instead of localhost

    Always bind vtc_haproxy with 127.0.0.1 instead of binding on ::1 or
    127.0.0.1 depending on the environment.

    This is related to bug https://github.com/haproxy/haproxy/issues/937
    here the ipv6 and ipv4 are randomly mixed which can't work with socks4.

And wrap the logic in a macro to reduce duplication.

It's more an action than housekeeping.

Make it possible to compare VSCs, to create a synchronization point
based on internal events (especially with custom VSCs).

For example:

    # block until baz catches up
    varnish v1 -expect FOO.bar == FOO.baz

This is still subject to roughly 5s before timing out.

... unless we f*up, which is the sole purpose of its existence

this needs to live within the exclusive circle of cache_vcl.h includers, so I
went for cache_vrt_vcl.c just because of the name.

This is the refactoring we agreed on to enable an alternative
implementation of #3454. This PR does not yet introduce the copy
callback needed to add the functionality suggested in #3454.

We replace the .free pointer of struct vmod_priv with a pointer to a
methods struct with callbacks. As of this commit, it only contains
the .free callback renamed to .fini. The purpose of the refactoring is
to allow addition of more callbacks later.

The new struct vmod_priv_methods also contains a .type member pointing
to a string to contain an arbitrary description of the type of data any
priv holds which uses these methods.

Implementation:

relevant changes are in cache_vrt_priv.c and vrt.h, other changes are to
the documentation and bundled vmods.

The implementation is a simple refactoring for indirection of the call
to the .fini callback via the methods structure.