This was a proof of concept for an alternative WS interface -- write
incrementally to reserved workspace as an "open buffer".

That may be generally useful for Varnish, but for now we return to
the usual WS_* idiom.

The error may be status 500 on 32-bit systems.

On the various platforms, esp 32-bit, workspace may be exhausted
before it gets to the VMOD at all, so we can't reliably expect
specific error messages.

This has nevertheless improved GCOV coverage, which was the goal.

To get some better GCOV coverage. So far just IDENTITY encoding.

Submitted by:	Frederic Lecaille <flecaille@haproxy.com>

Closes #2722

I've been torturing varnish with this change for some time and was
not able to reproduce the problem.

Should fix #2719.

Fixes	2727

Unless -1 is used too, but -1 already implies -d so it doesn't really
count.

Fixes #2721

This reverts commit d1b78e8d.

The problem is that upon exiting the screen is likely to be cleared on
most terminals (not the one we embed in varnishtest for example) so it
doesn't give any chance to the user to read the output.

The consensus is to restore the previous behavior and fix the manual
instead.

Conflicts:
	bin/varnishtest/tests/u00004.vtc
	bin/varnishtop/varnishtop.c

Breaks:
	bin/varnishtest/tests/r02686.vtc

Refs #2721

Fixes: #2572

with varnish user etc.

With the req.grace reintroduction, we got two identical cases where
we insert a busy object: When there is no exp_oc (obviously) or
when the exp_oc does not fulfil the req.grace limit. This lead
to some code duplication.

IMHO, handling first the case where we need to insert a busy object
(with or without exp_oc) and then handling the exception that we
do not wait on a busy object if the exp_oc is in grace is cleaner.

Please note that comparing this change using diff -u does not
make much sense, I believe the simplicity of the reshuffling
change becomes clear when looking at the old and new code side-by-side

The main point is to clearly recomend using req.grace for the most
common use case - using different grace time when the backend is
healthy.

Right now there are use cases that cannot be solved without req.ttl,
so we undeprecate it.

The req.grace variable can be set in vcl_recv to cap the grace
of objects in the cache, in the same way as in 3.0.x

The "keep" behavior changes with this patch. We now always go
to vcl_miss when the expired object is out of grace, or we go
to the waiting list. The result is that it is no longer
possible to deliver a "keep" object in vcl_hit.

Note that when we get to vcl_miss, we will still have the 304
candidate, but without the detour by vcl_hit.

This commit changes VCL, but only slightly, so we aim to back
port this to earlier versions of Varnish Cache.

Refs: #1799 and #2519

address family is neither of AF_INET or AF_INET6.

This means that UNKNOWN is sent in probes to UDS backends when
.proxy_header=1 is set.

Also verify that a UDS backend receives PROXY LOCAL in a probe when
.proxy_header=2 is set.

Fixes #2702
Closes #2726

Since we verify that header blocks are not interleaved, and we zero the
struct h2h_decode on every new header block, there is no need to malloc
a separate struct h2h_decode per stream.

... not in the caller

Future-proofing to avoid mistakenly introducing another race down the
line.

The current flow control code's use of h2->cond is racy.

h2->cond is already used for handing over a DATA frame to a stream
thread. In the event that we have both streams waiting on this condvar
for window updates and at the same time the rxthread gets signaled for a
DATA frame, we could end up waking up the wrong thread and the rxthread
gets stuck forever.

This commit addresses this by using a separate condvar for window
updates.

An alternative would be to always issue a broadcast on h2->cond instead
of signal, but I found this approach much cleaner.

Probably fixes: #2623

For ban statistics, we updated VSC_C_main directly, so if we raced
Pool_Sumstat(), that could undo our changes.

This patch fixes statistics by using the per-worker statistics
cache except for the following remaining corner cases:

* bans_persisted_* counters receive absolut updates, which does
  not seem to fit the incremental updates via the per-worker stats.

  I've kept these cases untouched and marked with comments. Worst
  that should happen here are temporary inconsistencies until the
  next absolute update.

* For BAN_Reload(), my understanding is that it should only
  happen during init, so we continue to update VSC_C_main
  directly.

* For bans via the cli, we would need to grab the wstat lock,
  which, at the moment, is private to the worker implementation.

  Until we make a change here, we could miss a ban increment
  from the cli.

* for VCL bans from vcl_init / fini, we do not have access
  to the worker struct at the moment, so for now we also
  accept an inconsistency here.

Fixes #2716 for relevant cases

A bound socket will timeout instead of refusing the connection.
Should fix b00015.vtc under macos.

The previous commit made the assumption that END_STREAM is in the last
of the frames in a header block. This is not necessarily the case.

Previously we've been incorrectly transtitioning to CLOS_REM on
END_HEADERS, which prevents us from seeing if a client incorrectly
transmitted a DATA frame on a closed stream.

This slightly complicates things in that we can now be in state OPEN
with an inactive hpack decoding state, and we need to make sure on
cleanup if that has already been finalized.

This would be simpler if the h/2 spec had split the OPEN state in two
parts, with an extra state transition on END_HEADERS.

Again big thanks to @xcir for his help in diagnosing this.

Fixes: #2623

Should help with the ASAN builds in travis.

Should address #2666 and #2711.

Tune down h2_rx_window_low_water to make sure we don't get a
window_update racing against the response frames.

Fixes: #2709

If we failed to schedule a thread for a stream and it's not cleaned up
prior to handling a request body, the rxthread would sit around waiting
in h2_rx_data indefinitely.