The previous changelog update contained a formatting mistake which is
corrected here.

which means not in vcl_backend_error{}

Fixes: #1865

Conflicts:
	lib/libvcc/generate.py

This is an adaptation of 86ae6dcf to the 4.1 branch. The point is to
avoid accidential ++ and -- in generated code.

IOW if we ended up in vcl_backend_error{} or failed after returning
delivery in vcl_backend_response{} (e.g. we did not receive enough bytes)
we will bump this counter.

Conflicts:
	bin/varnishd/cache/cache_fetch.c
	bin/varnishtest/tests/r01624.vtc
	include/tbl/vsc_f_main.h

Fixes: #2167

Conflicts:
	bin/varnishtest/tests/m00019.vtc
	lib/libvcc/vcc_expr.c

Spotted by Martin, when a VMOD relying on varnish.m4 is built with
--prefix in the configure command-line, a pseudo hard-coded vmoddir
is assumed:

    $libdir/varnish/vmods

Instead, it now asks pkg-config for the vmoddir relative to the
libdir. It worked in all cases with upstream Varnish Cache, but
downstream modifications of the vmoddir would then be ignored.

Fixes #1704

Conflicts:
	include/tbl/vsc_f_main.h

Fixes #2197

Also clarify %{X}i and %{X}o wrt multiple X headers.

Fixes #2190.

Conflicts:
	doc/sphinx/reference/varnishncsa.rst

Comparing a backend with various stuff was not handled well. The test
case shows what will give a compilation error, and the results of ways
of comparing backends. When using a director, the demonstrated results
are not obvious.

Fixes: #2168

Fixes: #2178

This reverts commit 95100cd0.

We already had a check that took care of this in an architecture
independent way.

See 452ac521
Closes #1750

There was an error in the last changelog update.

When going on a waitinglist, the HTTP1 step counter was set to
S_STP_H1BUSY to allow conditions to be checked on return specific to
having been on a waitinglist. Though the step counter was set after
entering the waitinglist, and if the session was rescheduled
immediately on another thread a race opened which would mess up the
state handling.

Fix this by elliminating the S_STP_H1BUSY step, and having condition
checks on req->hash_objhead in the S_STP_H1PROC state to handle the
waitinglist return specific checks.

The panic output has been changed to include the req->hash_objhead
pointer if set. This exposes the waitinglist condition in the panic
output which would otherwise be hidden by the removal of the
S_STP_H1BUSY step.

Fixes: #2117

Closes #1750

Fixes #2134.

The previous commit was not a good backport, as I did not check the
new test case. Now the test case has been updated, and everything
should work.

Fixes #2148.

Increase the Nagle-avoidance delay in r02035.vtc to make the test case
less timing sensitive.

HTC_RxInit and HTC_RxReInit could write a single '\0' NUL character
outside of the workspace when its called and there is zero bytes left
in the workspace. This would trigger the workspace canary causing
subsequent assertion.

Fix by releaving HTC_RxInit and HTC_RxReInit of adding the '\0'
character.

HTC_RxStuff and V1F_FetchRespHdr returns HTC_S_OVERFLOW if the
available buffer space is zero. Both make sure to insert the '\0'
character just before calling the completion check function.

Note that this fix does not change the fact that we have exchausted
the workspace and are unable to continue. Varnishd will panic
nonetheless, but at least we have not stepped out of our boundries.

Ref: #1834

Make sure that the workspace can accomodate the pipelined data before
memmoving it into place in SES_RxReInit.

When doing range delivery, make the delivery processing finish early
when all of the bytes of the requested range has been delivered. This
to avoid waiting around for a slow fetch to finish before handling the
next request on the connection.

Fixes: #2035

A delivery processor may want to finish the delivery early. Change the
return value from VDP_bytes and the delivery processors into 3
categories. 0 means continue, negative is error and positive is
finished.

The latching of errors (to help ESI delivery continue while still
recording the error) will latch the lowest non-zero value observed.

VDP_gunzip is fixed to not unconditionally return errors.

Also add some text to explain the behaviour of VDP_bytes and its
delivery processors.

(backend tasks for the time being)

Add parameter thread_pool_reserve to tweak

Conflicts:
	bin/varnishd/mgt/mgt_pool.c