On their own, those changes improve the documentation and don't mention
the reverted "hash_data() in vcl_recv" feature.

Refs e98e8e64

This reverts the following commits:

- e98e8e64.
  "Documentation updates for changed `vcl_hash{}` / `hash_data()`"
- 001279eb.
  "Document proper design pattern for using hash_data() in vcl_recv,"
- e36573e2.
  "Add a test-case for hash_data() in vcl_recv{}"
- 03fe0cee.
  "Allow hash_data() in vcl_recv{}"
- 4ebc3cfe.
  "Make it possible to override the initial digest, and explain in"
- d6ad52f5
  "Change the way we calculate the hash key for the cache."

Conflicts:
	doc/sphinx/reference/dp_vcl_recv_hash.rst
	doc/sphinx/reference/index.rst

Concerns were raised regarding a change of the way we compute the hash
key outside of the dot-zero release where we would expect such breaking
changes (among other things, vmod_shard relies on hash stability).

There is also no definite consensus of how to handle hashing from
vcl_recv.

... documented on Linux as POSIX.1

The exception here is ECONNREFUSED which so far we only tolerate for
Solaris and which seems to make sense for connect() only.

To be discussed in #3539

The watchdog mechanism currently triggers when any queueing is happening,
regardless of the priority. Strictly speaking it is only the backend
fetches that are critical to get executed, and this prevents the thread
limits to be used as limits on the amount of work the Varnish instance
should handle.

This can be especially important for instances with H/2 enabled, as these
connections will be holding threads for extended periods of time, possibly
triggering the watchdog in benign situations.

This patch limits the watchdog to only trigger for no queue development
on the highest priority queue.

When accepting new incoming connections in the acceptor thread, it would
schedule, they would be registered with the VCA priority. This priority is
reserved for the acceptor thread itself, and specifically is not included
in the TASK_QUEUE_CLIENT categorisation. This would interfere with the
thread reserve pools.

t02011.vtc had to be adjusted to account for the new priority
categorisation of the initial request.

in a place more likely to be noticed

I also played with other spatch options to replace ``--dir .``, but they
resulted in more clumsy shell commands.

Ref a98c1489

double quotes ourselves.

FreeBSD's code is undergoing some compatibility adaptations
so we go directly to the source from here.

PS: Please keep cochinelle out of libvgz.

Until now we'd send a zero-length DATA frame with the END_STREAM flag
set, masquerading fetch streaming failures as completed responses. In
the absence of a content-length header a user agent couldn't notice
there was a problem.

This does not apply to ESI sub-requests, they aren't streamed during
delivery.

Since the removal of dry signals, pools will spin when they run out of
threads and increment MAIN.threads_limited at a very high rate. That
spike in CPU consumption will also have detrimental effects on useful
tasks.

This change introduces a 1s delay when the pool is saturated. This
allows to periodically exercise the watchdog check. We could wait until
the watchdog times out but we would also miss potential updates to the
thread_pool_watchdog parameter.

To avoid spurious increments of MAIN.threads_limited we only take task
submissions into account and ignore timeouts of the condvar.

Refs #2942
Refs #3531

it was already accepted on Solaris and NetBSD, now we have seen it on
Linux and I think it does not make sense to keep the exception for
Apple.

Fixes #3532 (hopefully)

Spotted by: Rasmus Villemoes

Spotted by: Rasmus Villemoes

Spotted by: Rasmus Villemoes

Spotted by: Rasmus Villemoes

like the sign is wrong, we should never get EINVAL if the deadline is
in the future.

Fix the test to make sense, and hope for the best (or a M1 :-)

Spotted by: Rasmus Villemoes

Spotted by:	Rasmus Villemoes

Move the description of this feature from user to reference.

In pool_herder(), we do as much work as possible outside the pool
mutex. In particular, we check if we need to breed more threads, then
we do more checks, and finally we aquire the lock and either go sleep
or increase the threads_limited counter if, at that point, lqueue is
non-zero.

Yet it could have increased since we last checked, and there is
nothing wrong with that, we will breed a new thread in the next
iteration of the loop.

Increase threads_limited only if we would not bread a new thread right
away.

Fixes #3531

to handle correctly use of the "gunzip" filter at esi_level > 0

Fixes #3529

The bo fields err_code and err_reason need to be reset on a retry otherwise the values are kept.

Fixes #3525