The input argument assertions and checks in http_hdr_flags() were
misleading and lacking. With this patch it returns (NULL) on either input
being NULL, and also when called with an empty string instead of
asserting.

In http_EstimateWS(), all headers are passed to the http_isfiltered()
function to calculate how many bytes is needed to serialize the entire
struct http. http_isfiltered() will check the headers for whether they are
going to be filtered out later and if so skip them.

However http_isfiltered() would attempt to treat all elements of struct
http as regular headers with an implicit structure. That does not hold for
the first three pseudo-header entries, which would lead to asserts in
later steps.

This patch skips the filter step for pseudo-headers.

Fixes: #3830

Previously we would ignore errors to iterate the request body into
oblivion in VRB_Ignore(), keeping the connection open. This opens an
out-of-sync vulnerability on H/1 connections.

This patch tests the status of the request body in VRB_Ignore(), marking
the request failed and that it should be closed on errors.

Dont limit ourselves to a set number of VSL records to include
the "0 CLI" we want, various platforms emit a number of Debug
messages relating to sockopt

Use the `process p%d -expect-text` mechanism to wait only as
long as necessary for the "0 CLI"

Closes #3692

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

Instead of having a single global check that all acceptors may race
towards, this check now happens on a per listen socket basis. For
sockets with a different inheritance behavior on a single system, we
avoid having the first connection dictate what may be inherited by
a connection socket from its listen socket for all the other listen
addresses.

At least on Linux, Unix-domain sockets DO NOT inherit options like
SO_{RCV,SND}TIMEO even though TCP sockets do. On the off chance that
even sockets of the same family could behave differently, like for
example a regular vs a loopback TCP session, this is done on a per
listen address basis. To avoid cache-acceptor coordination with the
acceptor worker threads of a given listen address, workers will
individually perform this check once and for all when the first
connection is accepted.

We also stay defensive in the event of a parameter change, just in
case a previous test would assume inheritance because the Varnish
parameter value would match the kernel default value. Once a mismatch
is observed for a given connection with a given socket, the inheritance
test is no longer performed needlessly for this combination.

A race still exists between acceptors from different thread pools for
a given listen address, but this race is identical to the previous one
based on the former global need_test variable.

Although the inheritance check leaks into struct listen_sock, it is
opaque so everything can remain contained inside cache_acceptor.c.

Some aspects of this change (including the clarification comments) are
from @mbgrydeland.

Refs #2722

And while at it, update the SO_LINGER explanation to match reality.

Refs 7eba9460

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

The SO_KEEPALIVE justification used to precede such a constant and was
left confusingly lingering over the unrelated need_test variable.

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

Using the tmp sock_arg for storage, we can test all values with the same
logic and only differentiate hard-coded options from parameterized ones.

Stylistic polish by @mbgrydeland.

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

They can be stored directly in the sock_opts array or on the stack.

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

The sess and listen_sock structs contain everything we need to find or
infer the former `sock`, `uds` and `force` arguments. In particular, it
helps distinguish between working on a connection vs listen socket.

And conversely rename `to` variables to `so` for consistency.

Better diff with the --word-diff --word-diff-regex='\w+' options.

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

It is a 1:1 mapping with IPPROTO_TCP, so TCP_NODELAY was moved down to
be with the other IPPROTO_TCP options.

Conflicts:
	bin/varnishd/cache/cache_acceptor.c

Make a note in the documentation of CLI Command File that it is necessary
to include an explicit 'vcl.use' command in the script.

Limit the selection of the active VCL from MGT's view point to be the last
startup VCL given. That would be the VCL from the very last -f argument
given to varnishd, or the autogenerated VCL from the -b argument (-f and
-b are mutually exclusive). Because all startup VCLs are always set to
state AUTO, and AUTO VCLs are made warm when the child is not
running (which it is not at the time the startup VCLs are compiled), this
ensures that it is a warm VCL that is selected as the active VCL.

With this patch, VCLs loaded through the use of an initial CLI command
script (-I option) will not cause a VCL to automatically be selected as
the active VCL. Rather, it is expected that the initial CLI command script
should have an explicit 'vcl.use' statement to select the active VCL. When
an active VCL is not set, attempts to start the child will fail, which
again will fail the varnishd daemon startup (unless -d is given) with an
error code.

The behaviour prior to this patch when using -I, -f '' (empty field), -F
or -d was not well defined. The first VCL loaded (either by -I startup CLI
script or a CLI 'vcl.load' command) would become the active VCL, even if
that VCL is loaded cold. That is an illegal state and would lead to
asserts. It is also not very useful functionality, given the typical use
case for -I is to set up VCL labels. Those require the tips of the VCL
tree dependency graph to be loaded first, and then the VCLs that selects
the label. This means that the active VCL will typically be the last VCL
loaded, which then will require the use of a 'vcl.use' statement in the -I
script anyways. This makes it an acceptable change of default behaviour
that should not affect users.

mgt_new_vcl() now returns a pointer to the newly compiled VCL program, or
NULL on failure.

This also fixes a wrong return value used previously which would cause a
"VCL compiled.\n" to be output to CLI when the child is not running even
when the VCL compilation step failed.

mgt_has_vcl() now returns an error string describing why it doesn't have a
valid VCL.

This is taken from #3711 by @bsdphk

It is very easy to confuse the child process' vcl_active and the
management process' active_vcl variables.

Rename mgt's active_vcl to mgt_vcl_active, following the naming scheme
used elsewhere.

Add a ASSERT_VCL_ACTIVE() macro that asserts that vcl_active either is
NULL or points to a VCL that is warm.

Sprinkle this macro in the various calls throughout the child process that
are involved in VCL handling.

Patch by: @alveelde

Do not select the first VCL present as the active VCL in the child
process. Instead it should always use what the mgt process tells it
through the explicit 'vcl.use' commands it will send.

Previously, the child could select a cold VCL as the active VCL if that
happened to be the first one present to it during restarts. This could
lead to asserts as that is not an allowed state.

Using the wrong list variable would cause the dependency check to consider
VCLs that were not attempted discarded.

Fixes: #3734

For the REGEX type, we now peek ahead after the first CSTR token if a '+'
operator follows and, if yes, create the pattern as a concatenation.

Fixes #3726