Fixes #3491

Check for correct handling of missing pseudo-headers, and invalid
characters.
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

The :scheme pseudo header is not optional in H/2 except when doing CONNECT.
There is also a strict requirement for it appear only once.
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

We should apply the same restrictions on the list of allowed characters inside
H/2 pseudo-headers as we do for H/1. This error is translated into the
headers we send to a backend over H/1.

Failure to do so could permit various exploits against a backend not handling
malformed H/1 requests.
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

I find ":path: /foo" more descriptive than "/foo", even though I could
infer which one it was...

Now that http_DoConnection() is used without respecting the SC_RX_BAD
return value it should not return early when encountering a well-known
header.

Some browsers are strict about this and simply drop responses containing
such headers. Since this is not filtering a context switch between a
client and a backend transaction (or cache hit) a new filtering flag is
added to the HTTP headers table for connection-specific headers. This
new flag cannot be compounded as HTTPH_R_FETCH|HTTPH_A_INS because the
TE header is an exception and left alone, even though trailers aren't
supported.

Better diff with the --ignore-all-space option.

We could go further and consider any client request containing one as
malformed as mandated by RFC 7540.

Closes #3416

no semantic changes, just consolidating three places in one

(thus pushed during freeze)

Allow the user to log the request time in milliseconds through the new
format specifier: %{ms}T .
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

Previously we would ignore errors to iterate the request body into
oblivion in VRB_Ignore(), keeping the connection open. This opens an
out-of-sync vulnerability on H/1 connections.

This patch tests the status of the request body in VRB_Ignore(), marking
the request failed and that it should be closed on errors.

Conflicts:
	bin/varnishd/cache/cache_esi_deliver.c

This fixes a potential out-of-workspace panic.

Otherwise valid code can panic on workspace exhaustion:

    std.ip(req.http.X-Real-IP, std.ip(req.http.X-Client-IP, client.ip))

If the nested std.ip() call runs out of workspace, it will return a null
ip instead of the fallback, and the outer std.ip() call will panic upon
checking the suckaddr sanity.

Refs #3746

Dont limit ourselves to a set number of VSL records to include
the "0 CLI" we want, various platforms emit a number of Debug
messages relating to sockopt

Use the `process p%d -expect-text` mechanism to wait only as
long as necessary for the "0 CLI"

Instead of having a single global check that all acceptors may race
towards, this check now happens on a per listen socket basis. For
sockets with a different inheritance behavior on a single system, we
avoid having the first connection dictate what may be inherited by
a connection socket from its listen socket for all the other listen
addresses.

At least on Linux, Unix-domain sockets DO NOT inherit options like
SO_{RCV,SND}TIMEO even though TCP sockets do. On the off chance that
even sockets of the same family could behave differently, like for
example a regular vs a loopback TCP session, this is done on a per
listen address basis. To avoid cache-acceptor coordination with the
acceptor worker threads of a given listen address, workers will
individually perform this check once and for all when the first
connection is accepted.

We also stay defensive in the event of a parameter change, just in
case a previous test would assume inheritance because the Varnish
parameter value would match the kernel default value. Once a mismatch
is observed for a given connection with a given socket, the inheritance
test is no longer performed needlessly for this combination.

A race still exists between acceptors from different thread pools for
a given listen address, but this race is identical to the previous one
based on the former global need_test variable.

Although the inheritance check leaks into struct listen_sock, it is
opaque so everything can remain contained inside cache_acceptor.c.

Some aspects of this change (including the clarification comments) are
from @mbgrydeland.

Refs #2722

And while at it, update the SO_LINGER explanation to match reality.

Refs 7eba9460

The SO_KEEPALIVE justification used to precede such a constant and was
left confusingly lingering over the unrelated need_test variable.

Using the tmp sock_arg for storage, we can test all values with the same
logic and only differentiate hard-coded options from parameterized ones.

Stylistic polish by @mbgrydeland.

They can be stored directly in the sock_opts array or on the stack.

The sess and listen_sock structs contain everything we need to find or
infer the former `sock`, `uds` and `force` arguments. In particular, it
helps distinguish between working on a connection vs listen socket.

And conversely rename `to` variables to `so` for consistency.

Better diff with the --word-diff --word-diff-regex='\w+' options.

It is a 1:1 mapping with IPPROTO_TCP, so TCP_NODELAY was moved down to
be with the other IPPROTO_TCP options.

Closes #3692