The square brackets were probably the result of copy-pasta from vcl.load
where the initial state is indeed optional.

Before this patch, layered directors needed to be destroyed top to
bottom, and whenever that order was missed, we would panic, because a
to-be-destroyed director still had references to it.

One special case where this issue would always trigger are looped
directors. Those should not be used and will cause havoc, which is a
separate issue #3899. But we should still be able to unconfigure such
a configuration.

We solve the destruction order issue by making it a two step process:

When a director is destroyed through VRT_DelDirector, a new release
function is called, which has to disassociate any backends. The
director then loses a reference, and when all references are gone, the
destroy function is called.

The new callback would not be necessary for the cases in varnish-cache
today, directors could simply disassociate any backends before calling
VRT_DelDirector. But this would complicate or even make impossible
transfer of director ownership, where the code responsible for
creating a director is not the same as the one calling
VRT_DelDirector(). As a side effect, it also helps clarity.

Fixes #3895

The last reference to a director might go away with VRT_DelDirector
_or_ VRT_Asssign_Backend, which the former needs to account for.

We assert for the VDIR_FLG_NOREFCNT case that there was only one
reference such that a single deref yields no reference left.

Part one of the fix for #3895

This is in preparation of follow-up commits. Reasoning:

- in both call sites, we already use the struct vcldir *
- once call site actually used TAKE_OBJ semantics, but those
  can easily be moved

Use a local vdir variable for clarity like elsewhere in the code.
Use the lock in vdir, not the pointer to it in VCL_BACKEND for
consistency with VRT_DelDirector() a few lines above.

I noticed that users, apparently, have no way of finding out what
their distribution configured?

Now that we broke the VSL format, this is the time to act.

This prevents VUTs from attempting to read incompatible formats, while
preserving the current header size, aligning it with the SHMLOG header
at the format number 2.

It will no longer be valid for assignments, and in one case the
assignment was superfluous.

This means an older varnishlog can no longer read logs from a live
current varnishd server, and vice versa. It used to be interesting
to use a more modern VUT to process logs for example to get better
performance or new features like generalized -E.

Restructured so that:

* 'Upgrading' is limited to work that has to be done to upgrade from
  a current deployment to the new version.

* 'Changes' is a comprehensive, user-level description of changes and
  new features.

Conflicts:
	doc/sphinx/whats-new/index.rst

We keep s as a pointer to the start of an unaltered section and
move e to be able to call VSB_bcat() when a backslash is encountered
or substitution is complete.

With onerror=abort, the request is aborted as with a bad return code.

With onerror=continue, the include remains empty

This already behaved like I expected it to, this vtc merely adds
an explicit test.

suggested by Dridi

Connect to s1 and s2 via v2.

Note on the v2 VCL: We use this varnish instance as a PROXY protocol
aware forwarder, which takes the address to connect to from the
incoming PROXY header (to mimic haproxy instead of requiring it).

Previously, we used debug.dyn(), but that does not work with two
different backends because it does not create different backend
instances, so connection pooling fails on this level, unrelated to the
actual test subject.

We avoid this issue by an explicit VCL implementation.

If the .via field is also set, then the value of .authority is set
as the authority TLV in the PROXY header. This gives the "true"
backend (usually the ssl-onloader) the opportunity to set the SNI
(HostName field) from the TLV value, for the TLS handshake with the
remote backend.

This mandates that PROXYv2 is always used with a via backend (since
only version 2 supports TLVs).

If the value of .authority is the empty string, then the TLV is not
sent. If .authority is not set for the backend, then fall back to
.host_header, which itself may have been a fallback to .host. Note
that if neither .authority nor .host_header is set, and .host is
set to an IP address, then the IP address is forwarded as the SNI
value, which is not permitted for HostName (RFC4366 ch 3.1). So
users are advised to set either .authority or .host_header, or set
.authority="", when .via is set.

Usage note with haproxy:

To enable sending SNI when haproxy is used as a TLS onloader, ``sni
fc_pp_authority`` needs to be used with the backend configuration.

Full usage example with haproxy 2.2:

listen sslon
	mode	tcp
	maxconn	1000
	bind	/shared/varnish_haproxy/haproxy_sslon accept-proxy mode 777
	stick-table type ip size 100
	stick	on dst
	server	s00 0.0.0.0:443 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
	server	s01 0.0.0.0:443 ssl ca-file /etc/ssl/certs/ca-bundle.crt alpn http/1.1 sni fc_pp_authority
	# ...

A higher number of servers improves TLS session caching.