k8s uses re2, which does not support lookaround, so this rather
complicated pattern is what I came up with to forbid vcl and
vk8s prefixes.

The shard director supports many interesting use cases, for example

- retrying backend requests on different backends

- sharding by custom keys, for example for "client stickyness"
  or cache optimization

This patch adds the option to create parameter objects for backends
using the shard director.

Each named parameter object will be created only once and associated
with all directors for which the backend configuration names them.

Sleep a few seconds longer before grepping the controller log.
During a lengthy e2e test, the expected log line was not found
by grep in time.

The default format has been changed to include the instance name
and the test only needs to check for a sensible Via header at all.

... a COPYING file exists.

See
* https://github.com/libsndfile/libsndfile/pull/819
* https://github.com/libsndfile/libsndfile/pull/820

E: Version '2.2.9-2+deb11u2' for 'haproxy' was not found

We had updated to a very new version from bullseye-backports to test
if it had a necessary bugfix, but that brought no improvement. As it
turned out, the problem was ulimit -n as addressed in 03f46509.

2.2.9 is sufficient for our purposes. backports repos are problematic
because a version of a package ceases to exist from time to time.
This causes container builds to fail at times that are unpredictable
in practice.

The field is still optional, but if no default key is specified, then
it is implicitly the empty string. We no longer support a use case
in which an error response is returned for sharding by key if the
key is missing from the request. This will also be the case when
we support sharding by other keys besides cookies, such as request
headers.

This makes it possible to move the clause in vcl_recv that includes
the synthetic response to health checks from other shard instances
to come after the code for primaryOnly. That had been moved to the
start of vcl_recv, because health checks were failing in the shard
by cookie case without a default key (since health probes have no
Cookie header). But the result was that primaryOnly code did not
work as expected. Now we have moved that clause back to where it
was previously -- health probes now don't fail in the shard by
cookie case, because there is always a default key.

XXX: currently only used for sharding by cookie. Will be extended
for use with all by=KEY cases.

To ensure that health checks from other Varnish instances succeed
in the self-sharding configuration, move the clause in vcl_recv
that is executed when remote.ip matches the ACL for cluster members
to the start of the subroutine.

XXX: currently returns a synthetic 400 response if the configured
cookie name is not found in the Cookie header (or if there is no
Cookie header). In a further iteration, we will add an optional
field defaultKey, which if present sets a string value to be used
as the sharding key if the key is not found.

XXX: testing this feature has revealed bugs in sharding by key
(using by=KEY) unless primaryOnly is also set to true. These will
be fixed in a further iteration.

Fixing the dataplaneapi has resolved this problem. In the process of
debugging, a logger was added to the dataplaneapi client code, which
will now be retained.