we need a signal if the fix is in

The 503 synth and 500 minimal response status codes are too misleading
in this context, where the failure is attributed to the client. Among
existing 4XX status codes, this is the closest if we stretch the timeout
definition to "didn't complete rapidly enough before the client went
away".

When copying a stale object after a 304 revalidation, we iterated over
it and allocated new storage for each storage segment.

So, at best, we kept the fragmentation of the existing object, or made
it even worse. This is particularly relevant when the existing object
was created from a chunked response, in which case the original
segments might have been particularly small and, consequently, many in
number.

Because we wait for the stale object to complete, we know the total
length (OA_LEN) upfront, and can ask the storage serving the copy for
exactly the right length. This is much more efficient, as
fragmentation is lowered and the storage engine might make a much
better allocation decision when it knows the full length rather than
getting piecemeal requests.

We implement the improved allocation scheme through additional state
kept for the duration of the template object copy: struct
vbf_objiter_priv holds the pointer to the newly created busy object,
the total yet unallocated length (initialized to OA_LEN of the existing
object) and pointer/length to the unused portion of the currently open
segment, if any.

It is my understanding that the objtrimstore stevedore API function is
only to be called once when the object is complete, which I believe is
also in line with the comment on ObjExtend() that "The final flag must
be set on the last call".

If this understanding of the API is correct, we did not adhere to it in
the fetch code when we made a copy of an existing stale object after a
304 response: There, we iterated over the stale object and did not set
the final flag just once when the object was complete, but rather after
each storage segment was copied.

This commit fixes this, adds some pedentry to the simple storage and
extends b00062.vtc to test this behavior specifically. On top, g6.vtc
also triggered without the fix but the duplicate trim detection in place.

This issue has originally surfaced in the SLASH/fellow storage where the
trimstore function implicitly asserted to only be called once.

Ref 115742b0
Ref https://gitlab.com/uplex/varnish/slash/-/issues/33

See "cope with older automake" a couple of lines up

Should fix the issue seen with Solaris vtesters. bash -x output:

+ vtc=./varnish-trunk/_build/bin/varnishtest/tests/r02686.vtc
++ echo ./varnish-trunk/_build/bin/varnishtest/tests/r02686.trs
++ sed -e 's/trs$/log/'
+ logfile=./varnish-trunk/_build/bin/varnishtest/tests/r02686.log
+ log=r02686.log
++ git log -n 1 --pretty=format:%H ./varnish-trunk/_build/bin/varnishtest/tests/r02686.vtc
fatal: ambiguous argument './varnish-trunk/_build/bin/varnishtest/tests/r02686.vtc': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
+ rev=

It has nothing to do with the hash lookup implementation, this is the
outcome of a cache lookup.

There is room for further improvement in the dynamic between HPACK and
the HTTP/2 session, but this will serve as the first step.

The :scheme pseudo header is not optional in H/2 except when doing CONNECT.
There is also a strict requirement for it appear only once.
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

Conflicts:
	bin/varnishtest/tests/t02025.vtc

It is the VTE's job to deal with alignment now.

Since the maximum space between fields is 3, it should have been obvious
that something was off while inspecting test cases. Before this fix, the
fields were aligned, but sometimes the space between fields was too wide
and in particular before right-aligned fields.

Fixes vtest error on SunOS gcc4.7.3 -m32:

vtim.c: In function 'main':
vtim.c:605:2: error: overflow in implicit constant conversion [-Werror=overflow]
vtim.c:605:2: error: comparison is always true due to limited range of data type [-Werror=type-limits]

and, in particular, ignore lib/libvarnish/vtim_test too

The problem with 93974b3f is that the
negating option from Makefile.am comes before CFLAGS, which does not
disable it for gcc.

Tested locally with gcc and clang.

The ASLR is now so aggresive that it never works.

Ref 01bf0dd2

Ref 63e4f71e

Function:

With the fold acl flag enabled (default: disabled, keeping the
existing behavior), ACLs are optimized in that subnets contained in
other entries are skipped (e.g. if 1.2.3.0/24 is part of the ACL, an
entry for 1.2.3.128/25 will not be added) and adjacent entries get
folded (e.g. if both 1.2.3.0/25 and 1.2.3.128/25 are added, they will
be folded to 1.2.3.0/24).

Skip and fold operations on VCL entries are output as warnings during
VCL compilation as entries from the VCL are processed in order.

Logging under the VCL_acl tag can change with this parameter enabled:
Matches on skipped subnet entries are now logged as matches on the
respective supernet entry. Matches on folded entries are logged with a
shorter netmask which might not be contained in the original ACL as
defined in VCL. Such log entries are marked by "fixed: folded".

Negated ACL entries are excluded from folds.

Implementation:

The sort functions are changed such that the previous semantics are
preserved: negative return values signify "a < b", positive return
values signify "a > b". But additionally the values -2/2 and -3/3 are
introduced (and given enums) to signify "contained in supernet" and
"directly adjacent to". This allows for mostly unchanged code with
vcc_acl_fold disabled.

For the "subnet contained in supernet" case, all contained subnets are
removed. By sort order, caontained subnets are always to be found left
of supernets.

For the "fold adjacent" case, the netmask of the entry with the
smaller network number is decreased by one and the other entry
removed. Because changing the netmask might affect sort order, we
reinsert the changed entry.

This is in preparation of a follow-up commit to merge acl entries and
detect supersedes from supernets, but these changes are backwards
compatible with the previous CMP() if being used as a comparison
function for which only negative, zero and positive result are
relevant.

The A in CMPA() stands for "adjacent". CMPA() returns -3/3 for left
of/right of.

Reverts d89674d1
Reinstantiates bbb5db47

sorry for the glitch

Test case by Martin Blix Grydeland, taken from #4013

Since Python 3.12 unknown escape sequences trigger a warning.

A VCL failure on the client side transitions to vcl_synth, except
failures from vcl_synth that lead to minimal errors. The ESI transport
is not allowed to reply with minimal responses so this would lead to a
panic.

On top of that, the vcl_req_reset feature flag emulates `return (fail)`
statements when an HTTP/2 client disconnected, resulting in the same
panic scenario.

For sub-requests, we masquerade the fail transition as a deliver and
trade the illegal minimal response for the synthetic response.

Fixes #4022