And make vsc.am expect it as one of the input variables.

This is the convention for command line programs used in makefiles, like
CC, LD and plenty others. This is also what varnish.m4 exposes to out of
tree projects.

The new vsc.am file takes care of most of the logic, and include sites
only need to reference sources as such.

This change adds a new /usr/share/varnish/vcc (or equivalent) directory
where the built-in VMOD descriptors are installed. Only vmod_debug.vcc
is omitted since vmod_debug is not installed. Shipping VCC files can be
useful to end users, for example, to implement VCL autocompletion in a
text editor.

This change does not even begin to manage this for third-party VMODs.

With this, the documentation becomes authoritative for the VCL state
machine, just like VCL variables.

This also belongs in a reference manual and should not be disjoint. It
also avoids broken references with rst2man that only sphinx can resolve
across multiple pages.

Instead of documenting built-in subroutines in the user guide, a new
vcl-step(7) manual is added. It is structured with a vcl-step.rst file
that includes a vcl_step.rst file containing the actual list of steps.

This is similar to how the vcl-var(7) manual is constructed to later
make it authoritative.

The 304 handling and initial values for beresp.ttl, beresp.grace and
beresp.keep belong in a reference manual.

Transports should be free to keep a reference on the object to be
delivered until after their transport function returns, but
HSH_Cancel() in cnt_transmit() prevented the object from being of any
use for the case that it is final (pass/hfm/hfp).

We solve this by moving the HSH_Cancel() close to VDP_Close, which
also makes sense from the perspective of the VDP design: Until the VDP
close, filters could still reference object data.

HSH_Cancel() needs the objcore, which could be reachable also via
vdc->req. But that member is unset in VDP_DeliverObj(), presumably to
make it clear that VDP .bytes callbacks should not access request
data. Thus, we pass it as a new argument to VDP_Close() as well as any
boc being held by the caller.

The objcore can also be NULL for the case where a transport generates
the body without holding an objcore at all.

To potentially emit VCL_trace VSL records, we called VPI_count() from
VGC for, in the extreme, every line of VCL.

For a normal setup, VPI_count() would call into VSL only to find out
that VCL_trace is masked and not have any effect.

Issuing two additional function calls for each line of VCL obviously
is something we should avoid unless there is a real benefit.

To avoid the overhead, we move the check for the tracing flag to VGC
and decorate it with UNLIKELY(), which has no effect for now but is
expected to become a branch prediction hint soon.

This should bring the cost down close to zero.

Being at it, we also add VCL control over tracing through req.trace
and bereq.trace. req.trace gets initialized from feature +trace, and
bereq.trace inherits the value from req.trace. For now, tracing of the
housekeeping subs vcl_init and vcl_fini can only be controlled through
feature +trace.

not yet complete, I just had this change locally and want to avoid
repeating work on it.

Forgot to update the doc after changing scopes names to vcl_*

... and use the pedantic VDP to test that we do.

This should plug some less relevant memory leaks from VDPs which
allocate heap memory.

Note that we should not close VDPs for the happy path to allow
transports to keep them open until after cnt_transmit returns (e.g. in
vmod_pesi).

Calling VDP_Close() from cnt_transmit() avoids repetition for error
handling in transports.

The VDP now registers a PRIV_TASK to ensure that. The VDP fini method
is to be called before the task ends, so the priv_task fini method
checks that it ran.

When asserting that the context is a specific sub, we can compare equal.

Ref 478e310a

to facilitate custom filter parametrization

This requirement was dropped in the updated rfc 9113.

Fixes: #3911