This cursor is the simplest of all as of now, and also the fastest way
to skim through a -r input when it's a regular file. This is essentially
a file cursor with much less memory churn.

On a 13GB log dump captured during a performance benchmark varnishlog
takes around 6m15s to print all the transactions to /dev/null but the
mmap cursor consistently reduces this time to around 1m01s.

This is particularly useful to audit logs at rest, be it error logs
after an incident or transactions captured during a benchmark-type
scenario.

The shard director had the limitation to only support adding/removing
backends of one instance before `.reconfigure()` on that instance had
to be called, then the next instance could be changed.

This limitation came from the time when it was still unclear that
calling `VRT_priv_task()` on a per-object pointer was the accepted
interface for per-instance privs (varnish historians' recommended read:
https://github.com/varnishcache/varnish-cache/wiki/VIP1:-PRIV_*-visibility-and-lifetime-control
)

It is about time to lift that limitation now. o/ Dridi

Ref #3407

We now validate all header set operations to conform with the allowed
characters by RFC7230:

* HTAB 0x09
* VCHAR 0x20 to 0x7e
* obs-text 0x80 to 0xff

Ref https://httpwg.org/specs/rfc7230.html#header.fields

See #3407

I accidentally committed an earlier, wrong version. o/ Dridi

Ref #3407

https://github.com/varnishcache/varnish-cache/pull/3407#issuecomment-696146625

Ref: https://httpwg.org/specs/rfc7230.html#header.fields

  field-value    = *( field-content / obs-fold )
  field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]
  field-vchar    = VCHAR / obs-text

- we do not accept the obsolete line folding (obs-fold)
- VCHAR is 0x20 to 0x7e
- obs-text is 0x80 to 0xff

So we end up with !VCT_CTL || HTAB (LF)

This change would be the basis for new use cases, like

- a type method .valid_header
- a feature flag +validate_headers which would call VRT_ValidHdr
  for all SetHdr calls

Spotted by Martin.

It makes it easier to substitute the hash director with say, the round
robin director, when weight is not a concern.

This only concerns VDP_END at esi_level > 0, which I considered not
relevant until Martin pointed out that it is.

the sole purpose is to ensure that we send a VDP_END.

Later, we might consider to arm the assertion in VDP_Close(). At this
point, a couple of VTCs fail. We would need to at least improve error
handling.

... to the extent possible today. See comment in the VTC for more
background.

Ref Martin's comments in #3298

de6288cb was incorrect in that it could
generate multiple VDP_ENDs, which did not show up during testing because
of the buffer being large enough to fit any testcase.

Make sure such issues pop up with rot13 testing by using a deliberately
minuscule buffer.

(and, yeah, we could take it from workspace, but the point is to
 demonstrate and test VDPs...)

It added a random byte.

We use the `final` argument in object iteration to denote
`OC_F_PRIVATE | OC_F_HFM | OC_F_HFP`.

As we now have `VDP_END` to mark the final bit of data, so should the
iterator flag be named `_END`. This also parallels `VDP_FLUSH` vs.
`OBJ_ITER_FLUSH`.

Supported by martin

Ref #3298

sigsegv_handler is on by default since 2015, ref
1955e460

noticed via argv dump in panic d0e986a6

This is solely to make bug reports more useful

Some browsers are strict about this and simply drop responses containing
such headers. Since this is not filtering a context switch between a
client and a backend transaction (or cache hit) a new filtering flag is
added to the HTTP headers table for connection-specific headers. This
new flag cannot be compounded as HTTPH_R_FETCH|HTTPH_A_INS because the
TE header is an exception and left alone, even though trailers aren't
supported.

Better diff with the --ignore-all-space option.

We could go further and consider any client request containing one as
malformed as mandated by RFC 7540.

Closes #3416

Close #3417

They either take both a dynamic struct and buffer, or both are static.

For niov == 0, we reserve up to IOV_MAX io vectors, otherwise niov io
vectors capped to IOV_MAX.

We used to handle the two cases differently and delay the WS_Release for
niov == 0 until V1L_Close(). There is no need to do so, in both cases we
can just reduce the reservation (turned into allocation) to the amount
of space we actually use.

As we are rolling back the workspace in V1L_Close(), this neither has
any influence on the state after the close.

to clarify that it holds a workspace snapshot

Once we ask the backend to close the connection after a given request
there is no benefit from putting the backend connection back in the
pool. It's actually a surefire way to force a subsequent backend fetch
to fail its first attempt and go straight to its extra chance.

Since we try to maximize connection reuse this would have to come from
VCL and a user asking for the backend to close the connection should
have a good reason to do so, for example when the backend is known to
misbehave under certain circumstances.

Closes #3400
Refs #3405

I came across a VMOD that has a method named like the constructor to
conceptually reinitialize the object's state. That's an implementation
detail of the VMOD in question and arguably undefined behavior in
Varnish.

In order to not accidentally break this assumption, add some test
coverage and document this possibility.

- Complete the missing HTTP response codes / messages from :
https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml

ref 8816c6f9

so far, it is only used in assertions - this change enables use in
panic code.

motivated by #3415