We should apply the same restrictions on the list of allowed characters inside
H/2 pseudo-headers as we do for H/1. This error is translated into the
headers we send to a backend over H/1.

Failure to do so could permit various exploits against a backend not handling
malformed H/1 requests.
Signed-off-by: Asad Sajjad Ahmed <asadsa@varnish-software.com>

This expands the perfect hash lookup table to be able to match any entry
in the list of well-known headers from tbl/http_headers.h.

Previously only the headers that had a non-zero filter flag section was
kept in the fast match table.

Fixes: VSV00010

We do not need to periodically check the conditions in
obj_extend_condwait() if we ensure that HSH_Cancel() triggers a
wakeup.

Readers on private objects need to ensure they call HSH_Cancel() if
they abort the read, so this is consistent with our requirements.

While we could add a seperate function to the object API for the sole
purpose of signaling a cancel, HSH_Cancel() already calls
ObjWaitState() to syncronize with the backend thread, so adding the
signal there was deemed the simpler solution.

As agreed during bugwash, Martin wants to add back the timeout
as an optinal parameter (default: no timeout).

To override the global parameter on a per-fetch basis.
Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

It pauses the fetch progress when clients are lagging behind for
uncacheable streaming deliveries.
Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

This clarifies which length this field is about.

Better diff with the --word-diff --word-diff-regex='\w+' options.
Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

(May not work due to termcap/terminfo differences)

Due to string folding, I expect this change to be fully backwards compatible
- even with std.integert(req.xid) as proven by the vtcs which I am about to
change.

found by flexelint

because we run VRY_Validate() anyway before keeping a vary spec,
and rightly so.

Ref #3858

In dea05fa4, an assertion was added
that, if a vary prediction exists as marked by vary_l, that marker
matches the actual length.

This change uncovered an inconsistency where, for waitinglist returns,
the predictive vary was cleared unconditionally (req->vary_b[2] = '\0'
in VRY_Prep()), but vary_l was not.

Consequently, before dea05fa4,
VRY_Finish() would copy more of the (possibly empty) vary spec
than necessary.

This commit fixes the inconsistency and adds specific testing
for the case.

Fixes #3858

vary, req ttl/grace limits, storage

Motivated by #3858

Refs 8df30240

Fixes #3856

Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune@gmail.com>

The panic code would panic again because a backend object which it no
longer owned was still referenced in thread local storage (don't we
all love abiguous acronyms?).

Example:

 #5  0x0000563d8c20d595 in VAS_Fail (func=0x563d8c23fd13 "vsl_sanity", file=0x563d8c23f9f5 "cache/cache_shmlog.c",
     line=110, cond=0x563d8c23fd1e "(vsl->wlp) != 0", kind=VAS_ASSERT) at vas.c:67
 #6  0x0000563d8c1603e9 in vsl_sanity (vsl=0x7fabd08001f0) at cache/cache_shmlog.c:110
 ...
 #7  0x0000563d8c160298 in VSL_Flush (vsl=0x7fabd08001f0, overflow=0) at cache/cache_shmlog.c:314
 #8  0x0000563d8c14dd44 in pan_ic (func=0x563d8c23c596 "child_signal_handler",
     file=0x563d8c23c1b4 "cache/cache_main.c", line=323,
     cond=0x7fabdf24a590 "Signal 6 (Aborted) received at 0x3e800015528 si_code -6", kind=VAS_WRONG)
     at cache/cache_panic.c:814
 ...
 #19 0x0000563d8c14bc07 in ObjBocDone (wrk=0x7fabdc9e54b8, oc=0x7fabd0024180, boc=0x7fabdc9e4778)
     at cache/cache_obj.c:368
 #20 0x0000563d8c13f25b in HSH_DerefBoc (wrk=0x7fabdc9e54b8, oc=0x7fabd0024180) at cache/cache_hash.c:1014
 #21 0x0000563d8c13158f in VBF_Fetch (wrk=0x7fabdc9e54b8, req=0x7fabd0008b20, oc=0x7fabd0024180, oldoc=0x0,
     mode=VBF_PASS) at cache/cache_fetch.c:1204
 #22 0x0000563d8c158f7d in cnt_pass (wrk=0x7fabdc9e54b8, req=0x7fabd0008b20) at cache/cache_req_fsm.c:742
 #23 0x0000563d8c156f53 in CNT_Request (req=0x7fabd0008b20) at cache/cache_req_fsm.c:1182
 #24 0x0000563d8c198f9a in HTTP1_Session (wrk=0x7fabdc9e54b8, req=0x7fabd0008b20) at http1/cache_http1_fsm.c:390
 #25 0x0000563d8c1984e0 in http1_req (wrk=0x7fabdc9e54b8, arg=0x7fabd0008b20) at http1/cache_http1_fsm.c:88
 #26 0x0000563d8c189592 in Pool_Work_Thread (pp=0x7fabdc400140, wrk=0x7fabdc9e54b8) at cache/cache_wrk.c:487
 #27 0x0000563d8c188ca7 in WRK_Thread (qp=0x7fabdc400140, stacksize=81920, thread_workspace=2048)
     at cache/cache_wrk.c:153

Strictly speaking, these calls were already missing before
3a22f5f0, but now that bgthreads are
officially allowed to terminate, we should really finalize & cleanup
the wpriv and stats structs to provide a consistent interface.

isn't this something we would have wanted all along?

This simplifies the common pattern to allocate/initialize a miniobj
on the workspace to

	WS_TASK_ALLOC_OBJ(ctx, myobj, MYOBJ_MAGIC);
	if (myobj == NULL)
		return (error);

Assert on the return value of VRY_Validate() and call it only once.
(I would like to think that we can trust memcpy())

Validate VRY_Validate()

Call VRY_Len() only once where the compiler does not eliminate the
second call anyway.

Trying to use zlib from a vmod would lead to conflicts because
the varnish-defined zlib symbols were found first.