... hopefully, finally

The previous code pushing additional data was actually working around
other bugs, which _should_ be fixed with
4f9db605 and
6b5c4e45 as shown by permutations.vtc

This should avoid running into #3109 on 6.x

permutations-error.vtc triggers the bad assertion failure fixed in the
previous commit

The previous commit went in the right direction, but was not correct.

For CHK_ORDER, we must only ever look at type != T_NEXUS and check any
T_NEXUS by the criteria originally passed.

CHK_ORDER could be stricter still, after a subtree is completed, we
could communicate up the stricter CHK_PEND, but I do not want to
complicate things further for an assertion.

we do not unpend T_NEXUS nodes, so the following situation is perfectly
legal:

	T_NEXUS
	ST_CLOSED

T_NEXUS		T_DATA
ST_CLOSED	ST_UNPENDING

the purpose of the assertion is that, if a node below a nexus is
not yet unpended/delivered, the following need to be also, but this is
not true for T_NEXUS.

We used to set empty T_DATA nodes to delivered as soon as we encountered
them during unpending. While it seemed like a good idea to not spend
additional work with nodes which do not contain anything, the tree
integrity assertions as well as set_deliver() implicitly assume that

- a delivered node cannot follow an undelivered node unter a nexus and
- once the last node under a nexus is delivered, all of the nexus is
  delivered

Rather than sacrificing the sensible principle that a tree always is to
be delivered top to bottom, left to right, weakening our assertions and
possibly introducing new bugs when handling the resulting special cases,
we go for the simple, clean option and do away with a special case which
will not contribute much to overall performance anyway.

The log for permutations.vtc is too large for the default.

Avoids starting another process.

Avoids the clutter in /tmp.

we run into the problem solved by
https://github.com/varnishcache/varnish-cache/pull/2942

purely for convenience, we send pass'ed backend requests to ourself (in
order not to have to fiddle with the number of requests on servers), yet
it can happen that all threads are used up and the herder fails to breed
more.

So the solution is to ensure that we do not run into any dependency on
the herder.

this stabilizes permutations.vtc for -j 40 -n 1000

The src argument of vped_include() is owned by the caller
vdp_pesi_bytes(), so we need to make a copy on the subrequest's
workspace.

This is not an issue with varnish-cache ESI because the include is
not handed over to a different thread.

we should not emit ReqUnset / ReqHeader if A-E is already correct

reflects 6dbfff12dca895534b090ef9283e75f85659fcfa in varnish-cache

to fail earlier for a bug exposed by this test which surfaces as garbled
URLs.

Also add a test to demonstrate that the error can happen with serial
ESI, so the problem does not depend on pESI, and is not caused by
the VDP.

The moral of the story: don't set beresp.filters without testgunzip
when the beresp is gzipped, or just let varinshd set the default
VFP stack.

in vped_deliver, we used the wrong gzip state pointer for ESI objects,
which lead to the pretendgzip filter not being pushed.

Also, we should respect the RES_ESI bit to properly handle the case when
esi is deactivated on an esi object. (to be tested)

move a block of initializations statements which should run at all
levels

the continue statement continued the inner loop, in fact the while
(node->state >= ST_UNPENDING) was intended to be continued.

I could neither convince myself that this can not cause an infinite loop
nor could I convince myself that it does.

https://github.com/varnishcache/varnish-cache/pull/2796

Before the bugfix for Varnish #3109
(d235b3c90a631ef39fdf0a8103e44ebfb0ddbacb), the gunzip VDP went into
an infinite loop for such a case.