The use case are cluster requests: Intra-cluster bgfetches should
trigger a synchronous fetch on the peer-varnish in order to avoid
additional short-lived / expired objects being created. Or, in
other words, a bgfetch should actually get a fresh object and not
one in grace from another cache.

Merges #2376

This is a back port of d7f8b531.

Conflicts:
	bin/varnishd/cache/cache_fetch.c
	doc/changes.rst
	include/tbl/bo_flags.h

Fixes #2380.

Closes: #2357

Related to #2337 and #2366.

From Emmanuel Hocdet (manu-at-gandi-dot-net) via #2373.

This fixes a denial of service attack vector where bogusly large chunk
sizes in requests could be used to force restarts of the Varnish
server.

This is Varnish Security Vulnerability VSV00001

For more information visit: https://varnish-cache.org/security/VSV00001

Fixes: #2379

like ban(req.http.non-existing-header)

Probes currently running on a worker thread at the time they are
deleted will delay the release of the refcount they hold on the TCP
pool. Since this call will not be from the CLI thread we need locking
to protect these datastructures.

This is a backport of 98676319 by Dridi Boukelmoune, but this patch
is a bit different from the original.

Now name collisions with transient will be reported in a sane way
instead of producing an assert. Normal name collisions are also more
readable.

The test case is unmodified, thanks to resent back porting efforts in
varnishtest.

Fixes #2321

Conflicts:
	bin/varnishd/storage/mgt_stevedore.c

This enables waiting for the logging thread to idle to make sure all
generated log is caught.

Conflicts:
	bin/varnishtest/vtc_varnish.c

Original test case by Federico G. Schwindt

Fixes: #2219

Conflicts:
	bin/varnishd/proxy/cache_proxy_proto.c

We now process the inclusive globs first, and then the exclusive
ones.  The former are accumulative, while the latter will filter
the output from the first.

Discussed in detail with and final diff from @mbgrydeland.
Tests tweaking by yours truly.

Fixes #2320.

Conflicts:
	bin/varnishstat/varnishstat_options.h
	bin/varnishtest/tests/u00005.vtc
	lib/libvarnishapi/vsc.c

Update the code to accommodate this.  Originally reported by Reinis
Rozitis (r at roze.lv) on varnish-misc.

Fixes #2118.

Backport to 4.1

Fixes #2022

Conflicts:
	bin/varnishstat/varnishstat_options.h

Backport to 4.1

object creation.

Fixes #1764

Conflicts:
	bin/varnishd/cache/cache_fetch.c
	bin/varnishd/storage/stevedore.c
	bin/varnishd/storage/storage.h
	bin/varnishd/storage/storage_lru.c
	bin/varnishd/storage/storage_persistent.c
	bin/varnishd/storage/storage_simple.c

Conflicts:
	bin/varnishd/cache/cache.h
	bin/varnishd/cache/cache_ws.c

The parsing still requires some TLC but this does not make it any
worse so let's get this in first.

Prompted by and ideas from github::xcir via #2077.

Change some tests to use the new parameter and more coverage.

be found are options to "shell".  Deprecate err_shell.

Structure output to always be prefixed by "shell" since we're running
at the top level.

Also prefix commands with "exec 2>&1 ;" to join stdout+stderr by default.
This still doesn't cover the shells stderr in case of bad shell syntax,
which is annoying, but not worth fighting for right now.