Move most of the stat fields used in cache_center.c into the workers
private stats, hopefully reducing the lock contention and stats
precision at the same time.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4215 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4214 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4213 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4212 d4fa192b-c00b-0410-8231-f00ffab90ce4

The cruical difference is that we can remove from a VLIST without
having the head of the VLIST, which comes handy when expiring a
non-instantiated object from a -spersistent stevedore.

We flip the LRU list around, so that the frontmost element has not
been accessed for the longest time, and simulate tail-inserts by
putting a senteniel on the tail of the list.

Presently the senteniel is a full objcore, which is a non-issue with
few machines having more than a handful of stevedores, but once the
persistent stevedore moves to per-segment LRU lists, this needs to
be addressed, probably by wrapping the LRU linkage in a separate
structure.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4211 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4210 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4209 d4fa192b-c00b-0410-8231-f00ffab90ce4

This allows us to put items on a blacklist for a specific backend based on
the response it gave us. The basic syntax is (vcl_fetch):

if (beresp.status == 500) {
	set beresp.saintmode = 20s;
	restart;
}

Health checks are modified accordingly, which means graced objects are used
if no healthy backends are left to try.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4208 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4207 d4fa192b-c00b-0410-8231-f00ffab90ce4

and try to push objects away from the tail end so that the list
can be shortened.

It won't work if your bans contains any req.* tests, because the
lurker has no request to compare against.

By default the lurker is disabled, you enable it by setting the
'ban_lurker_sleep' parameter to the amount of time the lurker
should sleep between doing things.

If it cannot do anything, it always sleeps for a second.

NB: This feature is on probation, there is currently no guarantee
it will end up in a future varnish release, it depends on the
impact and usability.  Feedback much anticipated.


One of the quasi-nasty details of this feature, is the need to work
against the lock-order in cache_hash.c, see details in comments for
HSH_FindBan().




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4206 d4fa192b-c00b-0410-8231-f00ffab90ce4

the reference.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4205 d4fa192b-c00b-0410-8231-f00ffab90ce4

it was possible to have infinite recursion between vcl_error returning restart and cnt_recv calling vcl_error when you had too many restarts

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4204 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4203 d4fa192b-c00b-0410-8231-f00ffab90ce4

very first functions ever written in Varnish.   Since I knew
that the file stevedore would always return the size you asked for,
or panic, I didn't bother to actually check how big the storage
segment he allocated was, I just knew it would be the right size.

Guess what, segments are finite size in the -spersistent stevedore
so you may in fact _not_ get what you ask for.

(Cue music: Theme Music from The Keystone Kops)

DuH!


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4202 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4201 d4fa192b-c00b-0410-8231-f00ffab90ce4

Don't explode manager if it gets overwritten.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4200 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4199 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4198 d4fa192b-c00b-0410-8231-f00ffab90ce4

Add test-case for same.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4197 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4196 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4195 d4fa192b-c00b-0410-8231-f00ffab90ce4

for expiry of non-instantiated objects from -spersistent



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4194 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4193 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4192 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4191 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4190 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4189 d4fa192b-c00b-0410-8231-f00ffab90ce4

first, so combine them into one and avoid the issue entirely.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4188 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4187 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4186 d4fa192b-c00b-0410-8231-f00ffab90ce4

This is the number of good probes we pretend to have already seen when
we start up, in order to speed up getting healthy backends.

The default value is one less than the .threshold, so the backend
will be set healthy if it manages to respond correctly to the very 
first probe we send to it.

(A bit of this commit leaked in during r4184)



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4185 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4184 d4fa192b-c00b-0410-8231-f00ffab90ce4

so that we do not RST connections that have still not transmitted their
data.

Since we were able to get the writev(2) to detach the socket, we should
not end up sleeping in the close(2) either.

We still RST the socket for all error conditions.

Ideally I would still like to RST connections that have no outstanding
data after their sess_timeout, in order to avoid the 2*RTT+misc
timeouts delays associated with loosing a TCP socket for a client
that have gone to meet some other IP#.

In particular with load-balancers, this allows the load balancer to
declare the session dead right away, and reuse it for something more
productive.

Unfortunately, this lacks OS support in all presently released
OS'es: you cannot ask if a socket is done transmitting what you
asked it to.

FreeBSD-8.0 will have experimental support for this (FIONWRITE)
and I will revisit it in that context.




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4183 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4182 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4178 d4fa192b-c00b-0410-8231-f00ffab90ce4

to recycle



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4177 d4fa192b-c00b-0410-8231-f00ffab90ce4

Add a complementary vsb_unquote() function.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4176 d4fa192b-c00b-0410-8231-f00ffab90ce4

Reinstate code to embed the svn version (or git commit id) in the
output given by varnishd -V.

Also, make sure we don't overwrite version.c when the tree is
exported, but generate it if it's completely missing.  (This should
only happen if somebody does svn export rather than make dist, but
some people might do that.)

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4175 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4174 d4fa192b-c00b-0410-8231-f00ffab90ce4

varnishtop used to have a hard-coded max record length of 255 bytes.
This broke with the change that allows larger record length.  Use a
dynamic buffer instead.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4173 d4fa192b-c00b-0410-8231-f00ffab90ce4