git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3655 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3654 d4fa192b-c00b-0410-8231-f00ffab90ce4

Add preliminary version of lock-less tree based lookup (see below)

Enable SHA256 digests by default, and put it in the objhead.  This
increases the size of the objhead by 32 bytes, but may drop
a bit again later, when other now unnecessary fields go away.

Test SHA256 for correct operation on startup.

About the "critbit" lookup:

To enable this, use "-hcritbit" argument.

"Crit Bit" trees, are also known under various other names, the original
version of the idea is probably the PATRICIA tree.

The basic concept is a tree structure which has nodes only where necessary
to tell the indices apart.

Our version of it, has some additional bells and whistles.

First lookups do not require any locks until we reach the objhead
we were looking for, or until we need to insert one which wasn't
there.

Second, the branch nodes are part of the objhead, as all but the
very first will need one, this saves malloc operations big time.

Now the down-sides:

There are still missing bits, amongst these the "cooling off" list,
for objheads that have been dereferenced, but where the branch-node
is not.  Currently we just leak that memory.

There is a race relating to node deref and unlocked lookup that is
not closed, weird things may happen until I fix it.

I'd be interested to hear how long it survives before it croaks,
but apart from that, would not advocate that you use it, until
I fix those remaining issues.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3653 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3652 d4fa192b-c00b-0410-8231-f00ffab90ce4

Fix an embarrasing bug in my Flexlinting of this code yesterday, and
add a couple of test-vectors to avoid it happening again.

And now for the funny and educational story:

In july of 1994, I added the "libmd" to FreeBSD, containing the
MD2, MD4 and MD5 functions from RFC 1319, RFC 1186 and RFC1321.

I meticulously replicated the test-vectors from the RFCs, so that
"make test" would validate the result.

Duing the intermediate 14 years, various slight shifts and adjustments
to things like the make(1) programs defaults, the shared library
resolution algorithm and other totally unrelated things, meant that
"make test" now tests the installed version of the library, rather
than the version you just built with "make all".

Needless to say, when I tested my patch yesterday, I didn't install
the built version, wanting first to hear what Colin Percival, FreeBSD
Security Wiz, generally swell fella and the guy who wrote this
SHA256 implementation, thought of these "stylistic" patches.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3651 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3650 d4fa192b-c00b-0410-8231-f00ffab90ce4

Shut FlexeLint up about SHA256 implementation.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3649 d4fa192b-c00b-0410-8231-f00ffab90ce4

We cannot gain a reference to an object unless it has an objhead,
assert that we have one.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3648 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3647 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3646 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3645 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3644 d4fa192b-c00b-0410-8231-f00ffab90ce4

Make it possible to declare paramters in other source files, and move
the thread-pool related params into a new file mgt_pool.c as proof.

The paramters happen in management process context, and should therefore
not end up in cache_* files for namespace and sanity reasons.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3643 d4fa192b-c00b-0410-8231-f00ffab90ce4

Name that file "vparam.h" instead of "param.h" which is just too
generic for comfort.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3642 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3641 d4fa192b-c00b-0410-8231-f00ffab90ce4

now.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3640 d4fa192b-c00b-0410-8231-f00ffab90ce4

Try to get the endianess optimization working, by including an assortment
of possibly relevant headers and only go with the fast path if we have
credible information that this is a big-endian platform.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3639 d4fa192b-c00b-0410-8231-f00ffab90ce4

This does not do anything yet.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3638 d4fa192b-c00b-0410-8231-f00ffab90ce4

Hook SHA256 into the build

Neuter the FreeBSD specifics of SHA256 implementation.

In the end, it comes down to lack of POSIX definition of a way to find
out byte-endianess, sigh...



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3637 d4fa192b-c00b-0410-8231-f00ffab90ce4

This code was written by Colin Percival for the FreeBSD project.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3636 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3635 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3634 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3633 d4fa192b-c00b-0410-8231-f00ffab90ce4

Rename the write-buffering functions to WRW_*().

Make reservation and release explicit.

Add asserts that this it happens.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3632 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3631 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3630 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3629 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3628 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3627 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3626 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3625 d4fa192b-c00b-0410-8231-f00ffab90ce4

This isolates the mangement process from the compilers bugs
and memory usage.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3624 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3623 d4fa192b-c00b-0410-8231-f00ffab90ce4

The VCL file specified to -f must be read relative to the directory
from which varnishd is started, before we chdir to the workdir.

We used to deal with this by opening the file and passing the file
handle down.  It's simpler to just read the file and pass the actual
VCL code down.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3622 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3621 d4fa192b-c00b-0410-8231-f00ffab90ce4

Move the code for running stuff in a sub-process out to a library
function, and give it the ability to limit how many lines of
output we get back from the subprocess, in order to muzzle the
C-compiler somewhat.




git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3620 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3619 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3618 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3617 d4fa192b-c00b-0410-8231-f00ffab90ce4

Added support for setting read timeouts for backend requests
(first_byte_timeout and between_bytes_timeout), in addition to make the
connect_timeout available for the bereq object in vcl_miss and vcl_fetch.

first_byte_timeout is a read timeout from the connection to the backend is
created to when the first byte arrives. It can be set as a parameter to
varnish, as a field in the backend declaration or as bereq.first_byte_timeout
in vcl_miss and vcl_pass.

between_bytes_timeout is a read timeout between each read from the backend. It
can be set as a parameter to varnish, as a field in the backend declaration or
as bereq.between_bytes_timeout in vcl_miss and vcl_pass.

The time unit for these timeout values are seconds. NOTE: The connect_timeout
previously used milliseconds as time unit, so beware.



git-svn-id: http://www.varnish-cache.org/svn/branches/2.0@3616 d4fa192b-c00b-0410-8231-f00ffab90ce4