Associate ascii representation addr/port strings with each listen
socket, already in the manager process.

Postpone listen(2) call until client is ready, to limit initial 
connection spike.

XXX: I still miss an unlisten(2).



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2658 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2657 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2656 d4fa192b-c00b-0410-8231-f00ffab90ce4

Have the worker threads add themselves to the pool, while the hold the pool
lock anyway.

Collect stats per pool, and have the decimator aggregate over all pools
and into the shared memory counters.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2655 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2654 d4fa192b-c00b-0410-8231-f00ffab90ce4

This is necessary to generalize the thread pools to do other tasks
for us in the future.

Please read the descriptions of the new and changed thread_pool*
parameters carefully before you tweak them, some of them have
slightly different meanings now.

The high-level view of this is that we now have dedicated a thread to
adding threads to the pools, in addition to the thread we already
had that killed idle threads from the pools.

The difference is that this new thread is quite a bit more reluctant
to add threads than the previous code, which would add a thread any
time it could get away with it.

Hopefully that reduces the tendency for thread-pile-ups.

This commit also reduces the cross-pool locking contention by making
the overflow queue a per pool item.

The down side of that is that more of the stats counters have become
unlocked and thus can get out of sync if two threads race when updating
them.  This is an XXX item.

Thanks to Anders Nordby for testing this patch.




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2653 d4fa192b-c00b-0410-8231-f00ffab90ce4

unwritten lore better.

Closes 203


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2652 d4fa192b-c00b-0410-8231-f00ffab90ce4

I had the sense of FIONBIO args right in the first place.

Revert 2647 and promise not to commit when sleepy.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2651 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2650 d4fa192b-c00b-0410-8231-f00ffab90ce4

Don't hand-roll an assert.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2649 d4fa192b-c00b-0410-8231-f00ffab90ce4

bit.

Automatically calculate the margin, based on the width of the parameter
names we have.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2648 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2647 d4fa192b-c00b-0410-8231-f00ffab90ce4

seconds instead of cleaning it in every new connection received.
We are using it here, and the performance is much better now.

Submmitted by:	Rafael Umann <rafael.umann@terra.com.br>

Closes ticket #235



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2646 d4fa192b-c00b-0410-8231-f00ffab90ce4

Overhaul the regexp purge/ban code, with a detour around the CLI code.


CLI code:

	In CLI help, don't list commands with no syntax description.

	Add a CLI_HIDDEN macro to construct such entries.  They are
	useful as backwards compatibility entries which we do not want
	to show in the help.

CLI interface to BAN (purge) code:

	Get the CLI names right for purging so they are purge.FOO instead
	of FOO.purge.

	This means that you should use "purge.url" and "purge.hash"
	instead of "url.purge" and "hash.purge".

	Add compat entries for the old, and keep them through the 2.x
	release series.
	

	Add purge.list command to list purges currently in effect.
	NB: This is not 100% locking safe, so don't abuse it.


BAN (purge) code:

	Add reference counting and GC to bans.

	Since we now have full reference counting, drop the sequence
	number based soft references and go to "hard" pointer
	references from the object to the purges.

	Give the "ban" structure the miniobj treatment while we are
	at it.

	The cost of this is a lock operation to update refcounts
	once all applicable bans have been checked on an object.

	There is no locking cost if there is no bans to check.

	Add explicit call to new BAN_DestroyObj() when objects are
	destroyed to release the refcount.

	When we release an object refcount in BAN_DestroyObj(),
	check if we can destroy the last purge in the list also.

	We only destroy one ban per BAN_DestroyObj() call, to avoid
	getting stuck too long time, (tacitly assuming that we will
	destroy more objects than bans.)



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2645 d4fa192b-c00b-0410-8231-f00ffab90ce4

Implement "quit" (ticket 125) and drop the planned aliases.

Add CLIS_CLOSE return status, to communicate that close is happening.

Fix logical bug in fd-table allocation bug in event-engine.

Plug memory leak in CLI session destruction.

Increase listen depth of telnet socket to 10.

Ignore trailing whitespace and empty CLI input lines.

Log CLI sessions and commands into syslog.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2644 d4fa192b-c00b-0410-8231-f00ffab90ce4

(no ticket)



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2643 d4fa192b-c00b-0410-8231-f00ffab90ce4

This is how long time we wait for a TCP connection to the backend to
become established.

Typical usage:

		backend b1 {
			.host = "hex";
			.port = "80";
			.connect_timeout = 500 ms;
		}

It can also be used in backends in director declarations.

Also add a parameter called "connect_timeout" which sets the default
to 400 msec (a number pulled out of my old black magicians hat).




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2642 d4fa192b-c00b-0410-8231-f00ffab90ce4

any connections at all, that they didn't even consider that maybe
connect(2) should have a timeout argument.

Add TCP_connect() which addresses this shortcoming, using what I
belive is a widely supported workaround.

Not to POSIX: please fix.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2641 d4fa192b-c00b-0410-8231-f00ffab90ce4

It works just like regsub(), but replaces all occurrences of the regexp.

	regsub("1foofoofoo2", "foo", "bar") -> "1barfoofoo2"

	regsuball("1foofoofoo2", "foo", "bar") -> "1barbarbar2"

Fixes ticket 238



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2640 d4fa192b-c00b-0410-8231-f00ffab90ce4

Accept sockets are non-blocking, to avoid races where the client closes
before we get to accept it.  (Spotted by: "chen xiaoyong")

Unfortunately, that means that session sockets inherit non-blocking mode,
which is the opposite of what we want in the worker thread but correct
for the acceptor thread.

We prefer to have the extra syscalls in the worker thread, that complicates
things a little bit.

Use ioctl(FIONBIO) instead of fcntl(2) which is surprisingly expensive.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2639 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2638 d4fa192b-c00b-0410-8231-f00ffab90ce4

fondling fcntl(2) directly.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2637 d4fa192b-c00b-0410-8231-f00ffab90ce4

Closes ticket #239


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2636 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2634 d4fa192b-c00b-0410-8231-f00ffab90ce4

(not-RFC2616 compliant) request.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2633 d4fa192b-c00b-0410-8231-f00ffab90ce4

are passed to it.

If no method is provided, we fall back to vca_pipes.

closes #227


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2632 d4fa192b-c00b-0410-8231-f00ffab90ce4

Spotted by:	"postwait"


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2631 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2630 d4fa192b-c00b-0410-8231-f00ffab90ce4

debugging:

Implement an upper limit for allocated memory, specify as:
	-smalloc,40M

Give the malloc allocator its own stats counters so we can see deeply
into what it does.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2629 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2628 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2627 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2626 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2618 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2617 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2616 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2615 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2614 d4fa192b-c00b-0410-8231-f00ffab90ce4

Two really bad ones in Vary processing, found by Arjan (noosius).

One isignificant one related to -b arguments being wrong,
found by FlexeLint after I taught it how to spot this kind of issue.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2613 d4fa192b-c00b-0410-8231-f00ffab90ce4

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2612 d4fa192b-c00b-0410-8231-f00ffab90ce4

"Authentication" for this long without anybody noticing until now.

Mea Culpa.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2611 d4fa192b-c00b-0410-8231-f00ffab90ce4