Commits · 76bc7522b12c6afb17e2ad8b40f94ad3c4307053 · uplex-varnish / unique-xids

30 Jul, 2008 3 commits
- Add statistics about regexp purges. · 76bc7522
  Poul-Henning Kamp authored Jul 30, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3032 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
  76bc7522
- Try to connect to CLI 30 times rather than 10 to avoid some timeout errors. · b571db1a
  Tollef Fog Heen authored Jul 30, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3031 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
  b571db1a
- Add cache_backend.h to the list of headers that go in the tarball; fixes build failure · 7e1b629f
  Tollef Fog Heen authored Jul 30, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3030 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
  7e1b629f
27 Jul, 2008 1 commit

Forgot to add this file. · ff200402

Poul-Henning Kamp authored Jul 27, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3029 d4fa192b-c00b-0410-8231-f00ffab90ce4

ff200402

25 Jul, 2008 8 commits

Change param thread_pool_min to be the minimum number of threads per · 61a4f7de

Poul-Henning Kamp authored Jul 25, 2008

pool, and don't be so stingy with them: default to five.

Make sure we really do create the minimum complement of threads.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3028 d4fa192b-c00b-0410-8231-f00ffab90ce4

61a4f7de

· 714e0ef6

Poul-Henning Kamp authored Jul 25, 2008

NB: Be careful with -trunk, this is a quite intrusive change.

Continue the progress on synthetic/error pages

The default "guru meditation" is now generated from the vcl_error {}
function, which can be redefined to whatever suits your taste.

Techinical:

Add new vcl_error method and make variables available in it.

Fix a bug in "error" action, where the numeric argument would not get
interpreted correctly if it came from a variable like obj.status.

Forbid "error" action in vcl_deliver since vcl_error now goes to
vcl_deliver to send the error object.

Fix test v00001 to reflect this.

Move the obj->ws/http initialization to preallocation time to avoid
duplication.

Add read-only variable req.xid (a string!)

Add default guru meditation to default vcl.

Remove cache_synthetic.c which is now unecessary.




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3027 d4fa192b-c00b-0410-8231-f00ffab90ce4

714e0ef6

Add a facility to constructing a synthetic object from VCL. · f2fb353a

Poul-Henning Kamp authored Jul 25, 2008

This is just the initial version and it doesn't quite work the way
we need it to yet, but the idea is to be able to say:

	sub vcl_fetch {
	    if (obj.status == 503) {
		synthetic {"
		    <HTML>
		    <H1>Sorry, could not contact the backend server</H1>
		    <P>
			Try again later.
		    </P>
		    <HR>
		    <PRE>
			URL: "}  req.url {"
			User Agnet: "} req.http.user-agent {"
		    </PRE>
		    </HTML>
		"};
	    }
	}

A new VCL syntactic element have been introduced to do this: "the
long string".   A long string is anything from {" to "}, newlines,
controlcharacters and all.  (Normal "..." strings cannot contain
control characters.

Technical details:

Don't NULL terminate string sequences in VRT context, we may have
NULL cropping up as a legal value if a header is missing, add the
magic "vrt_magic_string_end" for terminating string contactenation.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3026 d4fa192b-c00b-0410-8231-f00ffab90ce4

f2fb353a

Add a stevedore for synthetic pages. · d45b49f4

Poul-Henning Kamp authored Jul 25, 2008

The storage from this stevedore is backed by an autoextending vsb, which
can be used to generate synthetic pages into.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3025 d4fa192b-c00b-0410-8231-f00ffab90ce4

d45b49f4

Give VSB's the mini-obj/magic treatment and enable asserts in general. · a53a333e
Poul-Henning Kamp authored Jul 25, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3024 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
a53a333e

Neuter test b00015 so that automatic scripts don't trip on it all · 45a41f36

Poul-Henning Kamp authored Jul 25, 2008

the time.

To be reenabled when the code supports the desired functionality.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3023 d4fa192b-c00b-0410-8231-f00ffab90ce4

45a41f36

polish dot graph a bit. · b422f451

Poul-Henning Kamp authored Jul 25, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3022 d4fa192b-c00b-0410-8231-f00ffab90ce4

b422f451

Figure out if we need O_LARGEFILE to be compatible with 1990 or later · 1c4dbff8

Poul-Henning Kamp authored Jul 25, 2008

size disks.

Submitted by:	wichert

Fixes ticket 283



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3021 d4fa192b-c00b-0410-8231-f00ffab90ce4

1c4dbff8

24 Jul, 2008 13 commits

Relax an assert to only check what we really care about, and not the · 926479e6

Poul-Henning Kamp authored Jul 24, 2008

principle of the situation.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3020 d4fa192b-c00b-0410-8231-f00ffab90ce4

926479e6

Don my peril sensitive sun-glasses, and go over the TTL calculation · cb1e1507

Poul-Henning Kamp authored Jul 24, 2008

routine.

Follow RFC2616 more closely, and make relative (ie: max-age) specifications
take precedence over absolute specifications (Expires:)

Add an explicit clock-skew window (parameter: clock_skew) and only
interpret Expires: in absolute terms if the Date: header is inside
this window.

Explicit check for the case where the backend sends an Expires: which
is before the Date:, even if the Date: is out side our window.




git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3019 d4fa192b-c00b-0410-8231-f00ffab90ce4

cb1e1507

Make the ESI testcases start with an XML tag to satsify our new · 1fc6b15c

Poul-Henning Kamp authored Jul 24, 2008

non-XML detector logic.

Add a testcase to see if it works and that we can override it.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3018 d4fa192b-c00b-0410-8231-f00ffab90ce4

1fc6b15c

Use "localhost" in our acls, so it works no matter which address it · 19acd520

Poul-Henning Kamp authored Jul 24, 2008

picks off your loopback interface.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3017 d4fa192b-c00b-0410-8231-f00ffab90ce4

19acd520

Add a check to avoid doing ESI parsing of objects that do not look · 8e307799

Poul-Henning Kamp authored Jul 24, 2008

like XML. Our definition of "looks like XML" is that the first
non-white-space character is '<'.

Add a new parameter "esi_syntax" with bits to steer the ESI parser.

Use the first bit to disable the "looks like XML" check.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3016 d4fa192b-c00b-0410-8231-f00ffab90ce4

8e307799

Properly truncate bytes before hexdumping. · 967111bf

Poul-Henning Kamp authored Jul 24, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3015 d4fa192b-c00b-0410-8231-f00ffab90ce4

967111bf

Test caching of synthetic error pages (broken since 2266). This will need... · 124d7701

Dag Erling Smørgrav authored Jul 24, 2008

Test caching of synthetic error pages (broken since 2266).  This will need revisiting once caching has been reimplemented.


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3014 d4fa192b-c00b-0410-8231-f00ffab90ce4

124d7701

Set svn:keywords · 9551bf21

Dag Erling Smørgrav authored Jul 24, 2008


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3013 d4fa192b-c00b-0410-8231-f00ffab90ce4

9551bf21

Set svn:keywords on files that didn't already have it · de75e2c0
Dag Erling Smørgrav authored Jul 24, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3012 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
de75e2c0

Fix temp file names · 5c2afe96

Dag Erling Smørgrav authored Jul 24, 2008


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3011 d4fa192b-c00b-0410-8231-f00ffab90ce4

5c2afe96

Correct copyright statement. · 649f8e16

Dag Erling Smørgrav authored Jul 24, 2008


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3010 d4fa192b-c00b-0410-8231-f00ffab90ce4

649f8e16

Reindent · be86ae4c

Dag Erling Smørgrav authored Jul 24, 2008


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3009 d4fa192b-c00b-0410-8231-f00ffab90ce4

be86ae4c

Markup · 9dca1d4e

Dag Erling Smørgrav authored Jul 24, 2008


git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3008 d4fa192b-c00b-0410-8231-f00ffab90ce4

9dca1d4e

23 Jul, 2008 14 commits

Make the manager respect the diag_bitmap for child core creation. · 122f52cc
Poul-Henning Kamp authored Jul 23, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3007 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
122f52cc

Prevent loading multiple VCL's with the same name if the client is not · 7dfca207

Poul-Henning Kamp authored Jul 23, 2008

there to stop us.

Fixes #281



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3006 d4fa192b-c00b-0410-8231-f00ffab90ce4

7dfca207

Copy&Paste'o · 5171381d

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3005 d4fa192b-c00b-0410-8231-f00ffab90ce4

5171381d

More Flexelint nitpickery · db2508e5

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3004 d4fa192b-c00b-0410-8231-f00ffab90ce4

db2508e5

Flexelint nits · fa8a1e71

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3003 d4fa192b-c00b-0410-8231-f00ffab90ce4

fa8a1e71

Add a missing error check. · 15877042

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3002 d4fa192b-c00b-0410-8231-f00ffab90ce4

15877042

Note that this testcase may fail on linux. · 396b0451

Poul-Henning Kamp authored Jul 23, 2008

The problem is that a TCP connection to 127.0.0.1 appearantly does
not originate on the loopback interface, but on some other interface
on the machine.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3001 d4fa192b-c00b-0410-8231-f00ffab90ce4

396b0451

We should not open the shm file until we have a cli connection, it · 2ddd2027

Poul-Henning Kamp authored Jul 23, 2008

might not be there yet.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@3000 d4fa192b-c00b-0410-8231-f00ffab90ce4

2ddd2027

Prefix acl functions with "named_" and "anon_" to avoid naming conflicts. · 251678f5
Poul-Henning Kamp authored Jul 23, 2008
```
git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2999 d4fa192b-c00b-0410-8231-f00ffab90ce4
```
251678f5

Don't make assumptions about pthread_t · e0731052

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2998 d4fa192b-c00b-0410-8231-f00ffab90ce4

e0731052

use strtod() instead of strtof() · 4eb4937d

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2997 d4fa192b-c00b-0410-8231-f00ffab90ce4

4eb4937d

don't make assumptions about pthread_t · db8bf1af

Poul-Henning Kamp authored Jul 23, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2996 d4fa192b-c00b-0410-8231-f00ffab90ce4

db8bf1af

ACL code rewrite: · 97178f9b

Poul-Henning Kamp authored Jul 23, 2008

	Add IPv6 support (untested!)

	Implement evil-acls

IPv6 support
------------

I have implemented IPv6 filtering support, but I have done so blindly
as I have no IPv6 networks to test with.

Please double check before relying on this to work, and please report
your findings back to us.

Syntax
------

The ACL rules still have the same syntax, but the sematics have
expanded to handle IPv6 also:

    acl foo {

        "foohost";              // Match, if the address is one of the 
                                // ipv4 or ipv6 addresses of "foohost"

        ! "foohost";            // Fail, if...

        "192.168.1.7" / 24;     // Use mask for comparison:  The '7' is
                                // ignored

				// Implicit masks:
	!"172.16";		// Fail 172.16.0.0 to 172.16.255.255
	"10.0.0";		// Match 10.0.0.0 to 10.0.0.255
	

        "www.freebsd.org" / 24; // This will give compile error, because
                                // the "www.freebsd.org" has both ipv4 
                                // and ipv6 addresses, and using the same
                                // mask for both kinds do not make sense.

        ( ... );                // Ignore this rule if DNS lookup fails.

        ( ! "idiot.net" );      // If we can resolve "idiot.net", then
                                // return Failure to match, if we see them.
    }

Please notice that DNS lookup happens *only* on VCL compilation, if a
DNS record changes you need to recompile (ie: vcl.load or vcl.inline)
your VCL code again, it is not enough to just switch vcl (vcl.use).

(This is the same as with backend DNS lookups)

Evil-acls
---------

Most firewall or ip-filtering facilities, compile the lists of
networks and masks to a table, and matches proceed sequentially
through that table until the table is exhausted or a match is found.

Since we compile our ACLs into C-code, we might as well implement the
"evil-acl" concept, and compile the rules directly into C-code instead.

An ACL like this:

	acl foo {
		"172.16";
		!"172.16.17";
		"172.16.17.18";
		"172.16"/18;
	}

Compiles to:

	if (fam == 2) {
	 if (a[0] == 172) {
          if (a[1] == 16) {
           if (a[2] == 17) {
            if (a[3] == 18) {
             VRT_acl_log(sp, "MATCH bar " "172.16.17.18");
             return (1);
            }
            VRT_acl_log(sp, "NEG_MATCH bar " "172.16.17");
            return (0);
           }
           else if ((a[3] & 0xc0) == 0) {
            VRT_acl_log(sp, "MATCH bar " "172.16" "/18" );
            return (1);
           }
           VRT_acl_log(sp, "MATCH bar " "172.16");
           return (1);
          }
         }
        }
	VRT_acl_log(sp, "NO_MATCH bar");
	return (0);

As can be seen, for example the comparison with "172" is now shared
for all four rules in the ACL, instead of being carried out once for
each of the four rules.

In addition to this optimization, the C-compiler will of course use
its usual (and unusual) tricks to speed things up, quite likely inlining
the ACL code in the VCL functions where they are referenced.

It will also be noticed, that the compiler sorts the rules in "most
specific order".

This means that:
		"172.16.17.18";
gets tested before
		!"172.16.17";
even though it is listed later in the ACL.

Previously we tested the rules in the order given.



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2995 d4fa192b-c00b-0410-8231-f00ffab90ce4

97178f9b

Updated the default.vcl and man page with the latest config from mgt_vcc.c. · dde15a6d

Petter Knudsen authored Jul 23, 2008

The following perl one-liner is useful:
perl -ne 'if( $print == 1 ) { exit if( /^$/ ); eval "print " . $_; } $print = 1 if /default_vcl/;' mgt_vcc.c
Fixes #135 (for now)

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2994 d4fa192b-c00b-0410-8231-f00ffab90ce4

dde15a6d

22 Jul, 2008 1 commit

Add a counter for shmlog wraparounds. · 6001fdc2

Poul-Henning Kamp authored Jul 22, 2008



git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@2993 d4fa192b-c00b-0410-8231-f00ffab90ce4

6001fdc2