cleaned up rst. added more text.

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4793 d4fa192b-c00b-0410-8231-f00ffab90ce4

doc/sphinx/tutorial/advanced_backend_servers.rst

 Advanced Backend configuration
-----------------
+==============================
 
 At some point you might need Varnish to cache content from several
 servers. You might want Varnish to map all the URL into one single
@@ -10,30 +10,29 @@ site. Lets say our Java application should handle URL beginning with
 /java/.
 
 We manage to get the thing up and running on port 8000. Now, lets have
-a look a default.vcl.
+a look a default.vcl.::
 
-backend default {
-    .host = "127.0.0.1";
-    .port = "8080";
-}
+  backend default {
+      .host = "127.0.0.1";
+      .port = "8080";
+  }
 
-We add a new backend.
+We add a new backend.::
 
-backend java {
-    .host = "127.0.0.1";
-    .port = "8000";
-}
+  backend java {
+      .host = "127.0.0.1";
+      .port = "8000";
+  }
 
-Now we need tell where to send the difference URL. Lets look at vcl_recv:
+Now we need tell where to send the difference URL. Lets look at vcl_recv.::
 
-
-sub vcl_recv {
-    if (req.url ~ "^/java/") {
-        set req.backend = java;
-    } else {
-        set req.backend = default.
-    }
-}
+  sub vcl_recv {
+      if (req.url ~ "^/java/") {
+          set req.backend = java;
+      } else {
+          set req.backend = default.
+      }
+  }
 
 It's quite simple, really. Lets stop and think about this for a
 moment. As you can see you can define how you choose backends based on
@@ -41,8 +40,8 @@ really arbitrary data. You want to send mobile devices to a different
 backend? No problem. if (req.User-agent ~ /mobile/) .... should do the
 trick. 
 
-Directors 
-~~~~~~~~~~
+Directors
+---------
 
 You can also group several backend into a group of backends. These
 groups are called directors. This will give you increased performance
@@ -52,13 +51,12 @@ together in a director.::
 	 backend server1 {
 	     .host = "192.168.0.10";
 	 }
-::
 	 backend server2{
 	     .host = "192.168.0.10";
 	 }
 
-
 Now we create the director.::
+
        	director example_director round-robin {
         {
                 .backend = server1;
@@ -81,7 +79,7 @@ requests to the healthy server? Sure it can. This is where the Health
 Checks come into play.
 
 Health checks
-~~~~~~~~~~~~
+-------------
 
 Lets set up a director with two backends and health checks. First lets
 define the backends.::
@@ -96,8 +94,7 @@ define the backends.::
                 .threshold = 3;
 	   }
          }
-::
-	backend server2 {
+       backend server2 {
   	  .host = "server2.example.com";
   	  .probe = {
                 .url = "/";
@@ -131,7 +128,7 @@ XXX: Ref to reference guide.
 
 Now we define the director.::
 
-director example_director round-robin {
+  director example_director round-robin {
         {
                 .backend = server1;
         }
@@ -145,4 +142,3 @@ director example_director round-robin {
 You use this director just as you would use any other director or
 backend. Varnish will not send traffic to hosts that are marked as
 unhealty.
-

doc/sphinx/tutorial/increasing_your_hitrate.rst

+.. _tutorial-increasing_your_hitrate:
+
 Achieving a high hitrate
 ========================
 
 Now that Varnish is up and running, and you can access your web
-application through Varnish we probably need to do some changes to
-either the configuration or the application so you'll get a high
-hitrate in varnish.
+application through Varnish. Unless your application is specifically
+written to work behind a web accelerator you'll probably need to do
+some changes to either the configuration or the application in order
+to get a high hitrate in Varnish.
+
+Note that you need a tool to see what HTTP headers fly between you and
+the web server. If you have Varnish the easiest might be to use
+varnishlog, but sometimes a separate tool makes sense. Here are the
+ones I use.
+
+lwp-request
+~~~~~~~~~~~
+
+lwp-request is part of The World-Wide Web library for Perl. It's
+couple of really basic programs that can execute a HTTP request and
+give you the result. I use two programs, GET and HEAD.
+
+vg.no was the first site to use Varnish and the people running Varnish
+there are quite clueful. So its interesting to look at their HTTP
+Headers. Lets send a GET requst for their home page.::
+
+  $ GET -H 'Host: vg.no' -Used http://vg.no/ 
+  GET http://www.vg.no/
+  User-Agent: Rickzilla 1.0
+  
+  200 OK
+  Cache-Control: must-revalidate
+  Refresh: 600
+  Title: VG Nett - Forsiden - VG Nett
+  X-Age: 463
+  X-Cache: HIT
+  X-Rick-Would-Never: Let you down
+  X-VG-Jobb: http://www.finn.no/finn/job/fulltime/result?keyword=vg+multimedia Merk:HeaderNinja
+  X-VG-Korken: http://www.youtube.com/watch?v=Fcj8CnD5188
+  X-VG-WebCache: joanie
+  X-VG-WebServer: leon
+
+OK. Let me explain what it does. GET usually send off HTTP 0.9
+requests, which lack the Host header. So I add a Host header with the
+-H option. -U print request headers, -s prints response status -e
+prints repsonse headers and -d discards the actual content. We dont
+really care about the content, only the headers.
+
+As you can see VG ads quite a bit of information in their headers. A
+lot of this information is added in their VCL.
+
+So, to check whether a site sets cookies for a specific URL just do
+``GET -Used http://example.com/ |grep Set-Cookie``
+
+Firefox plugins
+~~~~~~~~~~~~~~~
+
+There are also a couple of great plugins for Firefox. Both *Live HTTP
+Headers* and *Firebug* can show you what headers are beeing sent and
+recieved.
 
 HTTP Headers
 ------------
 
+Varnish considers itself part of the actual webserver, since its under
+your control. The role of *surrogate origin cache* is not really well
+defined by the IETF so RFC 2616 doesn't always tell us what we should
+do
+
 Cache-control
 ~~~~~~~~~~~~~
 
+The Cache-Control instructs caches how to handle the content. Varnish
+cares about the *max-age* parameter and uses it to calculate the TTL
+for an object. 
+
+"Cache-Control: nocache" is ignored. See
+:ref:`tutorial-increasing_your_hitrate-pragma:` for an example on how
+to implement support.
+
 Cookies
 ~~~~~~~
 
+Varnish will not cache a object comming from the backend with a
+Set-Cookie header present. Also, if the client sends a Cookie header,
+Varnish will bypass the cache and go directly to the backend.
+
+This can be overly conservative. A lot of sites use Google Analytics
+(GA) to analyse their traffic. GA sets a cookie to track you. This
+cookie is used by the client side java script and is therefore of no
+interest to the server. 
+
+For a lot of web application it makes sense to completly disregard the
+cookies unless you are accessing a special part of the web site. This
+VCL snipplet in vcl_recv will disregard cookies unless you are
+accessing /admin/.::
+
+  if ( !( req.url ~ ^/admin/) ) {
+    unset http.Cookie;
+  }
+
+Quite simple. If, however, you need to do something more complicated,
+like removing one out of several cookies, things get
+difficult. Unfornunatly Varnish doesn't have good tools for
+manipulating the Cookies. We have to use regular expressions to do the
+work. Let me show you an example where we remove everything the the cookies named COOKIE1 and COOKIE2  and you can marvel at it.::
+
+  sub vcl_recv {
+  if (req.http.Cookie) {
+      set req.http.Cookie = ";" req.http.Cookie;
+      set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
+      set req.http.Cookie = regsuball(req.http.Cookie, ";(COOKIE1|COOKIE2)=", "; \1=");
+      set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
+      set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
+
+      if (req.http.Cookie == "") {
+          remove req.http.Cookie;
+      }
+  }
+
+The example is taken from the Varnish Wiki, where you can find other
+scary examples of what can be done i VCL.
+
 Vary
 ~~~~
 
+The Vary header is sent by the web server to indicate what makes a
+HTTP object Vary.
+
+.. _tutorial-increasing_your_hitrate-pragma:
+
+Pragma
+~~~~~~
+
+
+HTTP 1.0 server might send "Pragma: nocache". Varnish ignores this
+header. You could easly add support for this header in VCL.
+
+In vcl_fetch::
+
+  if (beresp.http.Pragma ~ "nocache") {
+     pass;
+  }
+
 Authentication
 ~~~~~~~~~~~~~~
 
@@ -25,6 +150,7 @@ Normalizing your namespace
 --------------------------
 
 .. _tutorial-increasing_your_hitrate-purging:
+
 Purging
 -------
 
@@ -34,3 +160,6 @@ HTTP Purges
 
 Bans
 ~~~~
+
+
+

doc/sphinx/tutorial/logging.rst

@@ -61,7 +61,8 @@ want to know are:
 
 -i tag
  Only show lines with a certain tag. "varnishlog -i SessionOpen" 
- will only give you new sessions.
+ will only give you new sessions. Note that the tags are case 
+ insensitive. 
 
 -I Regex
  Filter the data through a regex and only show the matching lines. To

doc/sphinx/tutorial/starting_varnish.rst

 .. _tutorial-starting_varnish:
+
 Starting Varnish
 ----------------
 
@@ -38,5 +39,12 @@ Now you have Varnish running. Let us make sure that it works
 properly. Use your browser to go to http://192.168.2.2:8080/ - you
 should now see your web application running there.
 
+Whether or not the application actually goes faster when run through
+Varnish depends on a few factors. If you application uses cookies for
+every session (a lot of PHP and Java applications seem to send a
+session cookie if it is needed or not) or if it uses authentication
+chances are Varnish won't do much caching. Ignore that for the moment,
+we come back to that in :ref:`tutorial-increasing_your_hitrate`.
+
 Lets make sure that Varnish really does do something to your web
 site. To do that we'll take a look at the logs.

doc/sphinx/tutorial/statistics.rst

@@ -17,7 +17,11 @@ With suitable filtering using the -I, -i, -X and -x options, it can be
 used to display a ranking of requested documents, clients, user
 agents, or any other information which is recorded in the log.
 
-XXX Show some nice examples here.
+``varnishtop -i rxurl`` will show you what URLs are beeing asked for
+by the client. ``varnishtop -i txurl`` will show you what your backend
+is beeing asked the most. ``varnishtop -i RxHeader -I
+Accept-Encoding`` will show the most popular Accept-Encoding header
+the client are sendning you.
 
 varnishhist
 ===========
@@ -51,4 +55,3 @@ graphs of these counters. One such program is Munin. Munin can be
 found at http://munin-monitoring.org/ . There is a plugin for munin in
 the varnish source code.
 
-

doc/sphinx/tutorial/troubleshooting.rst

@@ -3,7 +3,7 @@ Troubleshooting Varnish
 
 
 When Varnish won't start
------------------------
+------------------------
 
 Sometimes Varnish wont start. There is a pletphora of reasons why
 Varnish wont start on your machine. We've seen everything from wrong

doc/sphinx/tutorial/vcl.rst

 Varnish Configuration Language - VCL
-==============================
+====================================
 
 How ordinary configuration files work
----------------------------------
+-------------------------------------
 
 Varnish has a really neat configuration system. Most other systems use
 configuration directives, where you basically turn on and off a bunch