Fix up poor results of SVN's merge tracking

git-svn-id: http://www.varnish-cache.org/svn/branches/2.1@5251 d4fa192b-c00b-0410-8231-f00ffab90ce4

doc/sphinx/faq/configuration.rst

+%%%%%%%%%%%%%%%
+Configuration
+%%%%%%%%%%%%%%%
+
+.. _faq-vcl:
+
+VCL
+===
+
+**What is VCL?**
+
+VCL is an acronym for Varnish Configuration Language.  In a VCL file, you configure how Varnish should behave.  Sample VCL files will be included in this Wiki at a later stage.
+
+**Where is the documentation on VCL?**
+
+We are working on documenting VCL. The `WIKI <http://varnish-cache.org/wiki/VCLExamples>`_ contains some examples.
+
+Please also see ``man 7 vcl``.
+
+
+**How do I load VCL file while Varnish is running?**
+
+* Place the VCL file on the server
+* Telnet into the managment port.
+* do a "vcl.load <configname> <filename>" in managment interface. <configname> is whatever you would like to call your new configuration.
+* do a "vcl.use <configname>" to start using your new config.
+
+**Should I use ''pipe'' or ''pass'' in my VCL code? What is the difference?**
+
+When varnish does a ``pass`` it acts like a normal HTTP proxy. It
+reads the request and pushes it onto the backend. The next HTTP
+request can then be handled like any other.
+
+``pipe`` is only used when Varnish for some reason can't handle the
+``pass``. ``pipe`` reads the request, pushes in onty the backend
+_only_ pushes bytes back and forth, with no other actions taken.
+
+Since most HTTP clients do pipeline several requests into one
+connection this might give you an undesirable result - as every
+subsequent request will reuse the existing ``pipe``.
+
+Varnish versions prior to 2.0 does not support handling a request body
+with ``pass`` mode, so in those releases ``pipe`` is required for
+correct handling.
+
+In 2.0 and later, ``pass`` will handle the request body correctly.
+
+If you get 503 errors when making a request which is ``pass`` ed, make sure
+that you're specifying the backend before returning from vcl_recv with ``pass``.
+
+
+

doc/sphinx/faq/http.rst

+%%%%%%%%%%%
+HTTP
+%%%%%%%%%%%
+
+**What is the purpose of the X-Varnish HTTP header?** 
+
+The X-Varnish HTTP header allows you to find the correct log-entries for the transaction. For a cache hit, X-Varnish will contain both the ID of the current request and the ID of the request that populated the cache. It makes debugging Varnish a lot easier.
+
+**Does Varnish support compression?**
+
+This is a simple question with a complicated answer; see `WIKI <http://varnish-cache.org/wiki/FAQ/Compression>`_.
+
+**How do I add a HTTP header?**
+
+To add a HTTP header, unless you want to add something about the client/request, it is best done in vcl_fetch as this means it will only be processed every time the object is fetched::
+
+        sub vcl_fetch {
+          # Add a unique header containing the cache servers IP address:
+          remove obj.http.X-Varnish-IP;
+          set    obj.http.X-Varnish-IP = server.ip;
+          # Another header:
+          set    obj.http.Foo = "bar";
+        }
+
+**How can I log the client IP address on the backend?**
+
+All I see is the IP address of the varnish server.  How can I log the client IP address?
+
+We will need to add the IP address to a header used for the backend request, and configure the backend to log the content of this header instead of the address of the connecting client (which is the varnish server).
+
+Varnish configuration::
+
+        sub vcl_recv {
+          # Add a unique header containing the client address
+          remove req.http.X-Forwarded-For;
+          set    req.http.X-Forwarded-For = client.ip;
+          # [...]
+        }
+
+For the apache configuration, we copy the "combined" log format to a new one we call "varnishcombined", for instance, and change the client IP field to use the content of the variable we set in the varnish configuration::
+
+        LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" varnishcombined
+
+And so, in our virtualhost, you need to specify this format instead of "combined" (or "common", or whatever else you use)::
+        
+	<VirtualHost *:80>
+          ServerName www.example.com
+          # [...]
+          CustomLog /var/log/apache2/www.example.com/access.log varnishcombined
+          # [...]
+        </VirtualHost>
+
+The [http://www.openinfo.co.uk/apache/index.html mod_extract_forwarded Apache module] might also be useful.
+
+

doc/sphinx/faq/logging.rst

+%%%%%%%%%%%%%%%%%%%
+Logging
+%%%%%%%%%%%%%%%%%%%
+
+**Where can I find the log files?**
+
+Varnish does not log to a file, but to shared memory log. Use the varnishlog utility to print the shared memory log or varnishncsa to present it in the Apache / NCSA "combined" log format.
+

doc/sphinx/index.rst

@@ -6,6 +6,12 @@
 Welcome to Varnish's documentation!
 ===================================
 
+Arnold's Laws of Documentation:
+        (1) If it should exist, it doesn't.
+        (2) If it does exist, it's out of date.
+        (3) Only documentation for useless programs transcends the
+            first two laws.
+
 Contents:
 
 .. toctree::
@@ -14,6 +20,9 @@ Contents:
    installation/index.rst
    tutorial/index.rst
    reference/index.rst
+   faq/index.rst
+   phk/index.rst
+   glossary/index.rst
 
 Indices and tables
 ==================
@@ -21,4 +30,3 @@ Indices and tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
-

doc/sphinx/installation/bugs.rst

+%%%%%%%%%%%%%%
+Reporting bugs
+%%%%%%%%%%%%%%
+
+Varnish can be a tricky beast to debug, having potentially thousands
+of threads crowding into a few data structures makes for *interesting*
+core dumps.
+
+Actually, let me rephrase that without irony:  You tire of the "no,
+not thread 438 either, lets look at 439 then..." routine really fast.
+
+So if you run into a bug, it is important that you spend a little bit
+of time collecting the right information, to help us fix the bug.
+
+The most valuable information you can give us, is **always** how
+to trigger and reproduce the problem.  If you can tell us that, we
+rarely need anything else to solve it.  The caveat being, that we
+do not have a way to simulate high levels of real-life web-traffic,
+so telling us to "have 10.000 clients hit at once" does not really
+allow us to reproduce.
+
+Roughly we have three clases of bugs with Varnish, and the information
+we need to debug them depends on the kind of bug.
+
+Varnish crashes
+===============
+
+Plain and simple: **boom**
+
+Varnish is split over two processes, the manager and the child.  The child
+does all the work, and the manager hangs around to resurect it, if it
+crashes.
+
+Therefore, the first thing to do if you see a varnish crash, is to examine
+your syslogs, to see if it has happened before.  (One site is rumoured
+to have had varnish restarting every 10 minutes and *still* provide better
+service than their CMS system.)
+
+When it crashes, if at all possible, Varnish will spew out a crash dump
+that looks something like::
+
+	Child (32619) died signal=6 (core dumped)
+	Child (32619) Panic message: Assert error in ccf_panic(), cache_cli.c line 153:
+	  Condition(!strcmp("", "You asked for it")) not true.
+	errno = 9 (Bad file descriptor)
+	thread = (cache-main)
+	ident = FreeBSD,9.0-CURRENT,amd64,-sfile,-hcritbit,kqueue
+	Backtrace:
+	  0x42bce1: pan_ic+171
+	  0x4196af: ccf_panic+4f
+	  0x8006b3ef2: _end+80013339a
+	  0x8006b4307: _end+8001337af
+	  0x8006b8b76: _end+80013801e
+	  0x8006b8d84: _end+80013822c
+	  0x8006b51c1: _end+800134669
+	  0x4193f6: CLI_Run+86
+	  0x429f8b: child_main+14b
+	  0x43ef68: start_child+3f8
+	[...]
+
+If you can get that information to us, we are usually able to
+see exactly where things went haywire, and that speeds up bugfixing
+a lot.
+
+There will be a lot more information than this, and before sending
+it all to us, you should obscure any sensitive/secret
+data/cookies/passwords/ip# etc.  Please make sure to keep context
+when you do so, ie: do not change all the IP# to "X.X.X.X", but
+change each IP# to something unique, otherwise we are likely to be
+more confused than informed.
+
+The most important line is the "Panic Message", which comes in two
+general forms:
+
+"Missing errorhandling code in ..."
+	This is a place where we can conceive ending up, and have not
+	(yet) written the padded-box error handling code for.
+
+	The most likely cause here, is that you need a larger workspace
+	for HTTP headers and Cookies. (XXX: which params to tweak)
+
+	Please try that before reporting a bug.
+
+"Assert error in ..."
+	This is something bad that should never happen, and a bug
+	report is almost certainly in order.  As always, if in doubt
+	ask us on IRC before opening the ticket.
+
+In your syslog it may all be joined into one single line, but if you
+can reproduce the crash, do so while running varnishd manually:
+
+	``varnishd -d <your other arguments> |& tee /tmp/_catch_bug``
+
+That will get you the entire panic message into a file.
+
+(Remember to type ``start`` to launch the worker process, that is not
+automatic when ``-d`` is used.)
+
+Varnish goes on vacation
+========================
+
+This kind of bug is nasty to debug, because usually people tend to
+kill the process and send us an email saying "Varnish hung, I
+restarted it" which gives us only about 1.01 bit of usable debug
+information to work with.
+
+What we need here is all the information can you squeeze out of
+your operating system **before** you kill the Varnish process.
+
+One of the most valuable bits of information, is if all Varnish'
+threads are waiting for something or if one of them is spinning
+furiously on some futile condition.
+
+Commands like ``top -H`` or ``ps -Haxlw`` or ``ps -efH`` should be
+able to figure that out.
+
+If one or more threads are spinning, use ``strace`` or ``ktrace`` or ``truss``
+(or whatever else your OS provides) to get a trace of which system calls
+the varnish process issues.  Be aware that this may generate a lot
+of very repetitive data, usually one second worth is more than enough.
+
+Also, run ``varnishlog`` for a second, and collect the output
+for us, and if ``varnishstat`` shows any activity, capture that also.
+
+When you have done this, kill the Varnish *child* process, and let
+the *master* process restart it.  Remember to tell us if that does
+or does not work.  If it does not, kill all Varnish processes, and
+start from scratch.  If that does not work either, tell us, that
+means that we have wedged your kernel.
+
+
+Varnish does something wrong
+============================
+
+These are the easy bugs: usually all we need from you is the relevant
+transactions recorded with ``varnishlog`` and your explanation of
+what is wrong about what Varnish does.
+
+Be aware, that often Varnish does exactly what you asked it to, rather
+than what you intended it to do, so it sounds like a bug that would
+have tripped up everybody else, take a moment to read though your
+VCL and see if it really does what you think.
+
+You can also try setting the ``vcl_trace`` parameter, that will
+generate log records with like and char number for each statement
+executed in your VCL program.
+

doc/sphinx/installation/help.rst

+%%%%%%%%%%%%%%%%%%
+Getting hold of us
+%%%%%%%%%%%%%%%%%%
+
+Getting hold of the gang behind Varnish is pretty straight forward,
+we try to help as much as time permits and have tried to streamline
+this process as much as possible.
+
+But before you grab hold of us, spend a moment composing your thoughts and
+formulate your question, there is nothing as pointless as simply telling
+us "Varnish does not work for me" with no further information to give
+any clue to why.
+
+And before you even do that, do a couple of searches to see if your
+question is already answered, if it has been, you will get your answer
+much faster that way.
+
+IRC Channel
+===========
+
+The most immediate way to get hold of us, is to join our IRC channel:
+
+	``#varnish on server irc.linpro.no``
+
+The main timezone of the channel is Europe+America.
+
+If you can explain your problem in a few clear sentences, without too
+much copy&paste, IRC is a good way to try to get help. If you do need
+to paste log files, VCL and so on, please use a pastebin_.
+
+If the channel is all quiet, try again some time later, we do have lives,
+families and jobs to deal with also.
+
+You are more than welcome to just hang out, and while we don't mind
+the occational intrusion of the real world into the flow, keep
+it mostly on topic, and dont paste random links unless they are
+*really* spectacular and intelligent.
+
+Mailing Lists
+=============
+
+Getting on or off our mailinglist happens through MailMan_.
+
+If you are going to use Varnish, subscribing to our ``varnish-announce``
+mailing list is probably a very good idea.  The typical pattern is that
+people spend some time getting Varnish running, and then more or less
+forget about it.   Therefore the announce list is a good way to be
+reminded about new releases, bad bugs or security holes.
+
+The ``varnish-misc`` mailing list is for general banter, questions,
+suggestions, ideas and so on.  If you are new to Varnish it may pay
+off to subscribe to -misc, simply to have an ear to the telegraph-pole
+and learn some smart tricks.  This is a good place to ask for help
+with more complex issues, that require quoting of files and long
+explanations.
+
+Make sure to pick a good subject line, and if the subject of the
+thread changes, please change the subject to match, some of us deal
+with hundreds of emails per day, after spam-filters, and we need all
+the help we can get to pick the interesting ones.
+
+The ``varnish-dev`` mailing list is used by the developers and is
+usually quite focused on source-code and such.  Everybody on
+the -dev list is also on -misc, so cross-posting only serves to annoy
+those people.
+
+Trouble Tickets
+===============
+
+Please do not open a trouble ticket, unless you have spotted an actual
+bug in Varnish.  Ask on IRC first if you are in doubt.
+
+The reason for this policy, is to avoid the bugs being drowned in a
+pile of sensible suggestions for future enhancements and call for help
+from people who forget to check back if they get it and so on.
+
+We track suggestions and ideas in our `"Shopping-List" wiki page`_, and user
+support via email and IRC.
+
+Commercial Support
+==================
+
+The following companies offer commercial Varnish support, and are listed
+here for your convenience.  If you want your company listed here, drop
+an email to phk@FreeBSD.org.
+
+	Varnish Software
+	sales@varnish-software.com
+
+
+.. _Mailman: http://lists.varnish-cache.org/mailman/listinfo
+.. _pastebin: http://gist.github.com/
+.. _"Shopping-List" wiki page: http://varnish-cache.org/wiki/PostTwoShoppingList

doc/sphinx/installation/index.rst

+.. _Installation:
+
 %%%%%%%%%%%%%%%%%%%%
 Varnish Installation
 %%%%%%%%%%%%%%%%%%%%
 
+This manual explains how to get Varnish onto your system, where
+to get help, how report bugs etc.  In other words, it is a manual
+about pretty much everything else than actually using Varnish to
+move traffic.
+
+.. toctree::
+
+	install.rst
+	help.rst
+	bugs.rst
+
 .. todo::
-        on this os, pull this package
-        .. that ..//..
-        to compile from source
-        how to get help
-        - mailing list
-        - IRC
-        - varnish-software.com
-        - other listed consultants
+        [V] on this os, pull this package
+        [V] .. that ..//..
+        [V] to compile from source
+        [V] how to get help
+        [V]- mailing list
+        [V] - IRC
+        [V] - varnish-software.com
+        [V] - other listed consultants
         reporting bugs
         - using varnishtest to reproduce
         - what data do we need

doc/sphinx/installation/install.rst

+.. _install-doc:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Installing Varnish on your computer
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+With open source software, you can choose to install binary
+packages or compile stuff from source-code. 
+
+In general, from a point of principle, I would argue that
+everybody should compile from source, but realistically
+binary packages are *so much easier* so lets cover that first:
+
+
+Installing Varnish from packages
+================================
+
+Installing Varnish on most relevant operating systems can usually 
+be done with with the systems package manager, typical examples
+being:
+
+**FreeBSD**
+
+	FreeBSD (from source)
+		``cd /usr/ports/varnish && make install clean``
+	FreeBSD (binary package)
+		``pkg_add -r varnish``
+
+**CentOS/RedHat 5.4 - RPM files**
+
+	We try to keep the lastest version available as prebuildt RPMs (el4 & el5) on `SourceForge <http://sourceforge.net/projects/varnish/files/>`_.
+
+	Varnish is included in the `EPEL <http://fedoraproject.org/wiki/EPEL>`_ repository. **BUT** unfortunatly we had a syntax change in Varnish 2.0.6->2.1.X. This means that we can not update Varnish in `EPEL <http://fedoraproject.org/wiki/EPEL>`_ so the latest version there is Varnish 2.0.6. In the future (EPEL6) we should be available with Varnish 2.1.X or higher.
+
+**Debian/Ubuntu - DEB files**
+
+	Varnish is distributed to the *unstable* repository of Debian. You should be able to get a hold of the lastest version there.
+	
+	Ubuntu syncronize the *unstable* Debian repository. See `Ubuntu Packages <http://packages.ubuntu.com/>`_.
+
+**Other systems**
+
+	You are probably best of compiling your own code. See `Compiling Varnish from source`_.
+
+If that worked for you, you can skip the rest of this document
+for now, and and start reading the much more interesting :ref:`Tutorial`
+instead.
+
+
+Compiling Varnish from source
+=============================
+
+If there are no binary packages available for your system, or if you
+want to compile Varnish from source for other reasons, follow these
+steps:
+
+First get a copy of the sourcecode using the ``svn`` command.  If
+you do not have this command, you need to install SubVersion_ on
+your system.  There is usually a binary package, try substituting
+"subversion" for "varnish" in the examples above, it might just work.
+
+To get the development source code::
+
+	svn co http://varnish-cache.org/svn/varnish/trunk
+
+or if you want the production branch::
+
+	svn co http://varnish-cache.org/svn/varnish/branches/2.1
+
+Next, configuration:  For this you will need ``libtoolize``, ``aclocal``,
+``autoheader``, ``automake`` and ``autoconf``, also known as *the
+autocrap tools* installed on your system.
+
+Once you have them::
+
+	cd varnish-cache
+	sh autogen.sh
+	sh configure
+	make
+
+The ``configure`` script takes some arguments, but more likely than
+not, you can forget about that for now, almost everything in Varnish
+are runtime parameters.
+
+Before you install, you may want to run the regression tests, make
+a cup of tea while it runs, it takes some minutes::
+
+	(cd bin/varnishtest && ./varnishtest tests/*.vtc)
+
+Don't worry of a single or two tests fail, some of the tests are a
+bit too timing sensitive (Please tell us which so we can fix it) but
+if a lot of them fails, and in particular if the ``b00000.vtc`` test 
+fails, something is horribly wrong, and you will get nowhere without
+figuring out what.
+
+And finally, the true test of a brave heart::
+
+	make install
+
+.. _SubVersion: http://subversion.tigris.org/

doc/sphinx/tutorial/index.rst

+.. _Tutorial:
+
 %%%%%%%%%%%%%%%%
 Varnish Tutorial
 %%%%%%%%%%%%%%%%
 
+Welcome to the Varnish Tutorial, we hope this will help you get to 
+know and understand Varnish.
+
+.. toctree::
+
+	intro.rst
+	tut001.rst
+	tut002.rst
+
 .. todo::
         starting varnish with -d, seeing a transaction go through
         explain varnishlog output for a miss and a hit

doc/sphinx/tutorial/vtc_files/C002.vtc

+# $Id$
+
+client c1 {
+	txreq
+	rxresp
+} -run

doc/sphinx/tutorial/vtc_files/S002.vtc

+# $Id$
+
+server s1 {
+	rxreq
+	txresp -hdr "Foo: bar" -body "Hello World!"
+} -start -wait