another bunch of man pages. Some have rendering issues that need I need to work out

git-svn-id: http://www.varnish-cache.org/svn/trunk/varnish-cache@4888 d4fa192b-c00b-0410-8231-f00ffab90ce4

doc/sphinx/installation/install.rst

 .. _install-doc:
 
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-Installing Varnish on your computer
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Installing Varnish
+==================
 
 With open source software, you can choose to install binary packages
 or compile stuff from source-code. To install a package or compile
@@ -11,8 +10,8 @@ method too choose read the whole document and choose the method you
 are most confortable with.
 
 
-Installing Varnish from packages
-================================
+Source or packages?
+~~~~~~~~~~~~~~~~~~~
 
 Installing Varnish on most relevant operating systems can usually 
 be done with with the systems package manager, typical examples

doc/sphinx/reference/index.rst

@@ -5,14 +5,19 @@ The Varnish Reference Manual
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 .. toctree::
-	
-	varnishadm.rst
-	shmem.rst
+
 	vcl.rst
-	varnishstat.rst
-	varnishsizes.rst
-	varnishreplay.rst
+	varnishadm.rst
+	varnishd.rst
+	varnishhist.rst
 	varnishlog.rst
+	varnishncsa.rst
+	varnishreplay.rst
+	varnishsizes.rst
+	varnishstat.rst
+	varnishtest.rst
+	varnishtop.rst
+	shmem.rst
 
 .. todo::
         The programs:

doc/sphinx/reference/varnishd.rst

+=========
+ varnishd
+=========
+
+-----------------------
+HTTP accelerator daemon
+-----------------------
+
+:Author: Dag-Erling Smørgrav
+:Author: Stig Sandbeck Mathisen
+:Author: Per Buer
+:Date:   2010-05-31
+:Version: 1.0
+:Manual section: 1
+
+
+SYNOPSIS
+========
+
+varnishd [-a address[:port]] [-b host[:port]] [-d] [-F] [-f config] 
+	 [-g group] [-h type[,options]] [-i identity]
+	 [-l shmlogsize] [-n name] [-P file] [-p param=value] 
+	 [-s type[,options]] [-T address[:port]] [-t ttl]
+	 [-u user] [-V] [-w min[,max[,timeout]]]
+
+DESCRIPTION
+===========
+
+The varnishd daemon accepts HTTP requests from clients, passes them on to a backend server and caches the
+returned documents to better satisfy future requests for the same document.
+
+OPTIONS
+=======
+
+-a address[:port][,address[:port][...]
+            Listen for client requests on the specified address and port.  The address can be a host
+            name (“localhost”), an IPv4 dotted-quad (“127.0.0.1”), or an IPv6 address enclosed in
+            square brackets (“[::1]”).  If address is not specified, varnishd will listen on all
+            available IPv4 and IPv6 interfaces.  If port is not specified, the default HTTP port as
+            listed in /etc/services is used.  Multiple listening addresses and ports can be speci‐
+            fied as a whitespace- or comma-separated list.
+
+-b host[:port]
+            Use the specified host as backend server.  If port is not specified, 
+	    the default is 8080.
+
+-d          Enables debugging mode: The parent process runs in the foreground with a CLI connection
+            on stdin/stdout, and the child process must be started explicitly with a CLI command.
+            Terminating the parent process will also terminate the child.
+
+-F          Run in the foreground.
+
+-f config   Use the specified VCL configuration file instead of the builtin default.  See vcl(7) for
+            details on VCL syntax.
+
+-g group    Specifies the name of an unprivileged group to which the child process should switch
+            before it starts accepting connections.  This is a shortcut for specifying the group
+            run-time parameter.
+
+-h type[,options]
+            Specifies the hash algorithm.  See Hash Algorithms for a list of supported algorithms.
+
+-i identity
+            Specify the identity of the varnish server.  This can be accessed using server.identity
+            from VCL
+
+-l shmlogsize
+            Specify size of shmlog file.  Scaling suffixes like 'k', 'm' can be used up to
+            (e)tabytes.  Default is 80 Megabytes.  Specifying less than 8 Megabytes is unwise.
+
+-n name     Specify a name for this instance.  Amonst other things, this name is used to construct
+            the name of the directory in which varnishd keeps temporary files and persistent state.
+            If the specified name begins with a forward slash, it is interpreted as the absolute
+            path to the directory which should be used for this purpose.
+
+-P file     Write the process's PID to the specified file.
+
+-p param=value
+            Set the parameter specified by param to the specified value.  See Run-Time 
+	    Parameters for a list of parameters. This option can be used multiple 
+	    times to specify multiple parameters.
+
+-S file     Path to a file containing a secret used for authorizing access to the management port.
+
+-s type[,options]
+            Use the specified storage backend.  See Storage Types for a list of supported storage
+            types.  This option can be used multiple times to specify multiple storage files.
+
+-T address[:port]
+            Offer a management interface on the specified address and port.  See Management
+            Interface for a list of management commands.
+
+-t ttl      Specifies a hard minimum time to live for cached documents.  This is a shortcut for
+            specifying the default_ttl run-time parameter.
+
+-u user     Specifies the name of an unprivileged user to which the child process should switch
+            before it starts accepting connections.  This is a shortcut for specifying the user run-
+            time parameter.
+	    
+            If specifying both a user and a group, the user should be specified first.
+
+-V          Display the version number and exit.
+
+-w min[,max[,timeout]]
+            Start at least min but no more than max worker threads with the specified idle timeout.
+            This is a shortcut for specifying the thread_pool_min, thread_pool_max and
+            thread_pool_timeout run-time parameters.
+
+            If only one number is specified, thread_pool_min and thread_pool_max are both set to
+            this number, and thread_pool_timeout has no effect.
+
+
+
+
+
+Hash Algorithms
+---------------
+
+The following hash algorithms are available:
+
+simple_list
+  A simple doubly-linked list.  Not recommended for production use.
+
+classic[,buckets]
+  A standard hash table.  This is the default.  The hash key is the
+  CRC32 of the object's URL modulo the size of the hash table.  Each
+  table entry points to a list of elements which share the same hash
+  key. The buckets parameter specifies the number of entries in the
+  hash table.  The default is 16383.
+
+critbit XXX
+  Very nice.
+
+Storage Types
+-------------
+
+The following storage types are available:
+
+malloc[,size]
+      Storage for each object is allocated with malloc(3).
+
+      The size parameter specifies the maximum amount of memory varnishd will allocate.  The size is assumed to
+      be in bytes, unless followed by one of the following suffixes:
+
+      K, k    The size is expressed in kibibytes.
+
+      M, m    The size is expressed in mebibytes.
+
+      G, g    The size is expressed in gibibytes.
+
+      T, t    The size is expressed in tebibytes.
+
+      The default size is unlimited.
+
+file[,path[,size[,granularity]]]
+      Storage for each object is allocated from an arena backed by a file.  This is the default.
+
+      The path parameter specifies either the path to the backing file or the path to a directory in which
+      varnishd will create the backing file.  The default is /tmp.
+
+      The size parameter specifies the size of the backing file.  The size is assumed to be in bytes, unless fol‐
+      lowed by one of the following suffixes:
+
+      K, k    The size is expressed in kibibytes.
+
+      M, m    The size is expressed in mebibytes.
+
+      G, g    The size is expressed in gibibytes.
+
+      T, t    The size is expressed in tebibytes.
+
+      %       The size is expressed as a percentage of the free space on the file system where it resides.
+
+      The default size is 50%.
+
+      If the backing file already exists, it will be truncated or expanded to the specified size.
+
+      Note that if varnishd has to create or expand the file, it will not pre-allocate the added space, leading
+      to fragmentation, which may adversely impact performance.  Pre-creating the storage file using dd(1) will
+      reduce fragmentation to a minimum.
+
+      The granularity parameter specifies the granularity of allocation.  All allocations are rounded up to this
+      size.  The size is assumed to be in bytes, unless followed by one of the suffixes described for size except
+      for %.
+
+      The default size is the VM page size.  The size should be reduced if you have many small objects.
+
+persistence[XXX]
+      New, shiny, better.
+
+
+Management Interface
+--------------------
+
+If the -T option was specified, varnishd will offer a command-line management interface on the specified address
+and port.  The following commands are available:
+
+help [command]
+      Display a list of available commands.
+
+      If the command is specified, display help for this command.
+
+param.set param value
+      Set the parameter specified by param to the specified value.  See Run-Time Parameters for a list of parame‐
+      ters.
+
+param.show [-l] [param]
+      Display a list if run-time parameters and their values.
+
+      If the -l option is specified, the list includes a brief explanation of each parameter.
+
+      If a param is specified, display only the value and explanation for this parameter.
+
+ping [timestamp]
+      Ping the Varnish cache process, keeping the connection alive.
+
+purge field operator argument [&& field operator argument [...]]
+      Immediately invalidate all documents matching the purge expression.  See Purge expressions for more docu‐
+      mentation and examples.
+
+purge.list
+      Display the purge list.
+
+      All requests for objects from the cache are matched against items on the purge list.  If an object in the
+      cache is older than a matching purge list item, it is considered "purged", and will be fetched from the
+      backend instead.
+
+      When a purge expression is older than all the objects in the cache, it is removed from the list.
+
+purge.url regexp
+      Immediately invalidate all documents whose URL matches the specified regular expression.
+
+quit  Close the connection to the varnish admin port.
+
+start
+      Start the Varnish cache process if it is not already running.
+
+stats
+      Show summary statistics.
+
+      All the numbers presented are totals since server startup; for a better idea of the current situation, use
+      the varnishstat(1) utility.
+
+status
+      Check the status of the Varnish cache process.
+
+stop  Stop the Varnish cache process.
+
+url.purge regexp
+      Deprecated, see purge.url instead.
+
+vcl.discard configname
+      Discard the configuration specified by configname.  This will have no effect if the specified configuration
+      has a non-zero reference count.
+
+vcl.inline configname vcl
+      Create a new configuration named configname with the VCL code specified by vcl, which must be a quoted
+      string.
+
+vcl.list
+      List available configurations and their respective reference counts.  The active configuration is indicated
+      with an asterisk ("*").
+
+vcl.load configname filename
+      Create a new configuration named configname with the contents of the specified file.
+
+vcl.show configname
+      Display the source code for the specified configuration.
+
+vcl.use configname
+      Start using the configuration specified by configname for all new requests.  Existing requests will con‐
+      tinue using whichever configuration was in use when they arrived.
+
+Run-Time Parameters
+-------------------
+
+Runtime parameters are marked with shorthand flags to avoid repeating the same text over and over in the table
+below.  The meaning of the flags are:
+
+experimental
+      We have no solid information about good/bad/optimal values for this parameter.  Feedback with experience
+      and observations are most welcome.
+
+delayed
+      This parameter can be changed on the fly, but will not take effect immediately.
+
+restart
+      The worker process must be stopped and restarted, before this parameter takes effect.
+
+reload
+      The VCL programs must be reloaded for this parameter to take effect.
+
+Here is a list of all parameters, current as of last time we remembered to update the manual page.  This text is
+produced from the same text you will find in the CLI if you use the param.show command, so should there be a new
+parameter which is not listed here, you can find the description using the CLI commands.
+
+Be aware that on 32 bit systems, certain default values, such as sess_workspace (=16k) and thread_pool_stack
+(=64k) are reduced relative to the values listed here, in order to conserve VM space.
+
+acceptor_sleep_decay
+      Default: 0.900
+      Flags: experimental
+
+      If we run out of resources, such as file descriptors or worker threads, the acceptor will sleep between
+      accepts.
+      This parameter (multiplicatively) reduce the sleep duration for each succesfull accept. (ie: 0.9 = reduce
+      by 10%)
+
+acceptor_sleep_incr
+      Units: s
+      Default: 0.001
+      Flags: experimental
+
+      If we run out of resources, such as file descriptors or worker threads, the acceptor will sleep between
+      accepts.
+      This parameter control how much longer we sleep, each time we fail to accept a new connection.
+
+acceptor_sleep_max
+      Units: s
+      Default: 0.050
+      Flags: experimental
+
+      If we run out of resources, such as file descriptors or worker threads, the acceptor will sleep between
+      accepts.
+      This parameter limits how long it can sleep between attempts to accept new connections.
+
+auto_restart
+      Units: bool
+      Default: on
+
+      Restart child process automatically if it dies.
+
+ban_lurker_sleep
+      Units: s
+      Default: 0.0
+
+      How long time does the ban lurker thread sleeps between successfull attempts to push the last item up the
+      purge  list.  It always sleeps a second when nothing can be done.
+      A value of zero disables the ban lurker.
+
+between_bytes_timeout
+      Units: s
+      Default: 60
+
+      Default timeout between bytes when receiving data from backend. We only wait for this many seconds between
+      bytes before giving up. A value of 0 means it will never time out. VCL can override this default value for
+      each backend request and backend request. This parameter does not apply to pipe.
+
+cache_vbe_conns
+      Units: bool
+      Default: off
+      Flags: experimental
+
+      Cache vbe_conn's or rely on malloc, that's the question.
+
+cc_command
+      Default: exec cc -fpic -shared -Wl,-x -o %o %s
+      Flags: must_reload
+
+      Command used for compiling the C source code to a dlopen(3) loadable object.  Any occurrence of %s in the
+      string will be replaced with the source file name, and %o will be replaced with the output file name.
+
+cli_buffer
+      Units: bytes
+      Default: 8192
+
+      Size of buffer for CLI input.
+      You may need to increase this if you have big VCL files and use the vcl.inline CLI command.
+      NB: Must be specified with -p to have effect.
+
+cli_timeout
+      Units: seconds
+      Default: 10
+
+      Timeout for the childs replies to CLI requests from the master.
+
+clock_skew
+      Units: s
+      Default: 10
+
+      How much clockskew we are willing to accept between the backend and our own clock.
+
+connect_timeout
+      Units: s
+      Default: 0.4
+
+      Default connection timeout for backend connections. We only try to connect to the backend for this many
+      seconds before giving up. VCL can override this default value for each backend and backend request.
+
+default_grace
+      Default: 10seconds
+      Flags: delayed
+
+      Default grace period.  We will deliver an object this long after it has expired, provided another thread is
+      attempting to get a new copy.
+
+default_ttl
+      Units: seconds
+      Default: 120
+
+      The TTL assigned to objects if neither the backend nor the VCL code assigns one.
+      Objects already cached will not be affected by changes made until they are fetched from the backend again.
+      To force an immediate effect at the expense of a total flush of the cache use "purge.url ."
+
+diag_bitmap
+      Units: bitmap
+      Default: 0
+      Bitmap controlling diagnostics code::
+
+        0x00000001 - CNT_Session states.
+        0x00000002 - workspace debugging.
+        0x00000004 - kqueue debugging.
+        0x00000008 - mutex logging.
+        0x00000010 - mutex contests.
+        0x00000020 - waiting list.
+        0x00000040 - object workspace.
+        0x00001000 - do not core-dump child process.
+        0x00002000 - only short panic message.
+        0x00004000 - panic to stderr.
+        0x00008000 - panic to abort2().
+        0x00010000 - synchronize shmlog.
+        0x00020000 - synchronous start of persistence.
+        0x80000000 - do edge-detection on digest.
+
+      Use 0x notation and do the bitor in your head :-)
+
+err_ttl
+      Units: seconds
+      Default: 0
+
+      The TTL assigned to the synthesized error pages
+
+esi_syntax
+      Units: bitmap
+      Default: 0
+      Bitmap controlling ESI parsing code::
+
+        0x00000001 - Don't check if it looks like XML
+        0x00000002 - Ignore non-esi elements
+        0x00000004 - Emit parsing debug records
+
+      Use 0x notation and do the bitor in your head :-)
+
+fetch_chunksize
+      Units: kilobytes
+      Default: 128
+      Flags: experimental
+
+      The default chunksize used by fetcher. This should be bigger than the majority of objects with short TTLs.
+      Internal limits in the storage_file module makes increases above 128kb a dubious idea.
+
+first_byte_timeout
+      Units: s
+      Default: 60
+
+      Default timeout for receiving first byte from backend. We only wait for this many seconds for the first
+      byte before giving up. A value of 0 means it will never time out. VCL can override this default value for
+      each backend and backend request. This parameter does not apply to pipe.
+
+group
+      Default: .....
+      Flags: must_restart
+
+      The unprivileged group to run as.
+
+http_headers
+      Units: header lines
+      Default: 64
+
+      Maximum number of HTTP headers we will deal with.
+      This space is preallocated in sessions and workthreads only objects allocate only space for the headers
+      they store.
+
+listen_address
+      Default: :80
+      Flags: must_restart
+
+      Whitespace separated list of network endpoints where Varnish will accept requests.
+      Possible formats: host, host:port, :port
+
+listen_depth
+      Units: connections
+      Default: 1024
+      Flags: must_restart
+
+      Listen queue depth.
+
+log_hashstring
+      Units: bool
+      Default: off
+
+      Log the hash string to shared memory log.
+
+log_local_address
+      Units: bool
+      Default: off
+
+      Log the local address on the TCP connection in the SessionOpen shared memory record.
+
+lru_interval
+      Units: seconds
+      Default: 2
+      Flags: experimental
+
+      Grace period before object moves on LRU list.
+      Objects are only moved to the front of the LRU list if they have not been moved there already inside this
+      timeout period.  This reduces the amount of lock operations necessary for LRU list access.
+
+max_esi_includes
+      Units: includes
+      Default: 5
+
+      Maximum depth of esi:include processing.
+
+max_restarts
+      Units: restarts
+      Default: 4
+
+      Upper limit on how many times a request can restart.
+      Be aware that restarts are likely to cause a hit against the backend, so don't increase thoughtlessly.
+
+overflow_max
+      Units: %
+      Default: 100
+      Flags: experimental
+
+      Percentage permitted overflow queue length.
+
+      This sets the ratio of queued requests to worker threads, above which sessions will be dropped instead of
+      queued.
+
+ping_interval
+      Units: seconds
+      Default: 3
+      Flags: must_restart
+
+      Interval between pings from parent to child.
+      Zero will disable pinging entirely, which makes it possible to attach a debugger to the child.
+
+pipe_timeout
+      Units: seconds
+      Default: 60
+
+      Idle timeout for PIPE sessions. If nothing have been received in either direction for this many seconds,
+      the session is closed.
+
+prefer_ipv6
+      Units: bool
+      Default: off
+
+      Prefer IPv6 address when connecting to backends which have both IPv4 and IPv6 addresses.
+
+purge_dups
+      Units: bool
+      Default: on
+
+      Detect and eliminate duplicate purges.
+
+rush_exponent
+      Units: requests per request
+      Default: 3
+      Flags: experimental
+
+      How many parked request we start for each completed request on the object.
+      NB: Even with the implict delay of delivery, this parameter controls an exponential increase in number of
+      worker threads.
+
+saintmode_threshold
+      Units: objects
+      Default: 10
+      Flags: experimental
+
+      The maximum number of objects held off by saint mode before no further will be made to the backend until
+      one times out.  A value of 0 disables saintmode.
+send_timeout
+      Units: seconds
+      Default: 600
+      Flags: delayed
+
+      Send timeout for client connections. If no data has been sent to the client in this many seconds, the ses‐
+      sion is closed.
+      See setsockopt(2) under SO_SNDTIMEO for more information.
+
+sendfile_threshold
+      Units: bytes
+      Default: -1
+      Flags: experimental
+
+      The minimum size of objects transmitted with sendfile.
+
+sess_timeout
+      Units: seconds
+      Default: 5
+
+      Idle timeout for persistent sessions. If a HTTP request has not been received in this many seconds, the
+      session is closed.
+
+sess_workspace
+      Units: bytes
+      Default: 65536
+      Flags: delayed
+
+      Bytes of HTTP protocol workspace allocated for sessions. This space must be big enough for the entire HTTP
+      protocol header and any edits done to it in the VCL code.
+      Minimum is 1024 bytes.
+
+session_linger
+      Units: ms
+      Default: 50
+      Flags: experimental
+
+      How long time the workerthread lingers on the session to see if a new request appears right away.
+      If sessions are reused, as much as half of all reuses happen within the first 100 msec of the previous
+      request completing.
+      Setting this too high results in worker threads not doing anything for their keep, setting it too low just
+      means that more sessions take a detour around the waiter.
+
+session_max
+      Units: sessions
+      Default: 100000
+
+      Maximum number of sessions we will allocate before just dropping connections.
+      This is mostly an anti-DoS measure, and setting it plenty high should not hurt, as long as you have the
+      memory for it.
+
+shm_reclen
+      Units: bytes
+      Default: 255
+
+      Maximum number of bytes in SHM log record.
+      Maximum is 65535 bytes.
+
+shm_workspace
+      Units: bytes
+      Default: 8192
+      Flags: delayed
+
+      Bytes of shmlog workspace allocated for worker threads. If too big, it wastes some ram, if too small it
+      causes needless flushes of the SHM workspace.
+      These flushes show up in stats as "SHM flushes due to overflow".
+      Minimum is 4096 bytes.
+
+syslog_cli_traffic
+      Units: bool
+      Default: on
+
+      Log all CLI traffic to syslog(LOG_INFO).
+
+thread_pool_add_delay
+      Units: milliseconds
+      Default: 20
+      Flags: experimental
+
+      Wait at least this long between creating threads.
+
+      Setting this too long results in insuffient worker threads.
+
+      Setting this too short increases the risk of worker thread pile-up.
+
+thread_pool_add_threshold
+      Units: requests
+      Default: 2
+      Flags: experimental
+
+      Overflow threshold for worker thread creation.
+
+      Setting this too low, will result in excess worker threads, which is generally a bad idea.
+
+      Setting it too high results in insuffient worker threads.
+
+thread_pool_fail_delay
+      Units: milliseconds
+      Default: 200
+      Flags: experimental
+
+      Wait at least this long after a failed thread creation before trying to create another thread.
+
+      Failure to create a worker thread is often a sign that  the end is near, because the process is running out
+      of RAM resources for thread stacks.
+      This delay tries to not rush it on needlessly.
+
+      If thread creation failures are a problem, check that thread_pool_max is not too high.
+
+      It may also help to increase thread_pool_timeout and thread_pool_min, to reduce the rate at which treads
+      are destroyed and later recreated.
+
+thread_pool_max
+      Units: threads
+      Default: 500
+      Flags: delayed, experimental
+
+      The maximum number of worker threads in all pools combined.
+
+      Do not set this higher than you have to, since excess worker threads soak up RAM and CPU and generally just
+      get in the way of getting work done.
+
+thread_pool_min
+      Units: threads
+      Default: 5
+      Flags: delayed, experimental
+
+      The minimum number of threads in each worker pool.
+
+      Increasing this may help ramp up faster from low load situations where threads have expired.
+
+      Minimum is 2 threads.
+
+thread_pool_purge_delay
+      Units: milliseconds
+      Default: 1000
+      Flags: delayed, experimental
+
+      Wait this long between purging threads.
+
+      This controls the decay of thread pools when idle(-ish).
+
+      Minimum is 100 milliseconds.
+
+thread_pool_stack
+      Units: bytes
+      Default: -1
+      Flags: experimental
+
+      Worker thread stack size.  In particular on 32bit systems you may need to tweak this down to fit many
+      threads into the limited address space.
+
+thread_pool_timeout
+      Units: seconds
+      Default: 300
+      Flags: delayed, experimental
+
+      Thread idle threshold.
+
+      Threads in excess of thread_pool_min, which have been idle for at least this long are candidates for purg‐
+      ing.
+
+      Minimum is 1 second.
+
+thread_pools
+      Units: pools
+      Default: 2
+      Flags: delayed, experimental
+
+      Number of worker thread pools.
+
+      Increasing number of worker pools decreases lock contention.
+
+      Too many pools waste CPU and RAM resources, and more than one pool for each CPU is probably detrimal to
+      performance.
+
+      Can be increased on the fly, but decreases require a restart to take effect.
+
+thread_stats_rate
+      Units: requests
+      Default: 10
+      Flags: experimental
+
+      Worker threads accumulate statistics, and dump these into the global stats counters if the lock is free
+      when they finish a request.
+      This parameters defines the maximum number of requests a worker thread may handle, before it is forced to
+      dump its accumulated stats into the global counters.
+
+user  Default: .....
+      Flags: must_restart
+
+      The unprivileged user to run as.  Setting this will also set "group" to the specified user's primary group.
+
+vcl_trace
+      Units: bool
+      Default: off
+
+      Trace VCL execution in the shmlog.
+      Enabling this will allow you to see the path each request has taken through the VCL program.
+      This generates a lot of logrecords so it is off by default.
+
+waiter
+      Default: default
+      Flags: must_restart, experimental
+
+      Select the waiter kernel interface.
+
+
+Purge expressions
+-----------------
+
+A purge expression consists of one or more conditions.  A condition consists of a field, an operator, and an
+argument.  Conditions can be ANDed together with "&&".
+
+A field can be any of the variables from VCL, for instance req.url, req.http.host or obj.set-cookie.
+
+Operators are "==" for direct comparision, "~" for a regular expression match, and ">" or "<" for size compar‐
+isons.  Prepending an operator with "!" negates the expression.
+
+The argument could be a quoted string, a regexp, or an integer.  Integers can have "KB", "MB", "GB" or "TB"
+appended for size related fields.
+
+Simple example: All requests where req.url exactly matches the string /news are purged from the cache.
+
+    req.url == "/news"
+
+Example: Purge all documents where the name does not end with ".ogg", and where the size of the object is greater
+than 10 megabytes.
+
+    req.url !~ "\.ogg$" && obj.size > 10MB
+
+Example: Purge all documents where the serving host is "example.com" or "www.example.com", and where the Set-
+Cookie header received from the backend contains "USERID=1663".
+
+    req.http.host ~ "^(www\.)example.com$" && obj.set-cookie ~ "USERID=1663"
+
+SEE ALSO
+========
+
+* varnishlog(1)
+* varnishhist(1)
+* varnishncsa(1)
+* varnishstat(1)
+* varnishtop(1)
+* vcl(7)
+
+HISTORY
+=======
+
+The varnishd daemon was developed by Poul-Henning Kamp in cooperation
+with Verdens Gang AS, Linpro AS and Varnish Software.
+
+This manual page was written by Dag-Erling Smørgrav with updates by
+Stig Sandbeck Mathisen ⟨ssm@debian.org⟩
+

doc/sphinx/reference/varnishhist.rst

+===========
+varnishhist
+===========
+
+-------------------------
+Varnish request histogram
+-------------------------
+
+SYNOPSIS
+========
+
+varnishhist [-b] [-C] [-c] [-d] [-I regex] [-i tag] [-n varnish_name] 
+[-r file] [-V] [-w delay] [-X regex] [-x tag]
+
+DESCRIPTION
+===========
+
+The varnishhist utility reads varnishd(1) shared memory logs and
+presents a continuously updated histogram show‐ ing the distribution
+of the last N requests by their processing.  The value of N and the
+vertical scale are dis‐ played in the top left corner.  The horizontal
+scale is logarithmic.  Hits are marked with a pipe character ("|"),
+and misses are marked with a hash character ("#").
+
+The following options are available:
+
+-b          Include log entries which result from communication with 
+	    a backend server.  If neither -b nor -c is
+	    specified, varnishhist acts as if they both were.
+
+-C          Ignore case when matching regular expressions.
+
+-c          Include log entries which result from communication with 
+	    a client.  If neither -b nor -c is specified, 
+	    varnishhist acts as if they both were.
+
+-d          Process old log entries on startup.  Normally, varnishhist 
+	    will only process entries which are written to the 
+	    log after it starts.
+
+-I regex    Include log entries which match the specified 
+   	    regular expression.  If neither -I nor -i is specified, 
+	    all log entries are included.
+
+-i tag      Include log entries with the specified tag.  If neither 
+   	    -I nor -i is specified, all log entries are included.
+
+-n          Specifies the name of the varnishd instance to get logs 
+	    from.  If -n is not specified, the host name is used.
+
+-r file     Read log entries from file instead of shared memory.
+
+-V          Display the version number and exit.
+
+-w delay    Wait at least delay seconds between each update.  The 
+   	    default is 1.  file instead of displaying them.  The file 
+	    will be overwritten unless the -a option was specified.
+
+-X regex    Exclude log entries which match the specified regular expression.
+
+-x tag      Exclude log entries with the specified tag.
+
+SEE ALSO
+========
+
+* varnishd(1)
+* varnishlog(1)
+* varnishncsa(1)
+* varnishstat(1) 
+* varnishtop(1)
+
+HISTORY
+=======
+The varnishhist utility was developed by Poul-Henning Kamp in cooperation with Verdens Gang
+AS and Linpro AS.  This manual page was written by Dag-Erling Smørgrav and Per Buer.
+
+COPYRIGHT
+=========
+
+This document is licensed under the same licence as Varnish
+itself. See LICENCE for details.
+
+* Copyright (c) 2006 Verdens Gang AS
+* Copyright (c) 2006-2008 Linpro AS
+* Copyright (c) 2008-2010 Redpill Linpro AS
+* Copyright (c) 2010 Varnish Software AS

doc/sphinx/reference/varnishncsa.rst

+===========
+varnishncsa
+===========
+
+---------------------------------------------------------
+Display Varnish logs in Apache / NCSA combined log format
+---------------------------------------------------------
+
+SYNOPSIS
+========
+
+varnishncsa [-a] [-b] [-C] [-c] [-D] [-d] [-f] [-I regex] 
+[-i tag] [-n varnish_name] [-P file] [-r file] [-V] 
+[-w file] [-X regex] [-x tag]
+
+DESCRIPTION
+===========
+
+The varnishncsa utility reads varnishd(1) shared memory logs and
+presents them in the Apache / NCSA "combined" log format.
+
+The following options are available:
+
+-a          When writing to a file, append to it rather than overwrite it.
+
+-b          Include log entries which result from communication with a 
+	    backend server.  If neither -b nor -c is
+	    specified, varnishncsa acts as if they both were.
+
+-C          Ignore case when matching regular expressions.
+
+-c          Include log entries which result from communication 
+	    with a client.  If neither -b nor -c is specified, 
+	    varnishncsa acts as if they both were.
+
+-D          Daemonize.
+
+-d          Process old log entries on startup.  Normally, varnishncsa 
+	    will only process entries which are written to the log 
+	    after it starts.
+
+-f          Prefer the X-Forwarded-For HTTP header over client.ip in 
+	    the log output.
+
+-I regex    Include log entries which match the specified regular 
+   	    expression.  If neither -I nor -i is specified, 
+	    all log entries are included.
+
+-i tag      Include log entries with the specified tag.  If neither -I nor 
+   	    -i is specified, all log entries are included.
+
+-n          Specifies the name of the varnishd instance to get logs 
+	    from.  If -n is not specified, the host name is used.
+
+-P file     Write the process's PID to the specified file.
+
+-r file     Read log entries from file instead of shared memory.
+
+-V          Display the version number and exit.
+
+-w file     Write log entries to file instead of displaying them.  
+   	    The file will be overwritten unless the -a
+	    option was specified.
+	    
+	    If varnishncsa receives a SIGHUP while writing to a file, 
+	    it will reopen the file, allowing the old one to be 
+	    rotated away.
+
+-X regex    Exclude log entries which match the specified 
+   	    regular expression.
+
+-x tag      Exclude log entries with the specified tag.
+
+SEE ALSO
+========
+
+* varnishd(1)
+* varnishhist(1)
+* varnishlog(1)
+* varnishstat(1)
+* varnishtop(1)
+
+HISTORY
+=======
+
+The varnishncsa utility was developed by Poul-Henning Kamp in
+cooperation with Verdens Gang AS and Linpro AS.  This manual page was
+written by Dag-Erling Smørgrav ⟨des@des.no⟩.

doc/sphinx/reference/varnishtest.rst

+===========
+varnishtest
+===========
+
+------------------------
+Test program for Varnish
+------------------------
+
+:Author: Stig Sandbeck Mathisen
+:Date:   2010-05-31
+:Version: 1.0
+:Manual section: 1
+
+
+SYNOPSIS
+========
+     varnishtest [-n iter] [-q] [-v] file [file ...]
+
+DESCRIPTION
+===========
+
+The varnishtest program is a script driven program used to test the
+varnish HTTP accelerator.
+
+The varnishtest program, when started and given one or more script
+files, can create a number of threads repre senting backends, some
+threads representing clients, and a varnishd process.
+
+The following options are available:
+
+-n iter     Run iter number of iterations.
+
+-q          Be quiet.
+
+-v          Be verbose.
+
+-L port     Listen on *port*. 
+
+-t          Dunno.
+
+file        File to use as a script
+
+
+SCRIPTS
+=======
+
+Example script
+~~~~~~~~~~~~~~
+::
+
+    # Start a varnish instance called "v1"
+    varnish v1 -arg "-b localhost:9080" -start
+    
+    # Create a server thread called "s1"
+    server s1 {
+        # Receive a request
+        rxreq
+        # Send a standard response
+        txresp -hdr "Connection: close" -body "012345\n"
+    }
+    
+    # Start the server thread
+    server s1 -start
+    
+    # Create a client thread called "c1"
+    client c1 {
+        # Send a request
+        txreq -url "/"
+        # Wait for a response
+        rxresp
+    # Insist that it be a success
+    expect resp.status == 200
+    }
+    
+    # Run the client
+    client c1 -run
+    
+    # Wait for the server to die
+    server s1 -wait
+
+    # (Forcefully) Stop the varnish instance.
+    varnish v1 -stop
+
+Example script output
+~~~~~~~~~~~~~~~~~~~~~
+
+The output, running this script looks as follows. The "bargraph" at
+the beginning of the line is an indication of the level of detail in
+the line. The second field where the message comes from. The rest of
+the line is anyones guess :-)
+::
+
+    #  TEST tests/b00000.vtc starting
+    ### v1  CMD: cd ../varnishd && ./varnishd -d -d -n v1 -a :9081 -T :9001 -b localhost:9080
+    ### v1  opening CLI connection
+    #### v1  debug| NB: Storage size limited to 2GB on 32 bit architecture,\n
+    #### v1  debug| NB: otherwise we could run out of address space.\n
+    #### v1  debug| storage_file: filename: ./varnish.Shkoq5 (unlinked) size 2047 MB.\n
+    ### v1  CLI connection fd = 3
+    #### v1  CLI TX| start
+    #### v1  debug| Using old SHMFILE\n
+    #### v1  debug| Notice: locking SHMFILE in core failed: Operation not permitted\n
+    #### v1  debug| bind(): Address already in use\n
+    #### v1  debug| rolling(1)...
+    #### v1  debug| \n
+    #### v1  debug| rolling(2)...\n
+    #### v1  debug| Debugging mode, enter "start" to start child\n
+    ### v1  CLI 200 <start>
+    ##  s1  Starting server
+    ### s1  listen on :9080 (fd 6)
+    ##  c1  Starting client
+    ##  c1  Waiting for client
+    ##  s1  started on :9080
+    ##  c1  started
+    ### c1  connect to :9081
+    ### c1  connected to :9081 fd is 8
+    #### c1  | GET / HTTP/1.1\r\n
+    #### c1  | \r\n
+    ### c1  rxresp
+    #### s1  Accepted socket 7
+    ### s1  rxreq
+    #### s1  | GET / HTTP/1.1\r\n
+    #### s1  | X-Varnish: 422080121\r\n
+    #### s1  | X-Forwarded-For: 127.0.0.1\r\n
+    #### s1  | Host: localhost\r\n
+    #### s1  | \r\n
+    #### s1  http[ 0] | GET
+    #### s1  http[ 1] | /
+    #### s1  http[ 2] | HTTP/1.1
+    #### s1  http[ 3] | X-Varnish: 422080121
+    #### s1  http[ 4] | X-Forwarded-For: 127.0.0.1
+    #### s1  http[ 5] | Host: localhost
+    #### s1  | HTTP/1.1 200 Ok\r\n
+    #### s1  | Connection: close\r\n
+    #### s1  | \r\n
+    #### s1  | 012345\n
+    #### s1  | \r\n
+    ##  s1  ending
+    #### c1  | HTTP/1.1 200 Ok\r\n
+    #### c1  | Content-Length: 9\r\n
+    #### c1  | Date: Mon, 16 Jun 2008 22:16:55 GMT\r\n
+    #### c1  | X-Varnish: 422080121\r\n
+    #### c1  | Age: 0\r\n
+    #### c1  | Via: 1.1 varnish\r\n
+    #### c1  | Connection: keep-alive\r\n
+    #### c1  | \r\n
+    #### c1  http[ 0] | HTTP/1.1
+    #### c1  http[ 1] | 200
+    #### c1  http[ 2] | Ok
+    #### c1  http[ 3] | Content-Length: 9
+    #### c1  http[ 4] | Date: Mon, 16 Jun 2008 22:16:55 GMT
+    #### c1  http[ 5] | X-Varnish: 422080121
+    #### c1  http[ 6] | Age: 0
+    #### c1  http[ 7] | Via: 1.1 varnish
+    #### c1  http[ 8] | Connection: keep-alive
+    #### c1  EXPECT resp.status (200) == 200 (200) match
+    ##  c1  ending
+    ##  s1  Waiting for server
+    #### v1  CLI TX| stop
+    ### v1  CLI 200 <stop>
+    #  TEST tests/b00000.vtc completed
+
+If instead of 200 we had expected 201 with the line:::
+
+  expect resp.status == 201
+
+The output would have ended with:::
+
+  #### c1  http[ 0] | HTTP/1.1
+  #### c1  http[ 1] | 200
+  #### c1  http[ 2] | Ok
+  #### c1  http[ 3] | Content-Length: 9
+  #### c1  http[ 4] | Date: Mon, 16 Jun 2008 22:26:35 GMT
+  #### c1  http[ 5] | X-Varnish: 648043653 648043652
+  #### c1  http[ 6] | Age: 6
+  #### c1  http[ 7] | Via: 1.1 varnish
+  #### c1  http[ 8] | Connection: keep-alive
+  ---- c1  EXPECT resp.status (200) == 201 (201) failed
+
+SEE ALSO
+========
+
+* varnishhist(1)
+* varnishlog(1)
+* varnishncsa(1)
+* varnishstat(1)
+* varnishtop(1)
+* vcl(7)
+
+HISTORY
+=======
+
+The varnishtest program was developed by Poul-Henning Kamp
+⟨phk@phk.freebsd.dk⟩ in cooperation with Linpro AS. This manual page
+was written by Stig Sandbeck Mathisen ⟨ssm@linpro.no⟩ using examples
+by Poul-Henning Kamp ⟨phk@phk.freebsd.dk⟩.
+
+COPYRIGHT
+=========
+
+This document is licensed under the same licence as Varnish
+itself. See LICENCE for details.
+
+* Copyright (c) 2007-2008 Linpro AS
+* Copyright (c) 2010 Varnish Software AS

doc/sphinx/reference/varnishtop.rst

+============
+varnishtop
+============
+
+-------------------------
+Varnish log entry ranking
+-------------------------
+
+SYNOPSIS
+========
+
+varnishtop [-1] [-b] [-C] [-c] [-d] [-f] [-I regex] 
+[-i tag] [-n varnish_name] [-r file] [-V] [-X regex]
+[-x tag]
+
+DESCRIPTION
+===========
+
+The varnishtop utility reads varnishd(1) shared memory logs and
+presents a continuously updated list of the most commonly occurring
+log entries.  With suitable filtering using the -I, -i, -X and -x
+options, it can be used to display a ranking of requested documents,
+clients, user agents, or any other information which is recorded in
+the log.
+
+The following options are available:
+
+-1          Instead of presenting of a continuously updated display, 
+	    print the statistics once and exit. Implies -d.
+
+-b          Include log entries which result from communication 
+	    with a backend server.  If neither -b nor -c is
+	    specified, varnishtop acts as if they both were.
+
+-C          Ignore case when matching regular expressions.
+
+-c          Include log entries which result from communication 
+	    with a client.  If neither -b nor -c is specified, 
+	    varnishtop acts as if they both were.
+
+-d          Process old log entries on startup.  Normally, varnishtop 
+	    will only process entries which are written to the log 
+	    after it starts.
+
+-f          Sort and group only on the first field of each log entry.  
+	    This is useful when displaying e.g. stataddr entries, 
+	    where the first field is the client IP address.
+
+-I regex    Include log entries which match the specified regular 
+   	    expression.  If neither -I nor -i is specified, all log 
+	    entries are included.
+
+-i tag      Include log entries with the specified tag.  If neither -I 
+   	    nor -i is specified, all log entries are included.
+
+-n          Specifies the name of the varnishd instance to get logs from.  
+	    If -n is not specified, the host name is used.
+
+-r file     Read log entries from file instead of shared memory.
+
+-V          Display the version number and exit.
+
+-X regex    Exclude log entries which match the specified regular expression.
+
+-x tag      Exclude log entries with the specified tag.
+
+EXAMPLES
+========
+
+The following example displays a continuously updated list of the most
+frequently requested URLs:::
+
+  varnishtop -i RxURL
+
+The following example displays a continuously updated list of the most
+commonly used user agents:::
+
+  varnishtop -i RxHeader -C -I ^User-Agent
+
+SEE ALSO
+========
+
+* varnishd(1)
+* varnishhist(1)
+* varnishlog(1)
+* varnishncsa(1)
+* varnishstat(1)
+
+HISTORY
+=======
+
+The varnishtop utility was originally developed by Poul-Henning Kamp
+in cooperation with Verdens Gang AS and Linpro AS, and later
+substantially rewritten by Dag-Erling Smørgrav.  This manual page was
+written by Dag-Erling Smørgrav.