Skip to content

L
libvdp-pesi
Commit 162f9ec9 authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Doc WIP

parent 5eb9be70
Hide whitespace changes
Inline Side-by-side
Showing with 931 additions and 193 deletions
README.rst
View file @ 162f9ec9
......@@ -23,142 +23,302 @@ SYNOPSIS
import pesi;
# Enable parallel ESI processing in vcl_deliver.
VOID pesi.activate()
# Set a boolean configuration parameter.
VOID pesi.set(ENUM, BOOL)
# Configure workspace pre-allocation for internal variable-sized
# data structures.
VOID pesi.workspace_prealloc(BYTES min_free, INT max_nodes)
# Configure the memory pool used when pre-allocated structures
# from the workspace are insufficient.
VOID pesi.pool(INT min, INT max, DURATION max_age)
# VDP version
STRING pesi.version()
DESCRIPTION
===========
.. _varnishd(1): https://varnish-cache.org/docs/trunk/reference/varnishd.html
VDP pesi is a Varnish Delivery Processor for parallel Edge Side
Includes (ESI). ...
.. _vcl(7): https://varnish-cache.org/docs/trunk/reference/vcl.html
.. _vmod_pesi.pool:
.. _varnishadm(1): https://varnish-cache.org/docs/trunk/reference/varnishadm.html
VOID pool(INT min=10, INT max=100, DURATION max_age=10)
-------------------------------------------------------
.. _varnishstat(1): https://varnish-cache.org/docs/trunk/reference/varnishstat.html
Configure the memory pool used by the VDP for internal variable-sized
data structures.
DESCRIPTION
===========
.. _standard ESI processing: https://varnish-cache.org/docs/trunk/users-guide/esi.html
VDP pesi is a Varnish Delivery Processor for parallel Edge Side
Includes (ESI). The VDP implements content composition in client
responses as specified by ``<esi>`` directives in the response body,
just as Varnish does with its `standard ESI processing`_. While
standard Varnish processes ESI subrequests serially, in the order in
which the ``<esi>`` directives appear in the response, the VDP
executes the subrequests in parallel. This can lead to a significant
reduction in latency for the complete response, if Varnish has to wait
for backend fetches for more than one of the included requests.
Backend applications that use ESI includes for standard Varnish can be
expected to work without changes with the VDP, provided that they do
not depend on assumptions about the serialization of ESI subrequests.
Serial ESI requests are processed in a predictable order, one after
the other, but the VDP executes them at roughly the same time. A
backend may conceivably receive a request forwarded for the second
include in a response before the first one. If the logic of ESI
composition in a standard Varnish deployment does not depend on the
serial order, then it will work the same way with VDP pesi.
Parallel ESI processing is enabled by invoking ``pesi.activate()`` in
``vcl_deliver``::
import pesi;
sub vcl_backend_response {
set beresp.do_esi = true;
}
sub vcl_deliver {
pesi.activate();
}
Other functions provided by the VDP serve to set configuration
parameters (or return the VDP version string). If your deployment uses
the default configuration, then ``pesi.activate()`` in ``vcl_deliver``
may be the only modification to VCL that you need.
The invocation of ``pesi.activate()`` can of course be subject to
logic in VCL::
sub vcl_deliver {
# Use parallel ESI only if the request header X-PESI is present.
if (req.http.X-PESI) {
pesi.activate();
}
}
But see below for restrictions on the use of ``pesi.activate()``.
All of the computing resources used by the VDP -- threads, storage,
workspace, locks, and so on -- can be configured, either with Varnish
runtime parameters or configuration settings made available by the
VDP. And their usage can be monitored with Varnish statistics. So you
can limit resource usage, and use monitoring tools such as
`varnishstat(1)`_ to ensure efficient parallel ESI processing. For
details see `CONFIGURATION AND MONITORING`_ below.
.. _vmod_pesi.activate:
VOID activate()
---------------
To be called from vcl_deliver {} only, must be called on all ESI
levels if called on any ESI level (unless you are a wizard).
Enable parallel ESI processing for the client response.
``activate()`` MUST be called in ``vcl_deliver`` only. If it is called
in any other VCL subroutine, VCL failure is invoked (see `ERRORS`_
below for details).
If ``activate()`` is called on *any* ESI level (any depth of include
nesting), then it MUST be called on *all* levels of the response. If
``activate()`` is invoked at some ESI levels but not others, then the
results are undefined, and will very likely lead to a Varnish panic.
Typically it suffices to simply call ``activate()`` in
``vcl_deliver``, since the code in ``vcl_deliver`` is executed at
every ESI level. It is also safe, for instance, to call ``activate()``
only if a request header is present, as in the example shown above;
since the same request headers are set for every ESI subrequest, the
result is the same at every ESI level. But that should *not* be done
if you have logic that unsets the header at some ESI levels but not at
others. Under no circumstances should the invocation of ``activate()``
depend on the value of ``req.esi_level``, or of ``req.url`` (since
URLs are different at different ESI levels).
See the documentation of ``set()`` below for a way to choose serial
ESI processing for all of the includes in the response at the current
ESI level. Even then, ``activate()`` must be called in ``vcl_deliver``
in addition to ``set()``.
XXX: test the following:
As with standard Varnish, ESI processing can be selectively disabled
for a client response, by setting ``resp.do_esi`` to ``false`` in VCL
since version 4.1, or setting ``req.esi`` to ``false`` in VCL 4.0 (see
`vcl(7)`_). The requirement remains: if ESI processing is enabled and
``activate()`` is called at any ESI level, then both must happen at
all levels.
``activate()`` has the effect of setting the VCL string variable
``resp.filters``, which is a whitespace-separated list of the names of
delivery processors to be applied to the client response (see
`vcl(7)`_). It configures the correct list of filters for the current
response, analogous to the default filter settings in Varnish when
sequential ESI is in use. These include the ``gunzip`` VDP for
uncompressed responses, and ``range`` for responses to range
requests. ``activate()`` checks the conditions for which the VDPs are
required, and arranges them in the correct order.
It is possible to manually set or change ``resp.filters`` to enable
parallel ESI, instead of calling ``activate()``, but that is only
advised to experts. If you do so, use the string ``pesi`` for this
VDP, and do *not* include ``esi``, for Varnish's standard ESI VDP, in
the same list with ``pesi``. As with the ``activate()`` call -- if
``pesi`` appears in ``resp.filters`` for a response at *any* ESI
level, it MUST be in ``resp.filters`` at *all* ESI levels.
Notice that all VCL code affecting ESI (such as setting
``resp.do_esi``), gzip (such as changes to
``req.http.Accept-Encoding``) or range processing (such as changes
``req.http.Range``) must execute before this function is called to
have an effect.
Example::
vcl 4.1;
Configure the correct parallel ESI filters for this response analogous
to the default filter settings in varnish-cache for sequential ESI.
import pesi;
Manually setting/changing ``resp.filters`` is possible but only
advised to experts.
sub vcl_recv {
# Disable gzipped responses by removing Accept-Encoding.
unset req.http.Accept-Encoding;
}
Notice that all settings affecting ESI (like ``resp.do_esi``), gzip
(like changing ``req.http.Accept-Encoding``) or Range processing (like
changing ``req.http.Range``) should happen before calling this
function to have an effect.
sub vcl_backend_response {
set beresp.do_esi = true;
}
If this function is called on any ESI level, it should be called on
all ESI levels. Failure to do so will likely cause panics caused by
missing error handling in varnish-cache. Notice that the VMOD has no
known way to check for this condition, so please address any blame
appropriately.
sub vcl_deliver {
# If the request header X-Debug-ESI is present, then disable ESI
# for the current response.
if (req.http.X-Debug-ESI) {
set resp.do_esi = false;
}
pesi.activate()
}
.. _vmod_pesi.set:
VOID set(ENUM {serial, thread} parameter, [BOOL bool])
------------------------------------------------------
To be called from ``vcl_deliver {}`` only.
Set a configuration parameter for the VDP, which holds for the current
(sub)request, as documented below. The parameter to be set is
identified by the ENUM ``parameter``. Currently the parameters can
only be set with a boolean value in ``bool`` (but future versions of
this function may allow for setting other data types).
``set()`` MUST be called in ``vcl_deliver`` only; otherwise VCL
failure is invoked (see `ERRORS`_).
Set per (sub)request parameters for pesi which are documented
below. Parameters may require one or more of the other additional
arguments as documented below. Failure to provide them triggers a VCL
error.
The parameters that can be set are currently ``serial`` and ``thread``:
* ``serial``, requires *bool* argument
``serial``
----------
Activates serial mode if *bool* is ``true``, default is ``false``.
Activates serial mode if ``bool`` is ``true``; default is ``false``.
In serial mode, no new threads will be started from this request, so
all ESI subrequests at the next level will be processed by the
current thread. In other words, the setting only affects include
processing for the current response body.
In serial mode, the ESI subrequests processed for includes in the
current response body are processed in serial, in the current thread.
In other words, all ESI subrequests at the next level will be
processed without requesting threads from the thread pool (which
potentially starts new threads, if necessary). This setting only
affects include processing at the current ESI level, not nested
includes at the next level.
It is strongly recommended to _not_ use serial mode from ESI level 0
because the ESI level 0 thread can send available data to the client
concurrently to other parallel ESI threads.
It is strongly recommended to *not* use serial mode from ESI level 0
(the top level request received from a client), because the ESI level
0 thread can send available data to the client concurrently with other
parallel ESI threads.
Serial mode may sensibly be used to reduce overhead and the number
of threads required without relevant drawbacks
Serial mode may sensibly be used to reduce overhead and the number of
threads required without relevant drawbacks:
- at esi_level > 0
* at ESI level > 0
- when the vcl author knows that all objects included by the current
request will be cacheable and thus highly likely cause cache hits.
* when the VCL author knows that all objects included by the current
request are cacheable, and thus are highly likely to lead to cache
hits.
Also see the discussion below for more detail.
XXX ... Also see the discussion below for more detail.
* ``thread``, requires *bool* argument
``thread``
----------
Whether we always request a new thread for includes, default is
``true``.
Whether we always request a new thread for includes, default is
``true``.
- ``false``
* ``false``
Only use a new thread if immediately available, process the
include in the same thread otherwise.
Only use a new thread if immediately available, process the include
in the same thread otherwise.
- ``true``
* ``true``
Request a new thread, potentially waiting for one to become
available.
Request a new thread, potentially waiting for one to become
available.
For parallel ESI to work as efficiently as possible, it should
traverse the ESI tree *breadth first*, processing any ESI object
completely, with new threads scheduled for any includes
encountered. Completing processing of an ESI object allows for data
from the subtree (the ESI object and anything below) to be sent to
the client concurrently. As soon as ESI object processing is
complete, the respective thread will be returned to the thread pool
and become available for any other varnish task (except for the
request for esi_level 0, which _has_ to wait for completion of the
entire ESI request anyway and will send data to the client in the
meantime).
XXX move the longer discussion to a document dedicated to the subjects
of tuning, efficiency etc
With this setting to ``true`` (the default), this is what happens
always, but a thread may not be immediately available if the thread
pool is not sufficiently sized for the current load and thus the
include request may have to be queued.
For parallel ESI to work as efficiently as possible, it should
traverse the ESI tree *breadth first*, processing any ESI object
completely, with new threads scheduled for any includes
encountered. Completing processing of an ESI object allows for data
from the subtree (the ESI object and anything below) to be sent to the
client concurrently. As soon as ESI object processing is complete, the
respective thread will be returned to the thread pool and become
available for any other varnish task (except for the request for
esi_level 0, which _has_ to wait for completion of the entire ESI
request anyway and will send data to the client in the meantime).
With this setting to ``false``, include processing happens in the
same thread as if ``serial`` mode had been activated, but only in
the case where there is no new thread available. While this may
sound like the more sensible option at first, we did not make this
the default for the following reasons:
With this setting to ``true`` (the default), this is what happens
always, but a thread may not be immediately available if the thread
pool is not sufficiently sized for the current load and thus the
include request may have to be queued.
- Before completion of the ESI processing, the subtree below it is
not yet available for delivery to the client because additional
VDPs behind pesi cannot be called from a different thread.
With this setting to ``false``, include processing happens in the same
thread as if ``serial`` mode had been activated, but only in the case
where there is no new thread available. While this may sound like the
more sensible option at first, we did not make this the default for
the following reasons:
- While processing of the include may take an arbitrarily long time
(for example because it requires a lengthy backend fetch), we know
that the ESI object is fully available in the stevedore (and
usually in memory already) when we parse an include because
streaming is not supported for ESI. So we know that completing the
processing of the current ESI object will be quick, while
descending into a subtree may be take a long time.
* Before completion of the ESI processing, the subtree below it is not
yet available for delivery to the client because additional VDPs
behind pesi cannot be called from a different thread.
- Except for ESI level 0, the current thread will become available
as soon as ESI processing has completed.
* While processing of the include may take an arbitrarily long time
(for example because it requires a lengthy backend fetch), we know
that the ESI object is fully available in the stevedore (and usually
in memory already) when we parse an include because streaming is not
supported for ESI. So we know that completing the processing of the
current ESI object will be quick, while descending into a subtree
may be take a long time.
- The thread herder may breed new threads and other threads may
terminate, so queuing a thread momentarily is not a bad thing per
se.
* Except for ESI level 0, the current thread will become available as
soon as ESI processing has completed.
In short, keeping ``thread`` at the default ``true`` should be the
right option, the alternative exists just in case.
* The thread herder may breed new threads and other threads may
terminate, so queuing a thread momentarily is not a bad thing per
se.
In short, keeping ``thread`` at the default ``true`` should be the
right option, the alternative exists just in case.
Example::
# Activate serial mode at ESI level > 0, if we know that all includes
# in the response at this level lead to cacheable responses.
sub vcl_deliver {
pesi.activate();
if (req.esi_level > 0 && req.url == "/all/cacheable/includes") {
pesi.set(serial, true);
}
}
.. _vmod_pesi.workspace_prealloc:
......@@ -172,7 +332,192 @@ VOID workspace_prealloc(BYTES min_free, INT max_nodes)
Configure workspace pre-allocation of objects in variable-sized
internal data structures.
XXX ...
For each request, the VDP builds such a structure, whose size is
roughly proportional to the size of the ESI tree -- the conceptual
tree with the top-level response at the root, and its includes and all
of their nested includes as branches. The nodes in this structure have
a fixed size, but the number of nodes used by the VDP varies with the
size of the ESI tree.
The VDP pre-allocates a constant number of such nodes in client
workspace, and initially takes nodes from the pre-allocation. If more
are needed for larger ESI trees, they are obtained from a global
memory pool as described below. The use of pre-allocated nodes from
workspace is preferred, since it never requires new system memory
allocations (workspaces themselves are pre-allocated by Varnish), and
because they are local to each request, so locking is never required
to access them (but is required for the memory pool).
The pre-allocation contributes a fixed size to client workspace usage,
since the number of pre-allocated nodes is constant. So any adjustment
to Varnish's ``workspace_client`` parameter that may be necessary due
to the pre-allocation will be valid for all requests.
``workspace_prealloc()`` configures the pre-allocation. The default
values of its parameters are defaults used by the VDP; that is, the
configuration if ``workspace_prealloc()`` is never called.
The ``min_free`` parameter sets the minimum amount of space that the
pre-allocation will always leave free in client workspace; if the
targeted number of pre-allocated nodes would result in less free space
than ``min_free`` bytes in workspace, then fewer nodes are
allocated. This ensures that free workspace is always left over for
other VMODs, VCL usage, and so forth. ``min_free`` defaults to 4 KiB.
The ``max_nodes`` parameter sets the number of nodes to be allocated,
unless the limit imposed by ``min_free`` is exceeded; ``max_nodes``
defaults to 32. ``max_nodes`` MUST be >= 0; otherwise, VCL failure is
invoked (see `ERRORS`_). If ``max_nodes`` is set to 0, then no nodes
are pre-allocated; they are all taken from the memory pool described
below.
When ``workspace_prealloc()`` is called, its configuration becomes
effective immediately for all new requests processed by the VDP. The
configuration remains valid for all instances of VCL, for as long as
the VDP remains loaded; that is, until the last instance of VCL using
the VDP is discarded.
``workspace_prealloc()`` can be called in ``vcl_init`` to set the
configuration at VCL load time. But you can also write VCL that calls
the function when a request is received by Varnish, for example using
a special URL for system administrators. This is similar to using the
``param.set`` command for `varnishadm(1)`_ to change a Varnish
parameter at runtime. Such a request should be protected, for example
with an ACL and/or Basic Authentication, so that it can be invoked
only by admins. Remember that as soon as such a request is processed
and ``workspace_prealloc()`` is executed, the changed configuration is
globally valid.
Examples::
# Configure workspace pre-allocation at VCL load time.
sub vcl_init {
pesi.workspace_prealloc(min_free=8k, max_nodes=64);
}
# Change the configuration at runtime, when Varnish receives an
# admin request.
import pesi;
import std;
sub vcl_recv {
if (req.url ~ "^/admin/pesi_ws") {
# Reject the request with "403 Forbidden" unless the client
# IP matches an ACL for admin requests.
if (client.ip !~ admin_acl) {
return (synth(403));
}
# Set min_free from a GET parameter, if present.
if (req.url ~ "\bmin_free=\d+[kmgtp]?") {
# Extract the BYTES parameter.
set req.http.Tmp-Bytes
= regsub(req.url, "^.+\bmin_free=(\d+[kmgtp]?).*$", "\1");
pesi.workspace_prealloc(std.bytes(req.http.Tmp-Bytes));
}
# Set max_nodes from a GET parameter.
if (req.url ~ "\bmax_nodes=\d+") {
# Extract the INT parameter.
set req.http.Tmp-Nodes
= regsub(req.url, "^.+\bmax_nodes=(\d+).*$", "\1");
pesi.workspace_prealloc(max_nodes=std.integer(req.http.Tmp-Nodes));
}
# Return status 204 to indicate success.
return (synth(204));
}
}
.. _vmod_pesi.pool:
VOID pool(INT min=10, INT max=100, DURATION max_age=10)
-------------------------------------------------------
Configure the memory pool used by the VDP for internal variable-sized
data structures, when more is needed than is provided by the client
workspace pre-allocation described above. The objects in the memory
pool are the nodes used in structures whose size is proportional to
the size of the ESI tree, as discussed above.
The VDP uses the same mechanism that Varnish uses for its memory
pools, and the configuration values have the same meaning and defaults
as the Varnish runtime parameters ``pool_req``, ``pool_sess`` and
``pool_vbo`` (see `varnishd(1)`_). ``min`` and ``max`` control the
size of the pool -- the number of pre-allocated nodes available for
allocation requests. ``max_age`` is the maximum lifetime for nodes in
the pool -- when there are no pending allocation requests, nodes in
the pool that are older than ``max_age`` are freed, down to the limit
imposed by ``min``.
The values of the parameters MUST fulfill the following requirements,
otherwise VCL failure is invoked (see `ERRORS`_):
* ``min`` and ``max`` MUST be both > 0.
* ``max`` MUST be >= ``min``.
* ``max_age`` MUST be >= 0s (and <= one million seconds).
Note that ``max`` is a soft limit. The memory pool satisfies all
allocation requests, even if ``max`` is execeeded when nodes are
returned to the pool. But the pool size will then be reduced to
``max``, without waiting for ``max_age`` to expire.
As with ``workspace_prealloc()``: when ``pool()`` is called, the
changed configuration immediately becomes valid (although it may take
some time for the memory pool to adjust to the new values). It remains
vaild for as long as the VDP is still loaded, unless ``pool()`` is
called again. ``pool()`` may be called in ``vcl_init`` to set a
configuration at VCL load time, but may also be called elsewhere in
VCL, for example to enable changing configurations at runtime using a
special "admin" request.
Examples::
# Configure the memory pool at VCL load time.
sub vcl_init {
pesi.pool(min=50, max=500, max_age=30s);
}
# Change the configuration at runtime, when Varnish receives an
# admin request.
import pesi;
import std;
sub vcl_recv {
if (req.url ~ "^/admin/pesi_pool") {
# Protect the call with an ACL, as in the example above.
if (client.ip !~ admin_acl) {
return (synth(403));
}
# Set max_age from a GET parameter.
if (req.url ~ "\bmax_age=\d+(\.\d+)?(ms|s|m|h|d|w|y)") {
# Extract the DURATION parameter.
set req.http.Tmp-Duration
= regsub(req.url,
"^.\bmax_age=(\d+(?:\.\d+)?(?:ms|s|m|h|d|w|y))+.*$",
"\1");
pesi.pool(max_age=std.duration(req.http.Tmp-Duration));
}
# Set min from a GET parameter.
if (req.url ~ "\bmin=\d+") {
# Extract the INT parameter.
set req.http.Tmp-Min = regsub(req.url, "^.+\bmin=(\d+).*$", "\1");
pesi.pool(min=std.integer(req.http.Tmp-Min));
}
# Extract max from a GET parameter, the same way as for min,
# not repeated here ...
# Status 204 indicates success.
return (synth(204));
}
}
.. _vmod_pesi.version:
......@@ -185,13 +530,31 @@ Example::
std.log("Using VDP pesi version: " + pesi.version());
REQUIREMENTS
============
ERRORS
======
As documented above, VCL failure is invoked under some of the error
conditions for functions provided by the VDP. VCL failure has the same
results as if ``return(fail)`` is called from a VCL subroutine:
The VDP currently requires the Varnish master branch. ...
* If the failure occurs in ``vcl_init``, then the VCL load fails with
an error message.
NOTES
=====
* If the failure occurs in any other subroutine besides ``vcl_synth``,
then a ``VCL_Error`` message is written to the log, and control is
directed immediately to ``vcl_synth``, with ``resp.status`` set to
503 and ``resp.reason`` set to ``"VCL failed"``.
* If the failure occurs in ``vcl_synth``, then ``vcl_synth`` is
aborted, and the response line "503 VCL failed" is sent.
CONFIGURATION AND MONITORING
============================
XXX ...
LIMITATIONS
===========
XXX @geoff pls reword
......@@ -200,8 +563,14 @@ events and whether or not we use (partial) sequential delivery (for
example, when no threads are available), ReqAcct adds n x 8 to the net
data size.
REQUIREMENTS
============
.. _bdf2f63: https://github.com/varnishcache/varnish-cache/commit/bdf2f63946e5abd4379f6a86ec0a273b387a4e59
This version of the VDP requires the Varnish master branch since
commit `bdf2f63`_. See the source repository for versions that are
compatible with other versions of Varnish (since 6.2.0).
INSTALLATION
============
......@@ -213,7 +582,6 @@ SEE ALSO
* varnishd(1)
* vcl(7)
* VMOD source repository: https://code.uplex.de/uplex-varnish/libvdp-pesi
COPYRIGHT
=========
......@@ -223,7 +591,8 @@ COPYRIGHT
Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
Nils Goroll <nils.goroll@uplex.de>
See LICENSE
src/vdp_pesi.vcc
View file @ 162f9ec9
......@@ -2,7 +2,8 @@
# Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
# All rights reserved
#
# Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
# Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
# Nils Goroll <nils.goroll@uplex.de>
#
# See LICENSE
#
......@@ -18,140 +19,485 @@ SYNOPSIS
import pesi;
# Enable parallel ESI processing in vcl_deliver.
VOID pesi.activate()
# Set a boolean configuration parameter.
VOID pesi.set(ENUM, BOOL)
# Configure workspace pre-allocation for internal variable-sized
# data structures.
VOID pesi.workspace_prealloc(BYTES min_free, INT max_nodes)
# Configure the memory pool used when pre-allocated structures
# from the workspace are insufficient.
VOID pesi.pool(INT min, INT max, DURATION max_age)
# VDP version
STRING pesi.version()
.. _varnishd(1): https://varnish-cache.org/docs/trunk/reference/varnishd.html
.. _vcl(7): https://varnish-cache.org/docs/trunk/reference/vcl.html
.. _varnishadm(1): https://varnish-cache.org/docs/trunk/reference/varnishadm.html
.. _varnishstat(1): https://varnish-cache.org/docs/trunk/reference/varnishstat.html
DESCRIPTION
===========
.. _standard ESI processing: https://varnish-cache.org/docs/trunk/users-guide/esi.html
VDP pesi is a Varnish Delivery Processor for parallel Edge Side
Includes (ESI). ...
Includes (ESI). The VDP implements content composition in client
responses as specified by ``<esi>`` directives in the response body,
just as Varnish does with its `standard ESI processing`_. While
standard Varnish processes ESI subrequests serially, in the order in
which the ``<esi>`` directives appear in the response, the VDP
executes the subrequests in parallel. This can lead to a significant
reduction in latency for the complete response, if Varnish has to wait
for backend fetches for more than one of the included requests.
Backend applications that use ESI includes for standard Varnish can be
expected to work without changes with the VDP, provided that they do
not depend on assumptions about the serialization of ESI subrequests.
Serial ESI requests are processed in a predictable order, one after
the other, but the VDP executes them at roughly the same time. A
backend may conceivably receive a request forwarded for the second
include in a response before the first one. If the logic of ESI
composition in a standard Varnish deployment does not depend on the
serial order, then it will work the same way with VDP pesi.
Parallel ESI processing is enabled by invoking ``pesi.activate()`` in
``vcl_deliver``::
import pesi;
sub vcl_backend_response {
set beresp.do_esi = true;
}
sub vcl_deliver {
pesi.activate();
}
Other functions provided by the VDP serve to set configuration
parameters (or return the VDP version string). If your deployment uses
the default configuration, then ``pesi.activate()`` in ``vcl_deliver``
may be the only modification to VCL that you need.
The invocation of ``pesi.activate()`` can of course be subject to
logic in VCL::
sub vcl_deliver {
# Use parallel ESI only if the request header X-PESI is present.
if (req.http.X-PESI) {
pesi.activate();
}
}
But see below for restrictions on the use of ``pesi.activate()``.
All of the computing resources used by the VDP -- threads, storage,
workspace, locks, and so on -- can be configured, either with Varnish
runtime parameters or configuration settings made available by the
VDP. And their usage can be monitored with Varnish statistics. So you
can limit resource usage, and use monitoring tools such as
`varnishstat(1)`_ to ensure efficient parallel ESI processing. For
details see `CONFIGURATION AND MONITORING`_ below.
$Function VOID pool(INT min=10, INT max=100, DURATION max_age=10)
$Function VOID activate()
Configure the memory pool used by the VDP for internal variable-sized
data structures.
Enable parallel ESI processing for the client response.
``activate()`` MUST be called in ``vcl_deliver`` only. If it is called
in any other VCL subroutine, VCL failure is invoked (see `ERRORS`_
below for details).
If ``activate()`` is called on *any* ESI level (any depth of include
nesting), then it MUST be called on *all* levels of the response. If
``activate()`` is invoked at some ESI levels but not others, then the
results are undefined, and will very likely lead to a Varnish panic.
Typically it suffices to simply call ``activate()`` in
``vcl_deliver``, since the code in ``vcl_deliver`` is executed at
every ESI level. It is also safe, for instance, to call ``activate()``
only if a request header is present, as in the example shown above;
since the same request headers are set for every ESI subrequest, the
result is the same at every ESI level. But that should *not* be done
if you have logic that unsets the header at some ESI levels but not at
others. Under no circumstances should the invocation of ``activate()``
depend on the value of ``req.esi_level``, or of ``req.url`` (since
URLs are different at different ESI levels).
See the documentation of ``set()`` below for a way to choose serial
ESI processing for all of the includes in the response at the current
ESI level. Even then, ``activate()`` must be called in ``vcl_deliver``
in addition to ``set()``.
XXX: test the following:
As with standard Varnish, ESI processing can be selectively disabled
for a client response, by setting ``resp.do_esi`` to ``false`` in VCL
since version 4.1, or setting ``req.esi`` to ``false`` in VCL 4.0 (see
`vcl(7)`_). The requirement remains: if ESI processing is enabled and
``activate()`` is called at any ESI level, then both must happen at
all levels.
``activate()`` has the effect of setting the VCL string variable
``resp.filters``, which is a whitespace-separated list of the names of
delivery processors to be applied to the client response (see
`vcl(7)`_). It configures the correct list of filters for the current
response, analogous to the default filter settings in Varnish when
sequential ESI is in use. These include the ``gunzip`` VDP for
uncompressed responses, and ``range`` for responses to range
requests. ``activate()`` checks the conditions for which the VDPs are
required, and arranges them in the correct order.
It is possible to manually set or change ``resp.filters`` to enable
parallel ESI, instead of calling ``activate()``, but that is only
advised to experts. If you do so, use the string ``pesi`` for this
VDP, and do *not* include ``esi``, for Varnish's standard ESI VDP, in
the same list with ``pesi``. As with the ``activate()`` call -- if
``pesi`` appears in ``resp.filters`` for a response at *any* ESI
level, it MUST be in ``resp.filters`` at *all* ESI levels.
Notice that all VCL code affecting ESI (such as setting
``resp.do_esi``), gzip (such as changes to
``req.http.Accept-Encoding``) or range processing (such as changes
``req.http.Range``) must execute before this function is called to
have an effect.
$Function VOID activate()
Example::
To be called from vcl_deliver {} only, must be called on all ESI
levels if called on any ESI level (unless you are a wizard).
vcl 4.1;
Configure the correct parallel ESI filters for this response analogous
to the default filter settings in varnish-cache for sequential ESI.
import pesi;
Manually setting/changing ``resp.filters`` is possible but only
advised to experts.
sub vcl_recv {
# Disable gzipped responses by removing Accept-Encoding.
unset req.http.Accept-Encoding;
}
Notice that all settings affecting ESI (like ``resp.do_esi``), gzip
(like changing ``req.http.Accept-Encoding``) or Range processing (like
changing ``req.http.Range``) should happen before calling this
function to have an effect.
sub vcl_backend_response {
set beresp.do_esi = true;
}
If this function is called on any ESI level, it should be called on
all ESI levels. Failure to do so will likely cause panics caused by
missing error handling in varnish-cache. Notice that the VMOD has no
known way to check for this condition, so please address any blame
appropriately.
sub vcl_deliver {
# If the request header X-Debug-ESI is present, then disable ESI
# for the current response.
if (req.http.X-Debug-ESI) {
set resp.do_esi = false;
}
pesi.activate()
}
$Function VOID set(ENUM { serial, thread } parameter, [BOOL bool])
To be called from ``vcl_deliver {}`` only.
Set a configuration parameter for the VDP, which holds for the current
(sub)request, as documented below. The parameter to be set is
identified by the ENUM ``parameter``. Currently the parameters can
only be set with a boolean value in ``bool`` (but future versions of
this function may allow for setting other data types).
``set()`` MUST be called in ``vcl_deliver`` only; otherwise VCL
failure is invoked (see `ERRORS`_).
The parameters that can be set are currently ``serial`` and ``thread``:
``serial``
----------
Activates serial mode if ``bool`` is ``true``; default is ``false``.
Set per (sub)request parameters for pesi which are documented
below. Parameters may require one or more of the other additional
arguments as documented below. Failure to provide them triggers a VCL
error.
In serial mode, the ESI subrequests processed for includes in the
current response body are processed in serial, in the current thread.
In other words, all ESI subrequests at the next level will be
processed without requesting threads from the thread pool (which
potentially starts new threads, if necessary). This setting only
affects include processing at the current ESI level, not nested
includes at the next level.
* ``serial``, requires *bool* argument
It is strongly recommended to *not* use serial mode from ESI level 0
(the top level request received from a client), because the ESI level
0 thread can send available data to the client concurrently with other
parallel ESI threads.
Activates serial mode if *bool* is ``true``, default is ``false``.
Serial mode may sensibly be used to reduce overhead and the number of
threads required without relevant drawbacks:
In serial mode, no new threads will be started from this request, so
all ESI subrequests at the next level will be processed by the
current thread. In other words, the setting only affects include
processing for the current response body.
* at ESI level > 0
It is strongly recommended to _not_ use serial mode from ESI level 0
because the ESI level 0 thread can send available data to the client
concurrently to other parallel ESI threads.
* when the VCL author knows that all objects included by the current
request are cacheable, and thus are highly likely to lead to cache
hits.
Serial mode may sensibly be used to reduce overhead and the number
of threads required without relevant drawbacks
XXX ... Also see the discussion below for more detail.
- at esi_level > 0
``thread``
----------
- when the vcl author knows that all objects included by the current
request will be cacheable and thus highly likely cause cache hits.
Whether we always request a new thread for includes, default is
``true``.
Also see the discussion below for more detail.
* ``false``
* ``thread``, requires *bool* argument
Only use a new thread if immediately available, process the include
in the same thread otherwise.
Whether we always request a new thread for includes, default is
``true``.
* ``true``
- ``false``
Request a new thread, potentially waiting for one to become
available.
Only use a new thread if immediately available, process the
include in the same thread otherwise.
XXX move the longer discussion to a document dedicated to the subjects
of tuning, efficiency etc
- ``true``
For parallel ESI to work as efficiently as possible, it should
traverse the ESI tree *breadth first*, processing any ESI object
completely, with new threads scheduled for any includes
encountered. Completing processing of an ESI object allows for data
from the subtree (the ESI object and anything below) to be sent to the
client concurrently. As soon as ESI object processing is complete, the
respective thread will be returned to the thread pool and become
available for any other varnish task (except for the request for
esi_level 0, which _has_ to wait for completion of the entire ESI
request anyway and will send data to the client in the meantime).
Request a new thread, potentially waiting for one to become
available.
With this setting to ``true`` (the default), this is what happens
always, but a thread may not be immediately available if the thread
pool is not sufficiently sized for the current load and thus the
include request may have to be queued.
For parallel ESI to work as efficiently as possible, it should
traverse the ESI tree *breadth first*, processing any ESI object
completely, with new threads scheduled for any includes
encountered. Completing processing of an ESI object allows for data
from the subtree (the ESI object and anything below) to be sent to
the client concurrently. As soon as ESI object processing is
complete, the respective thread will be returned to the thread pool
and become available for any other varnish task (except for the
request for esi_level 0, which _has_ to wait for completion of the
entire ESI request anyway and will send data to the client in the
meantime).
With this setting to ``false``, include processing happens in the same
thread as if ``serial`` mode had been activated, but only in the case
where there is no new thread available. While this may sound like the
more sensible option at first, we did not make this the default for
the following reasons:
With this setting to ``true`` (the default), this is what happens
always, but a thread may not be immediately available if the thread
pool is not sufficiently sized for the current load and thus the
include request may have to be queued.
* Before completion of the ESI processing, the subtree below it is not
yet available for delivery to the client because additional VDPs
behind pesi cannot be called from a different thread.
With this setting to ``false``, include processing happens in the
same thread as if ``serial`` mode had been activated, but only in
the case where there is no new thread available. While this may
sound like the more sensible option at first, we did not make this
the default for the following reasons:
* While processing of the include may take an arbitrarily long time
(for example because it requires a lengthy backend fetch), we know
that the ESI object is fully available in the stevedore (and usually
in memory already) when we parse an include because streaming is not
supported for ESI. So we know that completing the processing of the
current ESI object will be quick, while descending into a subtree
may be take a long time.
- Before completion of the ESI processing, the subtree below it is
not yet available for delivery to the client because additional
VDPs behind pesi cannot be called from a different thread.
* Except for ESI level 0, the current thread will become available as
soon as ESI processing has completed.
- While processing of the include may take an arbitrarily long time
(for example because it requires a lengthy backend fetch), we know
that the ESI object is fully available in the stevedore (and
usually in memory already) when we parse an include because
streaming is not supported for ESI. So we know that completing the
processing of the current ESI object will be quick, while
descending into a subtree may be take a long time.
* The thread herder may breed new threads and other threads may
terminate, so queuing a thread momentarily is not a bad thing per
se.
- Except for ESI level 0, the current thread will become available
as soon as ESI processing has completed.
In short, keeping ``thread`` at the default ``true`` should be the
right option, the alternative exists just in case.
- The thread herder may breed new threads and other threads may
terminate, so queuing a thread momentarily is not a bad thing per
se.
Example::
In short, keeping ``thread`` at the default ``true`` should be the
right option, the alternative exists just in case.
# Activate serial mode at ESI level > 0, if we know that all includes
# in the response at this level lead to cacheable responses.
sub vcl_deliver {
pesi.activate();
if (req.esi_level > 0 && req.url == "/all/cacheable/includes") {
pesi.set(serial, true);
}
}
$Function VOID workspace_prealloc(BYTES min_free=4096, INT max_nodes=32)
Configure workspace pre-allocation of objects in variable-sized
internal data structures.
XXX ...
For each request, the VDP builds such a structure, whose size is
roughly proportional to the size of the ESI tree -- the conceptual
tree with the top-level response at the root, and its includes and all
of their nested includes as branches. The nodes in this structure have
a fixed size, but the number of nodes used by the VDP varies with the
size of the ESI tree.
The VDP pre-allocates a constant number of such nodes in client
workspace, and initially takes nodes from the pre-allocation. If more
are needed for larger ESI trees, they are obtained from a global
memory pool as described below. The use of pre-allocated nodes from
workspace is preferred, since it never requires new system memory
allocations (workspaces themselves are pre-allocated by Varnish), and
because they are local to each request, so locking is never required
to access them (but is required for the memory pool).
The pre-allocation contributes a fixed size to client workspace usage,
since the number of pre-allocated nodes is constant. So any adjustment
to Varnish's ``workspace_client`` parameter that may be necessary due
to the pre-allocation will be valid for all requests.
``workspace_prealloc()`` configures the pre-allocation. The default
values of its parameters are defaults used by the VDP; that is, the
configuration if ``workspace_prealloc()`` is never called.
The ``min_free`` parameter sets the minimum amount of space that the
pre-allocation will always leave free in client workspace; if the
targeted number of pre-allocated nodes would result in less free space
than ``min_free`` bytes in workspace, then fewer nodes are
allocated. This ensures that free workspace is always left over for
other VMODs, VCL usage, and so forth. ``min_free`` defaults to 4 KiB.
The ``max_nodes`` parameter sets the number of nodes to be allocated,
unless the limit imposed by ``min_free`` is exceeded; ``max_nodes``
defaults to 32. ``max_nodes`` MUST be >= 0; otherwise, VCL failure is
invoked (see `ERRORS`_). If ``max_nodes`` is set to 0, then no nodes
are pre-allocated; they are all taken from the memory pool described
below.
When ``workspace_prealloc()`` is called, its configuration becomes
effective immediately for all new requests processed by the VDP. The
configuration remains valid for all instances of VCL, for as long as
the VDP remains loaded; that is, until the last instance of VCL using
the VDP is discarded.
``workspace_prealloc()`` can be called in ``vcl_init`` to set the
configuration at VCL load time. But you can also write VCL that calls
the function when a request is received by Varnish, for example using
a special URL for system administrators. This is similar to using the
``param.set`` command for `varnishadm(1)`_ to change a Varnish
parameter at runtime. Such a request should be protected, for example
with an ACL and/or Basic Authentication, so that it can be invoked
only by admins. Remember that as soon as such a request is processed
and ``workspace_prealloc()`` is executed, the changed configuration is
globally valid.
Examples::
# Configure workspace pre-allocation at VCL load time.
sub vcl_init {
pesi.workspace_prealloc(min_free=8k, max_nodes=64);
}
# Change the configuration at runtime, when Varnish receives an
# admin request.
import pesi;
import std;
sub vcl_recv {
if (req.url ~ "^/admin/pesi_ws") {
# Reject the request with "403 Forbidden" unless the client
# IP matches an ACL for admin requests.
if (client.ip !~ admin_acl) {
return (synth(403));
}
# Set min_free from a GET parameter, if present.
if (req.url ~ "\bmin_free=\d+[kmgtp]?") {
# Extract the BYTES parameter.
set req.http.Tmp-Bytes
= regsub(req.url, "^.+\bmin_free=(\d+[kmgtp]?).*$", "\1");
pesi.workspace_prealloc(std.bytes(req.http.Tmp-Bytes));
}
# Set max_nodes from a GET parameter.
if (req.url ~ "\bmax_nodes=\d+") {
# Extract the INT parameter.
set req.http.Tmp-Nodes
= regsub(req.url, "^.+\bmax_nodes=(\d+).*$", "\1");
pesi.workspace_prealloc(max_nodes=std.integer(req.http.Tmp-Nodes));
}
# Return status 204 to indicate success.
return (synth(204));
}
}
$Function VOID pool(INT min=10, INT max=100, DURATION max_age=10)
Configure the memory pool used by the VDP for internal variable-sized
data structures, when more is needed than is provided by the client
workspace pre-allocation described above. The objects in the memory
pool are the nodes used in structures whose size is proportional to
the size of the ESI tree, as discussed above.
The VDP uses the same mechanism that Varnish uses for its memory
pools, and the configuration values have the same meaning and defaults
as the Varnish runtime parameters ``pool_req``, ``pool_sess`` and
``pool_vbo`` (see `varnishd(1)`_). ``min`` and ``max`` control the
size of the pool -- the number of pre-allocated nodes available for
allocation requests. ``max_age`` is the maximum lifetime for nodes in
the pool -- when there are no pending allocation requests, nodes in
the pool that are older than ``max_age`` are freed, down to the limit
imposed by ``min``.
The values of the parameters MUST fulfill the following requirements,
otherwise VCL failure is invoked (see `ERRORS`_):
* ``min`` and ``max`` MUST be both > 0.
* ``max`` MUST be >= ``min``.
* ``max_age`` MUST be >= 0s (and <= one million seconds).
Note that ``max`` is a soft limit. The memory pool satisfies all
allocation requests, even if ``max`` is execeeded when nodes are
returned to the pool. But the pool size will then be reduced to
``max``, without waiting for ``max_age`` to expire.
As with ``workspace_prealloc()``: when ``pool()`` is called, the
changed configuration immediately becomes valid (although it may take
some time for the memory pool to adjust to the new values). It remains
vaild for as long as the VDP is still loaded, unless ``pool()`` is
called again. ``pool()`` may be called in ``vcl_init`` to set a
configuration at VCL load time, but may also be called elsewhere in
VCL, for example to enable changing configurations at runtime using a
special "admin" request.
Examples::
# Configure the memory pool at VCL load time.
sub vcl_init {
pesi.pool(min=50, max=500, max_age=30s);
}
# Change the configuration at runtime, when Varnish receives an
# admin request.
import pesi;
import std;
sub vcl_recv {
if (req.url ~ "^/admin/pesi_pool") {
# Protect the call with an ACL, as in the example above.
if (client.ip !~ admin_acl) {
return (synth(403));
}
# Set max_age from a GET parameter.
if (req.url ~ "\bmax_age=\d+(\.\d+)?(ms|s|m|h|d|w|y)") {
# Extract the DURATION parameter.
set req.http.Tmp-Duration
= regsub(req.url,
"^.\bmax_age=(\d+(?:\.\d+)?(?:ms|s|m|h|d|w|y))+.*$",
"\1");
pesi.pool(max_age=std.duration(req.http.Tmp-Duration));
}
# Set min from a GET parameter.
if (req.url ~ "\bmin=\d+") {
# Extract the INT parameter.
set req.http.Tmp-Min = regsub(req.url, "^.+\bmin=(\d+).*$", "\1");
pesi.pool(min=std.integer(req.http.Tmp-Min));
}
# Extract max from a GET parameter, the same way as for min,
# not repeated here ...
# Status 204 indicates success.
return (synth(204));
}
}
$Function STRING version()
......@@ -161,13 +507,31 @@ Example::
std.log("Using VDP pesi version: " + pesi.version());
REQUIREMENTS
============
ERRORS
======
The VDP currently requires the Varnish master branch. ...
As documented above, VCL failure is invoked under some of the error
conditions for functions provided by the VDP. VCL failure has the same
results as if ``return(fail)`` is called from a VCL subroutine:
NOTES
=====
* If the failure occurs in ``vcl_init``, then the VCL load fails with
an error message.
* If the failure occurs in any other subroutine besides ``vcl_synth``,
then a ``VCL_Error`` message is written to the log, and control is
directed immediately to ``vcl_synth``, with ``resp.status`` set to
503 and ``resp.reason`` set to ``"VCL failed"``.
* If the failure occurs in ``vcl_synth``, then ``vcl_synth`` is
aborted, and the response line "503 VCL failed" is sent.
CONFIGURATION AND MONITORING
============================
XXX ...
LIMITATIONS
===========
XXX @geoff pls reword
......@@ -176,8 +540,14 @@ events and whether or not we use (partial) sequential delivery (for
example, when no threads are available), ReqAcct adds n x 8 to the net
data size.
REQUIREMENTS
============
.. _bdf2f63: https://github.com/varnishcache/varnish-cache/commit/bdf2f63946e5abd4379f6a86ec0a273b387a4e59
This version of the VDP requires the Varnish master branch since
commit `bdf2f63`_. See the source repository for versions that are
compatible with other versions of Varnish (since 6.2.0).
INSTALLATION
============
......@@ -189,6 +559,5 @@ SEE ALSO
* varnishd(1)
* vcl(7)
* VMOD source repository: https://code.uplex.de/uplex-varnish/libvdp-pesi
$Event event
src/vmod_pesi.c
View file @ 162f9ec9
......@@ -3,7 +3,7 @@
* All rights reserved
*
* Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
* Nils Goroll <nils.goroll@uplex.de>
* Nils Goroll <nils.goroll@uplex.de>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment