Skip to content

S
slash
Unverified Commit 57252829 authored by Nils Goroll's avatar Nils Goroll
Browse files
Options

Add generated documentation for web access

parent 58ec40f9
Show whitespace changes
Inline Side-by-side
Showing with 999 additions and 0 deletions
src/vmod_slash.man.rst 0 → 100644
View file @ 57252829
..
.. NB: This file is machine generated, DO NOT EDIT!
..
.. Edit ./vmod_slash.vcc and run make instead
..
.. role:: ref(emphasis)
==========
vmod_slash
==========
---------------------------------------------------------------------------------
Varnish-Cache SLASH/ stevedores (buddy, fellow) and loadmasters (storage routers)
---------------------------------------------------------------------------------
:Manual section: 3
SYNOPSIS
========
global storages
---------------
* Make ``vmod_slash`` available::
varnishd -E /path/to/vmod_slash.so
* Configure a buddy (memory) storage::
varnishd -s<name>=buddy,<size>[,<minpage>]
* Configure a fellow (persistent disk with memory cache) storage::
varnishd -s<name>=fellow,<path>,<disk_size>,<memory_size>[=<storage>],<objsize_hint>
vcl storage objects and methods
-------------------------------
.. parsed-literal::
import slash [as name] [from "path"]
new xbuddy = slash.buddy(BYTES size, BYTES minpage)
STRING xbuddy.tune([INT chunk_exponent], [BYTES chunk_bytes], [INT reserve_chunks], [INT cram])
STEVEDORE xbuddy.storage()
VOID xbuddy.as_transient()
new xfellow = slash.fellow(STRING filename, BYTES dsksize, BYTES memsize, BYTES objsize_hint, BOOL delete)
STRING xfellow.tune([INT logbuffer_size], [DURATION logbuffer_flush_interval], [REAL log_rewrite_ratio], [INT chunk_exponent], [BYTES chunk_bytes], [INT dsk_reserve_chunks], [INT mem_reserve_chunks], [BYTES objsize_hint], [INT cram], [INT readahead], [BYTES discard_immediate], [INT io_batch_min], [INT io_batch_max], [ENUM hash_obj], [ENUM hash_log], [ENUM ioerr_obj], [ENUM ioerr_log], [ENUM allocerr_obj], [ENUM allocerr_log])
STEVEDORE xfellow.storage()
VOID xfellow.as_transient()
vcl functions
-------------
.. parsed-literal::
import slash [as name] [from "path"]
VOID as_transient(STEVEDORE)
STRING tune_buddy(STEVEDORE storage, [INT chunk_exponent], [BYTES chunk_bytes], [INT reserve_chunks], [INT cram])
STRING tune_fellow(STEVEDORE storage, [INT logbuffer_size], [DURATION logbuffer_flush_interval], [REAL log_rewrite_ratio], [INT chunk_exponent], [BYTES chunk_bytes], [INT dsk_reserve_chunks], [INT mem_reserve_chunks], [BYTES objsize_hint], [INT cram], [INT readahead], [BYTES discard_immediate], [INT io_batch_min], [INT io_batch_max], [ENUM hash_obj], [ENUM hash_log], [ENUM ioerr_obj], [ENUM ioerr_log], [ENUM allocerr_obj], [ENUM allocerr_log])
vcl loadmasters (storage routers)
---------------------------------
.. parsed-literal::
import slash [as name] [from "path"]
new xloadmaster_rr = slash.loadmaster_rr()
VOID xloadmaster_rr.add_storage(STEVEDORE)
STEVEDORE xloadmaster_rr.storage()
EXAMPLES
========
* Configure a global buddy (memory only) storage of 1 GB named ``mem``::
varnishd -E /path/to/libvmod_slash.so \
-s mem=buddy,1g
Use this storage with VCL code like this::
sub vcl_backend_response {
set beresp.storage = storage.mem;
}
sub vcl_backend_error {
set beresp.storage = storage.mem;
}
# ... more of your own VCL code
* Configure two global fellow (persistent, disk-backed) storages,
* one named ``fast`` of 1TB on a raw device
``/dev/mapper/ssd-volume`` using 100GB memory cache with an
expected object size of 10MB, and
* one named ``slow`` of 10TB on a file ``/hugefs/varnish-storage``,
which shares the memory cache with the ``fast`` storage and also
has the same expected object size::
varnishd -E /path/to/libvmod_slash.so \
-s fast=fellow,/dev/mapper/ssd-volume,1TB,100GB,10MB \
-s slow=fellow,/hugefs/varnish-storage,10TB,100GB=fast,10MB
Use these storages with VCL code, where responses to requests on
paths beginning with ``/archive/`` go to the ``slow`` storage::
sub vcl_backend_response {
if (bereq.url ~ "^/archive/") {
set beresp.storage = storage.slow;
}
else {
set beresp.storage = storage.fast;
}
}
* Configure a round-robin storage router in VCL::
# assumes that storages A .. C have been defined globaly
sub vcl_init {
new storageX = slash.loadmaster_rr();
storageX.add_storage(storage.A);
storageX.add_storage(storage.B);
storageX.add_storage(storage.C);
}
and use it::
sub vcl_backend_response {
set beresp.storage = rr.storage();
}
DESCRIPTION
===========
.. _buddy_memory_allocator: https://en.wikipedia.org/wiki/Buddy_memory_allocation
.. _README.rst: https://code.uplex.de/uplex-varnish/slash/blob/master/README.rst
.. _INSTALL.rst: https://code.uplex.de/uplex-varnish/slash/blob/master/INSTALL.rst
This module can be used both as a varnish extension (VEXT) and a
VCL module (VMOD).
It provides the two storage engines `buddy` and `fellow`, which can be
configured at ``varnishd`` startup and, with limitations, from VCL.
The `buddy` storage engine is an advanced, high performance stevedore
with a fixed memory size based on a new `buddy_memory_allocator`_
implementation from first principles.
The `fellow` storage engine is an advanced, high performance, eventually
persistent, always consistent implementation based on the same
allocator as the buddy storage engine.
See `README.rst`_ for more details.
Installation instructions can be found in `INSTALL.rst`_.
STORAGE VEXT INTERFACES
=======================
The two storage engines `buddy` and `fellow` should preferably be
configured globally by loading ``vmod_slash.so`` through the
``varnishd -E`` option and adding global storages with ``-s`` as shown
in `SYNOPSIS`_.
buddy
-----
For `buddy`, the ``-s`` parameter syntax is::
-s<name>=buddy,<size>[,<minpage>]
with
* *<name>* being a given name for the storage instance, which will
become available from vcl as ``storage.``\ *<name>*,
* *<size>* being a size expression like ``100m`` or ``5g`` for the
storage size to be configured,
* the optional *<minpage>* argument being a size expression for the
minimal allocation unit of the storage instance. See
`slash.buddy()`_ for details.
A global `buddy` storage can be tuned from VCL using
`slash.tune_buddy()`_ with ``storage.``\ *<name>* as the first
argument.
fellow
------
For `fellow`, the ``-s`` parameter syntax is::
-s<name>=fellow,<path>,<disk_size>,<memory_size>[=<storage>],<objsize_hint>
with
* *<name>* being a given name for the storage instance, which will
become available from vcl as ``storage.``\ *<name>*,
* *<path>* being the path to the storage file or device,
* *<disk_size>* being a size expression like ``100m`` or ``5g`` for
the storage size to be configured,
* *<memory_size>* being a size expression for the memory cache size to
be configured,
* optionally, *<storage>* being the name of a previously defined
fellow storage to share the memory cache with, and
* *<objsize_hint>* being a size expression for the expected average
object size with which the storage instance is being used.
See `slash.fellow()`_ for additional details.
A global `fellow` storage can be tuned from VCL using
`slash.tune_fellow()`_ with ``storage.``\ *<name>* as the first
argument.
Memory Cache Sharing
~~~~~~~~~~~~~~~~~~~~
When memory cache sharing with the ``<memory_size>[=<storage>]``
syntax is configured, *<memory_size>* is used for sanity checking
only. The actual memory size is always that of the referenced storage.
LRU with memory cache sharing is cooperative. Whenever memory is
needed by any storage, all storages using the shared cache are asked
to make room. Consequently, more frequently used storages are likely
to keep more of the shared memory cache.
STORAGE VMOD INTERFACES
=======================
.. _slash.buddy():
new xbuddy = slash.buddy(BYTES size, BYTES minpage=64)
------------------------------------------------------
Create or reference a buddy storage of size *size* with the given vmod
object name. The storage will remain in existence as long as
- any loaded VCL has an object by that name
- there are objects using it
The *minpage* argument can be used to define the smallest possible
allocation unit. The default and lowest possible *minpage* argument is
64B. The *minpage* argument will be rounded up to the next power of
two. Larger *minpage* arguments improve efficiency at the cost of
memory overhead.
The *size* argument will be rounded down to a multiple of the
(possibly rounded) *minpage* argument.
In addition to the configured memory, approximately 1 / ( *minpage* *
8) bytes are required for metadata (bitmaps). For the default
*minpage* of 64 Bytes, this amounts to approximately 0.2%.
This storage can *not* be used via ``storage.``\ *<name>*.
If the last vcl using this vmod is discarded before the storage is
empty, all its memory will remain allocated until a varnish restart.
.. _xbuddy.tune():
STRING xbuddy.tune([INT chunk_exponent], [BYTES chunk_bytes], [INT reserve_chunks], [INT cram])
-----------------------------------------------------------------------------------------------
::
STRING xbuddy.tune(
[INT chunk_exponent],
[BYTES chunk_bytes],
[INT reserve_chunks],
[INT cram]
)
)
Using the `xbuddy.tune()` method, the following parameters of the
buddy storage can be fine tuned:
* *chunk_exponent* / *chunk_bytes*
- unit: bytes as a power of two / bytes
- default: 20 / 1 MB
- minimum: 6 / 64 B
- maximum: 28 / 256 MB
*chunk_bytes* and *chunk_exponent* are alternative ways to configure
the chunk size. If *chunk_bytes* is used, the value is rounded up to
the next power of two and used as if *chunk_exponent* was used with
the 2-logarithm of that value.
Using both arguments at the same time triggers a VCL error.
*chunk_exponent* / *chunk_bytes* are very similar to the
``fetch_maxchunksize`` varnishd parameter, but can be configured per
storage instance: They specify the maximum contiguous memory region
which the storage will return for a single allocation request. The
default is the smaller of 1/16 the *size* of the storage and
256MB. The smallest possible value is 1/4 the *size* of the storage
and rounded down to the previous power of two.
* *reserve_chunks*
- unit: scalar
- default: 1
- minimum: 0
specifies a number of chunks to reserve in memory. The reserve is
used to immediately fulfill requests while LRU cache eviction is
running: When the cache is full, allocation requests need to wait
until LRU eviction has made room, and the reserve can help reduce
latencies in these situations at the expense of some memory
unavailable for caching.
* *cram*
- unit: powers of two
- default: 1
- minimum: -64
- maximum: 64
specifies to which extent the allocator should return regions
smaller than requested when it would need to wait for LRU to make
room.
Its unit is powers of two, valid values are -64 to 64, but sensible
values are much smaller.
* cram = 0: Always allocate the requested size
* cram != 0: Also return abs(*cram*) powers of two less than the
roundup of the requested size.
For example, with a *cram* value of 1 (the default) or -1, for 129
to 255 bytes requested, also 128 bytes could be returned.
For a *cram* value of 2 or -2, also 64 bytes could be returned for
129 to 255 bytes requested.
* For positive *cram* value, page splits are avoided - that is, if a
larger memory region would need to be split to fulfill all of the
request, but a memory region that is up to *cram* powers of two
smaller is available, the smaller memory region is returned.
* A negative *cram* value means that smaller memory regions are only
returned if the request could not be fulfilled otherwise.
Higher absolute *cram* values generally lead to higher fragmentation
in return for less unused space. Higher fragmentation is generally
bad for performance.
.. _xbuddy.storage():
STEVEDORE xbuddy.storage()
--------------------------
Return the the buddy storage. Can be used to set it for storing a
backend response::
set beresp.storage = mybuddy.storage();
.. _xbuddy.as_transient():
VOID xbuddy.as_transient()
--------------------------
Set this buddy storage as the transient storage.
Can only be called from ``vcl_init {}``.
.. _slash.tune_buddy():
STRING tune_buddy(STEVEDORE storage, [INT chunk_exponent], [BYTES chunk_bytes], [INT reserve_chunks], [INT cram])
-----------------------------------------------------------------------------------------------------------------
::
STRING tune_buddy(
STEVEDORE storage,
[INT chunk_exponent],
[BYTES chunk_bytes],
[INT reserve_chunks],
[INT cram]
)
)
Tune the given globally defined fellow storage, for all other
parameters see `xbuddy.tune()`.
.. _slash.fellow():
new xfellow = slash.fellow(STRING filename, BYTES dsksize, BYTES memsize, BYTES objsize_hint, BOOL delete)
----------------------------------------------------------------------------------------------------------
::
new xfellow = slash.fellow(
STRING filename,
BYTES dsksize,
BYTES memsize,
BYTES objsize_hint=262144,
BOOL delete=0
)
Create or reference a fellow storage on *filename* of size *dsksize*
with a memory cache of size *memsize*.
A VCL-defined fellow storage can not load persisted objects, so to
avoid accidentally emptying a storage, either the storage referenced
by *filename* must be empty, or the *delete* argument must be
``true``.
When a VCL-defined fellow storage goes out of scope because the last
VCL referencing it is discarded, all of its objects are removed from
the cache, but remain on disk. They can be loaded again by configuring
a global fellow storage. *Note* that this this kind of dynamic storage
removal is a new feature first introduced with `fellow` and might not
work perfectly yet.
On Linux, the memory cache will be allocated from huge pages, if
available and if *memsize* is larger than a huge page. *memsize* will
then be rounded up to a multiple of the respective huge page size.
*objsize_hint* (default 256KB) is used to sanity check *memsize* in
relation to *dsksize*. It should be set to a value lower than the
average object size (actual or expected). If *memsize* is configured
too low with respect to *dsksize* and *objsize_hint*, a higher value
will be used (which might fail of insufficient memory is available).
*delete* specifies if the storage is to be emptied.
.. _xfellow.tune():
STRING xfellow.tune([INT logbuffer_size], [DURATION logbuffer_flush_interval], [REAL log_rewrite_ratio], [INT chunk_exponent], [BYTES chunk_bytes], [INT dsk_reserve_chunks], [INT mem_reserve_chunks], [BYTES objsize_hint], [INT cram], [INT readahead], [BYTES discard_immediate], [INT io_batch_min], [INT io_batch_max], [ENUM hash_obj], [ENUM hash_log], [ENUM ioerr_obj], [ENUM ioerr_log], [ENUM allocerr_obj], [ENUM allocerr_log])
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
::
STRING xfellow.tune(
[INT logbuffer_size],
[DURATION logbuffer_flush_interval],
[REAL log_rewrite_ratio],
[INT chunk_exponent],
[BYTES chunk_bytes],
[INT dsk_reserve_chunks],
[INT mem_reserve_chunks],
[BYTES objsize_hint],
[INT cram],
[INT readahead],
[BYTES discard_immediate],
[INT io_batch_min],
[INT io_batch_max],
[ENUM {sha256, xxh32, xxh3_64, xxh3_128} hash_obj],
[ENUM {sha256, xxh32, xxh3_64, xxh3_128} hash_log],
[ENUM {panic, purge} ioerr_obj],
[ENUM {panic, fail} ioerr_log],
[ENUM {panic, purge} allocerr_obj],
[ENUM {panic, fail} allocerr_log]
)
Using the `xfellow.tune()` method, the following parameters of the
fellow storage can be fine tuned:
* *logbuffer_size*
- unit: scalar
- default: 24336
- minimum: 28
specifies an approximate number of objects to hold in a
logbuffer. Once a logbuffer is full, it is flushed if possible, so
this parameter constitutes an approximate upper bound on the number
of objects to hold unpersisted.
* *logbuffer_flush_interval*
- unit: duration
- default: 2.0s
- minimum: 0s
specifies the regular interval between regular logbuffer flushes,
persisting objects to disk. Logbuffer flushes can happen more often
if required.
* *log_rewrite_ratio*
- unit: ratio
- default: 0.5
- minimum: 0.001
specifies the minimum ratio of deleted by added objects (n_del / n_add)
in the log which triggers a log rewrite.
* *chunk_exponent* / *chunk_bytes*
- unit: bytes as a power of two / bytes
- default: 20 / 1 MB
- minimum: 12 / 4 KB
- maximum: 28 / 256 MB
*chunk_bytes* and *chunk_exponent* are alternative ways to configure
the chunk size. If *chunk_bytes* is used, the value is rounded up to
the next power of two and used as if *chunk_exponent* was used with
the 2-logarithm of that value.
Using both arguments at the same time triggers a VCL error.
See `xbuddy.tune()` for additional details.
* *dsk_reserve_chunks*
- unit: scalar
- default: 4
- minimum: 2
specifies a number of chunks to reserve on disk. The reserve is used
to fulfill storage requests when storage is otherwise full. Because
LRU cache eviction of disk objects is an expensive process involving
disk io, a reserve helps keeping response times for cache misses
low. It is also needed for the LRU algorithm itself, which, when the
fixed log space is full, might momentarily require additional space
before making room.
* *mem_reserve_chunks*
- unit: scalar
- default: 1
- minimum: 0
specifies a number of chunks to reserve in memory. The reserve is
used to provide memory for new objects or objects staged from disk
to memory when memory is otherwise full. It can help reduce
latencies in these situations at the expense of some memory
unavailable for caching.
* *objsize_hint*
- unit: bytes
- default: 256KB
- minimum: 4KB
specifies an estimate of the average object size in cache. It is
used to (re)size the fixed log space. The value should be chosen as
a safe *lower* bound of average object size.
* *cram* is documented in `xbuddy.tune()`_
* *readahead*
- unit: scalar
- default: 2
- minimum: 0
specifies how many additional segments of an object's body should be
staged into memory asynchronously before being required. This
parameter helps keeping response times low and throughput high for
objects which are not already present in the memory cache.
* *discard_immediate*
- unit: bytes
- default: 256KB
- minimum: 4KB
minimum size for which to attempt to issue immediate discards of
disk blocks to be freed.
To disable immediate discards, use a number higher than your storage
size. For most users, 42PB will work to disable.
The discard implementation attempts these methods in order:
- ``ioctl(x, BLKDISCARD, ...)``
- ``fallocate(x, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, ...)``
Methods are tried once and disabled upon failure, until a tune
operation is executed which re-enables discard.
If possible, discard commands are issued asynchronously, but they
need to be completed before disk space can be re-used, so discards
can impose additional latency.
Discard operations are skipped when a space deficit exists.
The potential advantage is improved performance and reduced wear on
flash storage.
See :ref:`fallocate(2)` and :ref:`blkdiscard(8)` which contains
related information, because there exists no man page for the
``BLKDISCARD`` :ref:`ioctl(2)`.
* *io_batch_min*, *io_batch_max*
- unit: I/O operations
- default: 8, 512
- minimum: 1
Minimum and maximum number of IO operations to batch within a single
submission to the kernel, where applicable.
Larger values save on system calls, but can increase latency.
* *hash_obj*, *hash_log*
- value: one of ``sha256``, ``xxh32``, ``xxh3_64``, ``xxh3_128``
- default: ``xxh3_64`` if xxhash has been compiled in, ``sha256`` otherwise
*hash_obj* specifies the hash algorithm to ensure data integrity of
objects and their data.
*hash_log* specifies the hash algorithm to ensure data integrity of
the log.
* *ioerr_obj*
*NOTE:* As of this release, this feature is not fully
implemented. IO errors may trigger ``panic`` mode even if another
mode is selected.
- value: ``panic`` or ``purge``
- default: ``panic``
*ioerr_obj* allows to select the action to taken when an IO error
is encountered while reading or writing object data or when a
checksum mismatch is found for object data:
- ``panic`` aborts varnish with a panic
- ``purge`` purges the object from the cache
With ``purge``, consider the following consequences:
* Read errors may lead to delivery of truncated object bodies and/or
other hard delivery errors such as early connection closure.
.. XXX implement .prefetch() from VCL to allow control over it
* Depending on whether or not the object's segment list is present
in RAM, storage may remain allocated until the next log rewrite.
* *ioerr_log*
*NOTE:* As of this release, this feature is not fully
implemented. IO errors may trigger ``panic`` mode even if another
mode is selected.
- value: ``panic`` or ``fail``
- default: ``panic``
*ioerr_log* allows to select the action to taken when an IO error
is encountered while reading or writing the log or when a
checksum mismatch is found for log data:
- ``panic`` aborts varnish with a panic
- ``fail`` causes all allocation requests to the stevedore to fail
(`xfellow.storage()`_ return ``NULL``)
* *allocerr_obj*
*NOTE:* As of this release, this feature is not fully
implemented. IO errors may trigger ``panic`` mode even if another
mode is selected.
- value: ``panic`` or ``purge``
- default: ``panic``
*allocerr_obj* allows to select the action to take when insufficient
memory or storage is available for reading or writing object data:
- ``panic`` aborts varnish with a panic
- ``purge`` purges the object from the cache
For ``purge``, depending on whether or not the object's segment list
is present in RAM, storage may remain allocated until a restart.
Because the fellow storage is designed to not fail allocations under
normal circumstances and instead wait for LRU to make room,
``panic`` is intended also for production use.
* *allocerr_log*
*NOTE:* As of this release, this feature is not fully
implemented. IO errors may trigger ``panic`` mode even if another
mode is selected.
- value: ``panic`` or ``fail``
- default: ``panic``
*allocerr_log* allows to select the action to taken when when insufficient
memory or storage is available for reading or writing the log:
- ``panic`` aborts varnish with a panic
- ``fail`` causes all allocation requests to the stevedore to fail
(`xfellow.storage()`_ return ``NULL``)
Because the fellow storage is designed to not fail allocations under
normal circumstances and instead wait for LRU to make room,
``panic`` is intended also for production use.
.. _xfellow.storage():
STEVEDORE xfellow.storage()
---------------------------
Return the the buddy storage. Can be used to set it for storing a
backend response::
set beresp.storage = myfellow.storage();
.. _xfellow.as_transient():
VOID xfellow.as_transient()
---------------------------
Set this fellow storage as the transient storage.
.. _slash.as_transient():
VOID as_transient(STEVEDORE)
----------------------------
Set this storage as the transient storage.
Can only be called from ``vcl_init {}``.
.. _slash.tune_fellow():
STRING tune_fellow(STEVEDORE storage, [INT logbuffer_size], [DURATION logbuffer_flush_interval], [REAL log_rewrite_ratio], [INT chunk_exponent], [BYTES chunk_bytes], [INT dsk_reserve_chunks], [INT mem_reserve_chunks], [BYTES objsize_hint], [INT cram], [INT readahead], [BYTES discard_immediate], [INT io_batch_min], [INT io_batch_max], [ENUM hash_obj], [ENUM hash_log], [ENUM ioerr_obj], [ENUM ioerr_log], [ENUM allocerr_obj], [ENUM allocerr_log])
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
::
STRING tune_fellow(
STEVEDORE storage,
[INT logbuffer_size],
[DURATION logbuffer_flush_interval],
[REAL log_rewrite_ratio],
[INT chunk_exponent],
[BYTES chunk_bytes],
[INT dsk_reserve_chunks],
[INT mem_reserve_chunks],
[BYTES objsize_hint],
[INT cram],
[INT readahead],
[BYTES discard_immediate],
[INT io_batch_min],
[INT io_batch_max],
[ENUM {sha256, xxh32, xxh3_64, xxh3_128} hash_obj],
[ENUM {sha256, xxh32, xxh3_64, xxh3_128} hash_log],
[ENUM {panic, purge} ioerr_obj],
[ENUM {panic, fail} ioerr_log],
[ENUM {panic, purge} allocerr_obj],
[ENUM {panic, fail} allocerr_log]
)
Tune the given globally defined fellow storage, for all other
parameters see `xfellow.tune()`.
STATISTICS / COUNTERS
=====================
`buddy` and `fellow` expose statistics and counters which can be
observed with VSC clients like :ref:`varnishstat(1)`.
The counter documentation is available through :ref:`varnishstat(1)`
and the :ref:`slash-counters(7)` man page.
FELLOW DIAGNOSTICS
==================
`fellow` writes diagnostic information about initialization, the
initial load and log rewrites to :ref:`vsl(7)`.
To extract the relevant information, query the log in raw mode for
lines with tag ``Storage`` and no vxid (``vxid == 0``), as for example
with :ref:`varnishlog(1)`::
varnishlog -t off -g raw -i Storage -q 'vxid == 0'
During startup, additional diagnostic information is written to
standard error (stderr).
FELLOW CACHE LOADING
====================
Upon :ref:`varnishd(1)` startup with a globally configured `fellow`,
the log is read to recreate all persisted object sparsely as *vampire
objects* (that is, only minimal metadata is added to the cache).
Until `fellow` is fully initialized and the cache loaded, the varnish
instance remains unusable. This is because free space on the storage
is implicitly defined as not being used by any object. Further
improvements of the initial load time might be possible, though.
Cache loading can be observed using the folowing methods:
* By observing the :ref:`varnish-counters(7)` ``MAIN.n_objectcore``
and ``MAIN.n_vampireobject``. Note that to see the latter with
:ref:`varnishstat(1)` in interactive mode, the ``v`` key needs to be
pressed to select at least ``DIAG`` verbosity.
* By running :ref:`slashmap(1)` to observe how the disk space shown as
allocated fills up as the log is processed.
* By running the :ref:`varnishlog(1)` command given under `FELLOW
DIAGNOSTICS`_. It will continiously display updates on the number of
loaded objects like in this example::
...
0 Storage - fellow fellow: resurrected 8231700
0 Storage - fellow fellow: resurrected 8416700
...
When loading is complete, a summary will be shown like::
0 Storage - ... done: 53.485482s
0 Storage - fellow fellow: first load t1 = 0.154415
0 Storage - fellow fellow: 10010027 resurrected in 53.485892s (187160.720410/s), 431 already expired
FELLOW PLANNED BUT MISSING FEATURES
===================================
The following features are planned for implementation:
* More Statistics
* I/O Timeouts
* Support some successor of xkey (additional cache keys)
* Further improve cache loading speed
Please see `README.rst`_ for how to support the project in order to
get them implemented.
FELLOW PLANNED BUT ONLY PARTIALLY IMPLEMENTED FEATURES
======================================================
* Error handling and I/O error handling in particular is
incomplete. Assertion failures might be triggered where errors
could, in principle, be handled gracefully.
FELLOW KNOWN ISSUES
===================
* On Linux with ``io_uring``, by default, `fellow` registers all of
the memory cache as buffers using
:ref:`io_uring_register_buffers(3)` to achieve optimal performance
at runtime, if supported by the system.
Buffer registrations happen in multiple threads in parallel, one for
each io ring.
During initialization, however, this takes considerable amounts of
time for larger memory caches.
If this is an issue for you, please ask the kernel developers to
make buffer registration more efficient.
If you are willing to sacrifice runtime performance for a faster
startup, :ref:`varnishd(1)` can be started with the environment
variable ``slash_fellow_options`` set to contain
``skip-uring-register-buffers``.
If the variable contains ``sync-uring-register-buffers``, buffer
registration is forced to be done as serial, syncronous registration
operations.
* If initialization takes longer than the :ref:`varnishd(1)`
``cli_timeout``, it might not come up properly in that it does not
accept connections after finishing storage initialization. To avoid
such issues, set the parameter to a value longer than the startup
time, e.g. add to the :ref:`varnishd(1)` arguments::
-p cli_timeout=3600
* Because `fellow` might use varnish threads for some or all IOs and
those might be issued in huge bursts, the infamous *Worker Pool
Queue does not move* panic is more likely to occur when there is
otherwise no problem. It is thus recommended to set the
``thread_pool_watchdog`` parameter to a value significantly higher
than the default, e.g. by adding to the :ref:`varnishd(1)`
arguments::
-p 'thread_pool_watchdog=600'
FELLOW ADDITIONAL TUNING KNOBS
==============================
These options are not expected to ever require tuning, but exist just
in case:
* The environment variable ``fellow_log_io_entries`` can be used to
set the log io ring size, which is configured when the storage
engine starts. The default is 1024, values below 128 are not
generally recommended, and for higher values, the stack size will
likely need to be adjusted or stack overflows might occur.
Three leased log IO rings are used for reading and writing log data.
* Likewise, the environment variable ``fellow_cache_io_entries`` can
be used to set the cache io ring size.
A single shared IO ring is used for reading and writing object data.
Both options affect all IO backends, but in different ways:
* For io_uring, they set the submission and completion ring sizes,
which, simply put, define the maximum number of IOs to be handled
through a single system call. With io_uring, this specifically does
not affect the maximum number of IOs "in flight".
* For the other IO backends, they define the maximum number of IOs "in
flight".
LOADMASTER VMOD INTERFACES
==========================
We call storage routers loadmasters because they coordinate
stevedores.
.. _slash.loadmaster_rr():
new xloadmaster_rr = slash.loadmaster_rr()
------------------------------------------
Defines a round-robin loadmaster which allocates objects from
associated storages in turn. If the preferred, round-robin selected
storage fails, other storages are tried in order until one succeeds,
if at all.
For performance reasons, the implementation does not serialize
requests, so concurrent requests might receive object allocations from
the same backend momentarily. This effect should average out.
.. _xloadmaster_rr.add_storage():
VOID xloadmaster_rr.add_storage(STEVEDORE)
------------------------------------------
Add a storage to the loadmaster. May only be called from ``vcl_init
{}``.
.. _xloadmaster_rr.storage():
STEVEDORE xloadmaster_rr.storage()
----------------------------------
Return a reference to the loadmaster, mostly for use with ``set
beresp.backend = loadmaster.storage()``.
.. _slash.loadmaster_hash():
new xloadmaster_hash = slash.loadmaster_hash()
----------------------------------------------
Defines a hashing loadmaster which selects the preferred storage by
taking the first four bytes of the object's hash key (basically
``req.hash``) modulo the number of storages defined.
As with `slash.loadmaster_rr()`_, if the preferred storage fails,
other storages are tried in order until one succeeds, if at all.
.. _xloadmaster_hash.add_storage():
VOID xloadmaster_hash.add_storage(STEVEDORE)
--------------------------------------------
Same as `xloadmaster_rr.add_storage()`_.
.. _xloadmaster_hash.storage():
STEVEDORE xloadmaster_hash.storage()
------------------------------------
Same as `xloadmaster_rr.storage()`_.
SEE ALSO
========
:ref:`vcl(7)`, :ref:`varnishd(1)`
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment