Skip to content

L
libvmod-selector
Commit ba5da02a authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Start updating the docs.

parent e75ef8bf
Hide whitespace changes
Inline Side-by-side
Showing with 98 additions and 58 deletions
README.rst
View file @ ba5da02a
......@@ -10,9 +10,9 @@
vmod_selector
=============
------------------------------------------------------------------------------------
Varnish Module for matching strings associated with backends, regexen and other data
------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------
Varnish Module for matching fixed strings, and mappping strings to backends, regexen and other data
---------------------------------------------------------------------------------------------------
:Manual section: 3
......@@ -28,6 +28,7 @@ SYNOPSIS
new <obj> = selector.set([BOOL case_sensitive])
VOID <obj>.add(STRING [, STRING string] [, STRING regex]
[, BACKEND backend] [, INT integer])
VOID <obj>.compile()
VOID <obj>.create_stats()
# Matching
......@@ -57,8 +58,9 @@ DESCRIPTION
.. _VMOD re2: https://code.uplex.de/uplex-varnish/libvmod-re2
Varnish Module (VMOD) for matching strings against sets of fixed
strings, and optionally associating the matched string with a backend,
another string, an integer, or a regular expression.
strings. A VMOD object may also function as an associative array,
mapping the matched string to one or more of a backend, another
string, an integer, or a regular expression.
The VMOD is intended to support a variety of use cases that are
typical for VCL deployments, such as:
......@@ -101,6 +103,7 @@ lines. For example::
url_prefix.add("/foo/", backend=foo_backend);
url_prefix.add("/bar/", backend=bar_backend);
url_prefix.add("/baz/", backend=baz_backend);
url_prefix.compile();
# For requests with these Host headers, generate a redirect
# response, using the associated string to construct the
......@@ -110,6 +113,7 @@ lines. For example::
redirect.add("www.bar.com", string="/bar", integer=302);
redirect.add("www.baz.com", string="/baz", integer=303);
redirect.add("www.quux.com", string="/quux", integer=307);
redirect.compile();
# Requests for these URLs are rewritten by altering the
# query string, using the associated regex for a
......@@ -119,6 +123,7 @@ lines. For example::
rewrite.add("/alpha/beta", regex="(\?.*)\bfoo=[^&]+&?(.*)$");
rewrite.add("/delta/gamma", regex="(\?.*)\bbar=[^&]+&?(.*)$");
rewrite.add("/epsilon/zeta", regex="(\?.*)\bbaz=[^&]+&?(.*)$");
rewrite.compile();
}
sub vcl_recv {
......@@ -173,18 +178,11 @@ lines. For example::
Matches with the ``.match()`` and ``.hasprefix()`` methods scale well
as the number of strings in the set increases. The time needed for
``.match()`` is proportional to the length of the string to be
matched; for ``.hasprefix()``, it is proportional to the length of the
longest string in the set that forms a prefix of the string to be
matched. In both cases, the time for execution is independent of the
number of strings in the set, and is predictable and fast for large
sets of strings.
In the case of non-matches, the search for a match stops as soon as it
encounters a character in the string such that no string in the set
can match. Thus if a set contains the strings ``foo``, ``bar`` and
``baz``, then the search stops after the first character if it is
neither of ``f`` or ``b``.
``.match()`` is formally O(1); for ``.hasprefix()`` it is O(*p*),
where *p* is the length of the longest string in the set that forms a
prefix of the string to be matched. In both cases, the number of
execution steps is independent of the number of strings in the set,
and is predictable and fast for large sets of strings.
When new strings are added to a set (with new ``.add()`` statements in
``vcl_init``), the VCL code that executes the various operations
......@@ -199,6 +197,14 @@ using the regex saved with the ``regex`` parameter. But if you need
to match against sets of patterns, consider using the set interface
of `VMOD re2`_, which provides techniques similar to the present VMOD.
The limited expressiveness of strings to be matched means that this
VMOD can apply fast algorithms. While regexen and a VMOD like re2 can
be used to match fixed strings and prefixes, the matching operations
of VMOD selector are orders of magnitude faster. That in turn
contributes to scalability by consuming less CPU time for matches. So
if your use case allows matches against strings without patterns,
prefer the use of this VMOD.
Selecting matched elements of a set
-----------------------------------
......@@ -240,11 +246,10 @@ method, starting from 1. In all of the following, the ``n`` and
* If ``n`` >= 1, then the ``n``-th element of the set is chosen, and
the ``select`` parameter has no effect. A method with ``n`` >= 1 can
be called in any context, and does not depend on prior match
operations.
operations. This is essentially a lookup by index.
* If ``n`` is greater than the number of elements in the set, the
method fails, with an error message written to the Varnish
log. See `ERRORS`_ below for details about method failure.
method invokes VCL failure (see `ERRORS`_).
* If ``n`` <= 0, then the ``select`` parameter is used to choose an
element based on the most recent ``.match()`` or ``.hasprefix()``
......@@ -260,7 +265,7 @@ method, starting from 1. In all of the following, the ``n`` and
* If ``n`` <= 0 and neither of ``.match()`` or ``.hasprefix()`` has
been called for the same set object in the same task scope, or if
the most recent call resulted in a failed match, then the method
fails.
invokes VCL failure.
* When ``n`` <= 0 after a successful ``.match()`` call, then for any
value of ``select``, the element chosen is the one that matched.
......@@ -269,16 +274,16 @@ method, starting from 1. In all of the following, the ``n`` and
value of ``select`` determines the element chosen, as follows:
* ``UNIQUE`` (default): if exactly one element of the set matched,
choose that element. The method fails in this case if more than
one element matched.
choose that element. The method invokes VCL failure in this case
if more than one element matched.
Since the defaults for ``n`` and ``select`` are 0 and ``UNIQUE``,
``select=UNIQUE`` is in effect if both parameters are left out of
the method call.
* ``EXACT``: if one of the elements in the set matched exactly
(even if other prefixes in the set matched as well), choose
that element. The method fails if there was no exact match.
* ``EXACT``: if one of the elements in the set matched exactly (even
if other prefixes in the set matched as well), choose that
element. VCL failure is invoked if there was no exact match.
Thus if a prefix match for ``/foo/bar`` is run against a set
containing ``/foo`` and ``/foo/bar``, the latter element is chosen
......@@ -331,19 +336,34 @@ new xset = selector.set(BOOL case_sensitive, BOOL allow_overlaps)
BOOL allow_overlaps=1
)
Create a set object. When ``case_sensitive`` is ``false``, matches
using the ``.match()`` and ``.hasprefix()`` methods are
case-insensitive. By default, ``case_sensitive`` is ``true``.
Create a set object.
Example::
When ``case_sensitive`` is ``false``, matches using the ``.match()``
and ``.hasprefix()`` methods are case-insensitive. By default,
``case_sensitive`` is ``true``.
When ``allow_overlaps`` is ``false``, the VCL load fails if any string
added to the set is a prefix of another string in the set. This can be
used to ensure that methods using the ``select=UNIQUE`` enum will
always succeed after ``.has_prefix()`` matches (and to fail fast if
the restriction is not met). By default, ``allow_overlaps`` is
``true``.
Examples::
sub vcl_init {
# By default, matches are case-sensitive, and overlapping
# prefixes are permitted.
new myset = selector.set();
# ...
# For case-insensitive matching.
new caseless = selector.set(case_sensitive=false);
# ...
# Forbid overlapping prefixes.
new allunique = selector.set(allow_overlaps=false);
# ...
}
.. _xset.add():
......
src/vmod_selector.vcc
View file @ ba5da02a
......@@ -7,7 +7,7 @@
# See LICENSE
#
$Module selector 3 "Varnish Module for matching strings associated with backends, regexen and other data"
$Module selector 3 "Varnish Module for matching fixed strings, and mappping strings to backends, regexen and other data"
$ABI vrt
......@@ -24,6 +24,7 @@ SYNOPSIS
new <obj> = selector.set([BOOL case_sensitive])
VOID <obj>.add(STRING [, STRING string] [, STRING regex]
[, BACKEND backend] [, INT integer])
VOID <obj>.compile()
VOID <obj>.create_stats()
# Matching
......@@ -53,8 +54,9 @@ DESCRIPTION
.. _VMOD re2: https://code.uplex.de/uplex-varnish/libvmod-re2
Varnish Module (VMOD) for matching strings against sets of fixed
strings, and optionally associating the matched string with a backend,
another string, an integer, or a regular expression.
strings. A VMOD object may also function as an associative array,
mapping the matched string to one or more of a backend, another
string, an integer, or a regular expression.
The VMOD is intended to support a variety of use cases that are
typical for VCL deployments, such as:
......@@ -97,6 +99,7 @@ lines. For example::
url_prefix.add("/foo/", backend=foo_backend);
url_prefix.add("/bar/", backend=bar_backend);
url_prefix.add("/baz/", backend=baz_backend);
url_prefix.compile();
# For requests with these Host headers, generate a redirect
# response, using the associated string to construct the
......@@ -106,6 +109,7 @@ lines. For example::
redirect.add("www.bar.com", string="/bar", integer=302);
redirect.add("www.baz.com", string="/baz", integer=303);
redirect.add("www.quux.com", string="/quux", integer=307);
redirect.compile();
# Requests for these URLs are rewritten by altering the
# query string, using the associated regex for a
......@@ -115,6 +119,7 @@ lines. For example::
rewrite.add("/alpha/beta", regex="(\?.*)\bfoo=[^&]+&?(.*)$");
rewrite.add("/delta/gamma", regex="(\?.*)\bbar=[^&]+&?(.*)$");
rewrite.add("/epsilon/zeta", regex="(\?.*)\bbaz=[^&]+&?(.*)$");
rewrite.compile();
}
sub vcl_recv {
......@@ -169,18 +174,11 @@ lines. For example::
Matches with the ``.match()`` and ``.hasprefix()`` methods scale well
as the number of strings in the set increases. The time needed for
``.match()`` is proportional to the length of the string to be
matched; for ``.hasprefix()``, it is proportional to the length of the
longest string in the set that forms a prefix of the string to be
matched. In both cases, the time for execution is independent of the
number of strings in the set, and is predictable and fast for large
sets of strings.
In the case of non-matches, the search for a match stops as soon as it
encounters a character in the string such that no string in the set
can match. Thus if a set contains the strings ``foo``, ``bar`` and
``baz``, then the search stops after the first character if it is
neither of ``f`` or ``b``.
``.match()`` is formally O(1); for ``.hasprefix()`` it is O(*p*),
where *p* is the length of the longest string in the set that forms a
prefix of the string to be matched. In both cases, the number of
execution steps is independent of the number of strings in the set,
and is predictable and fast for large sets of strings.
When new strings are added to a set (with new ``.add()`` statements in
``vcl_init``), the VCL code that executes the various operations
......@@ -195,6 +193,14 @@ using the regex saved with the ``regex`` parameter. But if you need
to match against sets of patterns, consider using the set interface
of `VMOD re2`_, which provides techniques similar to the present VMOD.
The limited expressiveness of strings to be matched means that this
VMOD can apply fast algorithms. While regexen and a VMOD like re2 can
be used to match fixed strings and prefixes, the matching operations
of VMOD selector are orders of magnitude faster. That in turn
contributes to scalability by consuming less CPU time for matches. So
if your use case allows matches against strings without patterns,
prefer the use of this VMOD.
Selecting matched elements of a set
-----------------------------------
......@@ -236,11 +242,10 @@ method, starting from 1. In all of the following, the ``n`` and
* If ``n`` >= 1, then the ``n``-th element of the set is chosen, and
the ``select`` parameter has no effect. A method with ``n`` >= 1 can
be called in any context, and does not depend on prior match
operations.
operations. This is essentially a lookup by index.
* If ``n`` is greater than the number of elements in the set, the
method fails, with an error message written to the Varnish
log. See `ERRORS`_ below for details about method failure.
method invokes VCL failure (see `ERRORS`_).
* If ``n`` <= 0, then the ``select`` parameter is used to choose an
element based on the most recent ``.match()`` or ``.hasprefix()``
......@@ -256,7 +261,7 @@ method, starting from 1. In all of the following, the ``n`` and
* If ``n`` <= 0 and neither of ``.match()`` or ``.hasprefix()`` has
been called for the same set object in the same task scope, or if
the most recent call resulted in a failed match, then the method
fails.
invokes VCL failure.
* When ``n`` <= 0 after a successful ``.match()`` call, then for any
value of ``select``, the element chosen is the one that matched.
......@@ -265,16 +270,16 @@ method, starting from 1. In all of the following, the ``n`` and
value of ``select`` determines the element chosen, as follows:
* ``UNIQUE`` (default): if exactly one element of the set matched,
choose that element. The method fails in this case if more than
one element matched.
choose that element. The method invokes VCL failure in this case
if more than one element matched.
Since the defaults for ``n`` and ``select`` are 0 and ``UNIQUE``,
``select=UNIQUE`` is in effect if both parameters are left out of
the method call.
* ``EXACT``: if one of the elements in the set matched exactly
(even if other prefixes in the set matched as well), choose
that element. The method fails if there was no exact match.
* ``EXACT``: if one of the elements in the set matched exactly (even
if other prefixes in the set matched as well), choose that
element. VCL failure is invoked if there was no exact match.
Thus if a prefix match for ``/foo/bar`` is run against a set
containing ``/foo`` and ``/foo/bar``, the latter element is chosen
......@@ -317,19 +322,34 @@ or the longest match, and so on::
$Object set(BOOL case_sensitive=1, BOOL allow_overlaps=1)
Create a set object. When ``case_sensitive`` is ``false``, matches
using the ``.match()`` and ``.hasprefix()`` methods are
case-insensitive. By default, ``case_sensitive`` is ``true``.
Create a set object.
Example::
When ``case_sensitive`` is ``false``, matches using the ``.match()``
and ``.hasprefix()`` methods are case-insensitive. By default,
``case_sensitive`` is ``true``.
When ``allow_overlaps`` is ``false``, the VCL load fails if any string
added to the set is a prefix of another string in the set. This can be
used to ensure that methods using the ``select=UNIQUE`` enum will
always succeed after ``.has_prefix()`` matches (and to fail fast if
the restriction is not met). By default, ``allow_overlaps`` is
``true``.
Examples::
sub vcl_init {
# By default, matches are case-sensitive, and overlapping
# prefixes are permitted.
new myset = selector.set();
# ...
# For case-insensitive matching.
new caseless = selector.set(case_sensitive=false);
# ...
# Forbid overlapping prefixes.
new allunique = selector.set(allow_overlaps=false);
# ...
}
$Method VOID .add(STRING, [STRING string], [STRING regex], [BACKEND backend],
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment