Skip to content

L
libvmod-selector
Commit 477d5567 authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Document .suboutine() and .check_call().

parent 7c1df247
Hide whitespace changes
Inline Side-by-side
Showing with 344 additions and 52 deletions
README.rst
View file @ 477d5567
......@@ -28,7 +28,8 @@ SYNOPSIS
new <obj> = selector.set([BOOL case_sensitive]
[, BOOL allow_overlaps])
VOID <obj>.add(STRING [, STRING string] [, STRING regex]
[, BACKEND backend] [, INT integer] [, BOOL bool])
[, BACKEND backend] [, INT integer] [, BOOL bool]
[, SUB sub])
VOID <obj>.create_stats()
# Matching
......@@ -50,6 +51,7 @@ SYNOPSIS
[, ENUM select])
STRING <obj>.sub(STRING text, STRING rewrite [, BOOL all] [, INT n]
[, STRING element] [, ENUM select])
SUB <obj>.subroutine([INT n] [, STRING element] [, ENUM select])
# VMOD version
STRING selector.version()
......@@ -62,7 +64,8 @@ DESCRIPTION
Varnish Module (VMOD) for matching strings against sets of fixed
strings. A VMOD object may also function as an associative array,
mapping the matched string to one or more of a backend, another
string, an integer, or a regular expression.
string, an integer, or a regular expression. The string may also map
to a subroutine that can be invoked with ``call``.
The VMOD is intended to support a variety of use cases that are
typical for VCL deployments, such as:
......@@ -82,8 +85,13 @@ typical for VCL deployments, such as:
* Matching media types in the Content-Type header of a backend
response to determine if the content is compressible.
* Matching headers or URLs to conditionally execute code that depends
on features of the request or response.
* Accessing data by string match, as in an associative array, or by
numeric index, as in a standard array.
* Dispatching subroutine calls based on string matches.
* Executing conditional logic that depends on features of the request
or response that can be determined by matching headers or URLs.
Operations such as these are commonly implemented in native VCL with
an ``if-elsif-elsif`` sequence of string comparisons or regex matches.
......@@ -97,14 +105,26 @@ lines. For example::
import selector;
# Assume that you have defined these subroutines to execute logic
# in vcl_recv for URLs beginning with /foo/, /bar/ or /baz/.
sub foo { # ...
}
sub bar { # ...
}
sub baz { # ...
}
sub vcl_init {
# Requests for URLs with these prefixes will be sent to the
# associated backend.
# associated backend. In vcl_recv, the associated subroutine
# will be called.
new url_prefix = selector.set();
url_prefix.add("/foo/", backend=foo_backend);
url_prefix.add("/bar/", backend=bar_backend);
url_prefix.add("/baz/", backend=baz_backend);
url_prefix.add("/foo/", backend=foo_backend, sub=foo);
url_prefix.add("/bar/", backend=bar_backend, sub=bar);
url_prefix.add("/baz/", backend=baz_backend, sub=baz);
# For requests with these Host headers, generate a redirect
# response, using the associated string to construct the
......@@ -151,6 +171,12 @@ lines. For example::
if (rewrite.match(req.url)) {
set req.url = rewrite.sub(req.url, "\1\2");
}
# If the URL has a prefix in the url_prefix set, call the
# associated subroutine.
if (url_prefix.hasprefix(req.url)) {
call url_prefix.subroutine();
}
}
sub vcl_synth {
......@@ -228,11 +254,10 @@ same set::
Just calling ``.hasprefix()`` may be sufficient if all that matters is
whether a string has any prefix that appears in the set. But for some
uses it may be necessary to identify one matching element of the set;
this is done in particular for the ``.element()``, ``.backend()``,
``.string()``, ``.integer()``, ``.re_match()`` and ``.sub()`` methods,
which retrieve data associated with a specific set element. For such
cases, the method parameters ``INT n``, ``STRING element`` and ``ENUM
select`` are used to choose a matched element.
this is done in particular for the methods that retrieve data
associated with a specific set element. For such cases, the method
parameters ``INT n``, ``STRING element`` and ``ENUM select`` are used
to choose a matched element.
As indicated in the example, elements of a set are implicitly numbered
in the order in which they were added to the set using the ``.add()``
......@@ -393,7 +418,7 @@ and ``.hasprefix()`` methods are case-insensitive. By default,
When ``allow_overlaps`` is ``false``, the VCL load fails if any string
added to the set is a prefix of another string in the set. This can be
used to ensure that methods using the ``select=UNIQUE`` enum will
always succeed after ``.has_prefix()`` matches (and to fail fast if
always succeed after ``.hasprefix()`` matches (and to fail fast if
the restriction is not met). By default, ``allow_overlaps`` is
``true``.
......@@ -457,15 +482,31 @@ Add the given string to the set. As indicated above, elements added to
the set are implicitly numbered in the order in which they are added
with ``.add()``, starting with 1.
If values are set for the optional parameters ``string``, ``regex``,
``backend``, ``integer`` or ``bool``, then those values are associated
with this element, and can be retrieved with the ``.string()``,
``.backend()``, ``.integer()``, ``.bool()``, ``.re_match()`` or
``.sub()`` methods, as described below.
A regular expression in the ``regex`` parameter is compiled at VCL load
time. If the compile fails, then the VCL load fails with an error message.
Regular expressions are evaluated exactly as native regexen in VCL.
If values are set for any of the following optional parameters, then
those values are associated with this element, and can be retrieved
with the method shown in the second column. The retrieval methods are
documented below.
==================== ===========================
``.add()`` parameter Retrieval methods
-------------------- ---------------------------
``string`` ``.string()``
``regex`` ``.re_match()``, ``.sub()``
``backend`` ``.backend()``
``integer`` ``.integer()``
``bool`` ``.bool()``
``sub`` ``.subroutine()``
==================== ===========================
A regular expression in the ``regex`` parameter is compiled at VCL
load time. If the compile fails, then the VCL load fails with an error
message. Regular expressions are evaluated exactly as native regexen
in VCL.
A VCL subroutine specified by the ``sub`` parameter MUST be defined
*prior* to the definition of ``vcl_init`` in which ``.add()`` is
invoked. The VCL compiler does not support forward definitions for
this purpose.
``.add()`` invokes VCL failure if it is called in any subroutine
besides ``vcl_init``. The VCL load fails if:
......@@ -474,17 +515,24 @@ besides ``vcl_init``. The VCL load fails if:
* A regular expression in the ``regex`` parameter fails to compile.
* A subroutine specified by the ``sub`` parameter was not defined
previously in the VCL source.
* The deprecated ``.compile()`` method has already been called.
Example::
sub my_quux_sub {
set req.http.Quux = "xyzzy";
}
sub vcl_init {
new myset = selector.set();
myset.add("www.foo.com");
myset.add("www.bar.com", string="/bar");
myset.add("www.baz.com", string="/baz", backend=baz_backend);
myset.add("www.quux.com", string="/quux", backend=quux_backend,
regex="^/quux/([^/]+)/");
regex="^/quux/([^/]+)/", sub=my_quux_sub);
}
.. _xset.compile():
......@@ -1012,6 +1060,10 @@ Using the regular expression set by the ``regex`` parameter for the
element of the set indicated by ``n``, ``element`` and ``select``,
return the result of a substitution using ``str`` and ``sub``.
Note that the method name "sub" refers to string substitution. To
retrieve the subroutine set with the ``sub`` parameter in ``.add()``,
use the ``.subroutine()`` method documented below.
If ``all`` is ``false``, then return the result of replacing the first
portion of ``str`` that matches the regex with ``sub``. ``sub`` may
contain backreferences ``\0`` through ``\9``, to include captured
......@@ -1070,7 +1122,63 @@ SUB xset.subroutine(INT n, STRING element, ENUM select)
ENUM {UNIQUE, EXACT, FIRST, LAST, SHORTEST, LONGEST} select=UNIQUE
)
XXX ...
Returns the subroutine set by the ``sub`` parameter for the element of
the set indicated by ``n``, ``element`` and ``select``, according to
the rules given above. The subroutine may be invoked with VCL
``call``.
**Note**: you must ensure that the subroutine may invoked legally in
the context in which it is called. This means that:
* The subroutine may only refer to VCL elements that are legal in the
invocation context. For example, if the subroutine only refers to
headers in ``req.http.*``, then it may be called in ``vcl_recv``,
but not if it refers to any header in ``resp.http.*``. See
``vcl-var(7)`` for the specification of which VCL variables may be
used in which contexts.
* Recursive subroutine calls are not permitted in VCL. The subroutine
invocation may not appear anywhere in its own call stack.
For standard subroutine invocations with ``call``, the VCL compiler
checks these conditions and issues a compile-time error if either one
is violated. This is not possible with invocations using
``.subroutine()``; the error can only be determined at runtime. So it
is advisable to test the use of ``.subroutine()`` carefully before
using it in production. You can use the ``.check_call()`` method
described below to determine if the subroutine call is legal.
``.subroutine()`` invokes VCL failure if:
* The rules for ``n``, ``element`` and ``select`` indicate failure.
* No subroutine was set with the ``sub`` parameter in ``.add()``.
* The subroutine is invoked with ``call``, but the call is not legal
in the invocation context, for the reasons given above.
Example::
# Due to the use of resp.http.*, this subroutine may only be invoked
# in vcl_deliver or vcl_synth, as documented in vcl-var(7). Note
# that subroutine definitions must appear before vcl_init to
# permitted for the sub parameter in .add().
sub resp_sub {
set resp.http.Call-Me = "but only in deliver or synth";
}
sub vcl_init {
new myset = selector.set();
myset.add("/foo", sub=resp_sub);
myset.add("/foo/bar", sub=some_other_sub);
# ...
}
sub vcl_deliver {
if (resp_sub.hasprefix(req.url)) {
call resp_sub.subroutine(select=LONGEST);
}
}
.. _xset.check_call():
......@@ -1085,7 +1193,30 @@ BOOL xset.check_call(INT n, STRING element, ENUM select)
ENUM {UNIQUE, EXACT, FIRST, LAST, SHORTEST, LONGEST} select=UNIQUE
)
XXX ...
Returns ``true`` iff the subroutine returned by ``.subroutine()`` for
the element of the set indicated by ``n``, ``element`` and ``select``
may be invoked legally in the current context. The conditions for
legal invocation are documented for ``.subroutine()`` above.
``.check_call()`` never invokes VCL failure, but rather returns
``false`` under conditions for which the use of ``.subroutine()``
would invoke VCL failure, as described above. In that case, a message
is emitted to the Vanrish log using the ``Notice`` tag (the same
message that would appear with the ``VCL_Error`` tag if the subroutine
were called).
Example::
sub vcl_deliver {
if (resp_sub.hasprefix(req.url)) {
if (resp_sub.check_call(select=LONGEST)) {
call resp_sub.subroutine(select=LONGEST);
}
else {
call do_if_resp_sub_is_illegal;
}
}
}
.. _selector.version():
......@@ -1184,6 +1315,9 @@ If you need to check against possible failure conditions:
matched previously -- ``.matched()`` returns ``false`` if the
``element`` is not in the set.
* The ``.check_call()`` method may be used to avoid VCL failure if a
subroutine call using ``.subroutine()`` would be illegal.
See `LIMITATIONS`_ for considerations if you encounter conditions such
as workspace exhaustion.
......@@ -1204,6 +1338,16 @@ log with the tag ``Notice``, in this format::
... where ``<obj>`` is the object name and ``<method>`` is either
``match`` or ``hasprefix``.
If ``.check_call()`` returns ``false``, indicating that the use of
``.subroutine()`` would be illegal in that context, then the VMOD
emits a log meesage using ``Notice`` in this format::
vmod_selector: <obj>.check_call(): <errmsg>
... where ``<obj>`` is the object name and ``<errmsg>`` is the message
that would have been logged with ``VCL_Error`` if the subroutine were
invoked.
As noted above, VCL failure during request/response transactions
(after successful VCL load) is logged with an error message using the
``VCL_Error`` tag. These messages begin with the prefix ``vmod
......@@ -1260,6 +1404,8 @@ SEE ALSO
* vcl(7)
* vcl-var(7)
* varnishstat(1)
* varnishlog(1)
......
src/vmod_selector.vcc
View file @ 477d5567
......@@ -24,7 +24,8 @@ SYNOPSIS
new <obj> = selector.set([BOOL case_sensitive]
[, BOOL allow_overlaps])
VOID <obj>.add(STRING [, STRING string] [, STRING regex]
[, BACKEND backend] [, INT integer] [, BOOL bool])
[, BACKEND backend] [, INT integer] [, BOOL bool]
[, SUB sub])
VOID <obj>.create_stats()
# Matching
......@@ -46,6 +47,7 @@ SYNOPSIS
[, ENUM select])
STRING <obj>.sub(STRING text, STRING rewrite [, BOOL all] [, INT n]
[, STRING element] [, ENUM select])
SUB <obj>.subroutine([INT n] [, STRING element] [, ENUM select])
# VMOD version
STRING selector.version()
......@@ -58,7 +60,8 @@ DESCRIPTION
Varnish Module (VMOD) for matching strings against sets of fixed
strings. A VMOD object may also function as an associative array,
mapping the matched string to one or more of a backend, another
string, an integer, or a regular expression.
string, an integer, or a regular expression. The string may also map
to a subroutine that can be invoked with ``call``.
The VMOD is intended to support a variety of use cases that are
typical for VCL deployments, such as:
......@@ -78,8 +81,13 @@ typical for VCL deployments, such as:
* Matching media types in the Content-Type header of a backend
response to determine if the content is compressible.
* Matching headers or URLs to conditionally execute code that depends
on features of the request or response.
* Accessing data by string match, as in an associative array, or by
numeric index, as in a standard array.
* Dispatching subroutine calls based on string matches.
* Executing conditional logic that depends on features of the request
or response that can be determined by matching headers or URLs.
Operations such as these are commonly implemented in native VCL with
an ``if-elsif-elsif`` sequence of string comparisons or regex matches.
......@@ -93,14 +101,26 @@ lines. For example::
import selector;
# Assume that you have defined these subroutines to execute logic
# in vcl_recv for URLs beginning with /foo/, /bar/ or /baz/.
sub foo { # ...
}
sub bar { # ...
}
sub baz { # ...
}
sub vcl_init {
# Requests for URLs with these prefixes will be sent to the
# associated backend.
# associated backend. In vcl_recv, the associated subroutine
# will be called.
new url_prefix = selector.set();
url_prefix.add("/foo/", backend=foo_backend);
url_prefix.add("/bar/", backend=bar_backend);
url_prefix.add("/baz/", backend=baz_backend);
url_prefix.add("/foo/", backend=foo_backend, sub=foo);
url_prefix.add("/bar/", backend=bar_backend, sub=bar);
url_prefix.add("/baz/", backend=baz_backend, sub=baz);
# For requests with these Host headers, generate a redirect
# response, using the associated string to construct the
......@@ -147,6 +167,12 @@ lines. For example::
if (rewrite.match(req.url)) {
set req.url = rewrite.sub(req.url, "\1\2");
}
# If the URL has a prefix in the url_prefix set, call the
# associated subroutine.
if (url_prefix.hasprefix(req.url)) {
call url_prefix.subroutine();
}
}
sub vcl_synth {
......@@ -224,11 +250,10 @@ same set::
Just calling ``.hasprefix()`` may be sufficient if all that matters is
whether a string has any prefix that appears in the set. But for some
uses it may be necessary to identify one matching element of the set;
this is done in particular for the ``.element()``, ``.backend()``,
``.string()``, ``.integer()``, ``.re_match()`` and ``.sub()`` methods,
which retrieve data associated with a specific set element. For such
cases, the method parameters ``INT n``, ``STRING element`` and ``ENUM
select`` are used to choose a matched element.
this is done in particular for the methods that retrieve data
associated with a specific set element. For such cases, the method
parameters ``INT n``, ``STRING element`` and ``ENUM select`` are used
to choose a matched element.
As indicated in the example, elements of a set are implicitly numbered
in the order in which they were added to the set using the ``.add()``
......@@ -379,7 +404,7 @@ and ``.hasprefix()`` methods are case-insensitive. By default,
When ``allow_overlaps`` is ``false``, the VCL load fails if any string
added to the set is a prefix of another string in the set. This can be
used to ensure that methods using the ``select=UNIQUE`` enum will
always succeed after ``.has_prefix()`` matches (and to fail fast if
always succeed after ``.hasprefix()`` matches (and to fail fast if
the restriction is not met). By default, ``allow_overlaps`` is
``true``.
......@@ -429,15 +454,31 @@ Add the given string to the set. As indicated above, elements added to
the set are implicitly numbered in the order in which they are added
with ``.add()``, starting with 1.
If values are set for the optional parameters ``string``, ``regex``,
``backend``, ``integer`` or ``bool``, then those values are associated
with this element, and can be retrieved with the ``.string()``,
``.backend()``, ``.integer()``, ``.bool()``, ``.re_match()`` or
``.sub()`` methods, as described below.
A regular expression in the ``regex`` parameter is compiled at VCL load
time. If the compile fails, then the VCL load fails with an error message.
Regular expressions are evaluated exactly as native regexen in VCL.
If values are set for any of the following optional parameters, then
those values are associated with this element, and can be retrieved
with the method shown in the second column. The retrieval methods are
documented below.
==================== ===========================
``.add()`` parameter Retrieval methods
-------------------- ---------------------------
``string`` ``.string()``
``regex`` ``.re_match()``, ``.sub()``
``backend`` ``.backend()``
``integer`` ``.integer()``
``bool`` ``.bool()``
``sub`` ``.subroutine()``
==================== ===========================
A regular expression in the ``regex`` parameter is compiled at VCL
load time. If the compile fails, then the VCL load fails with an error
message. Regular expressions are evaluated exactly as native regexen
in VCL.
A VCL subroutine specified by the ``sub`` parameter MUST be defined
*prior* to the definition of ``vcl_init`` in which ``.add()`` is
invoked. The VCL compiler does not support forward definitions for
this purpose.
``.add()`` invokes VCL failure if it is called in any subroutine
besides ``vcl_init``. The VCL load fails if:
......@@ -446,17 +487,24 @@ besides ``vcl_init``. The VCL load fails if:
* A regular expression in the ``regex`` parameter fails to compile.
* A subroutine specified by the ``sub`` parameter was not defined
previously in the VCL source.
* The deprecated ``.compile()`` method has already been called.
Example::
sub my_quux_sub {
set req.http.Quux = "xyzzy";
}
sub vcl_init {
new myset = selector.set();
myset.add("www.foo.com");
myset.add("www.bar.com", string="/bar");
myset.add("www.baz.com", string="/baz", backend=baz_backend);
myset.add("www.quux.com", string="/quux", backend=quux_backend,
regex="^/quux/([^/]+)/");
regex="^/quux/([^/]+)/", sub=my_quux_sub);
}
$Method VOID .compile()
......@@ -886,6 +934,10 @@ Using the regular expression set by the ``regex`` parameter for the
element of the set indicated by ``n``, ``element`` and ``select``,
return the result of a substitution using ``str`` and ``sub``.
Note that the method name "sub" refers to string substitution. To
retrieve the subroutine set with the ``sub`` parameter in ``.add()``,
use the ``.subroutine()`` method documented below.
If ``all`` is ``false``, then return the result of replacing the first
portion of ``str`` that matches the regex with ``sub``. ``sub`` may
contain backreferences ``\0`` through ``\9``, to include captured
......@@ -935,13 +987,92 @@ $Method SUB .subroutine(INT n=0, STRING element=0,
ENUM {UNIQUE, EXACT, FIRST, LAST, SHORTEST, LONGEST}
select=UNIQUE)
XXX ...
Returns the subroutine set by the ``sub`` parameter for the element of
the set indicated by ``n``, ``element`` and ``select``, according to
the rules given above. The subroutine may be invoked with VCL
``call``.
**Note**: you must ensure that the subroutine may invoked legally in
the context in which it is called. This means that:
* The subroutine may only refer to VCL elements that are legal in the
invocation context. For example, if the subroutine only refers to
headers in ``req.http.*``, then it may be called in ``vcl_recv``,
but not if it refers to any header in ``resp.http.*``. See
``vcl-var(7)`` for the specification of which VCL variables may be
used in which contexts.
* Recursive subroutine calls are not permitted in VCL. The subroutine
invocation may not appear anywhere in its own call stack.
For standard subroutine invocations with ``call``, the VCL compiler
checks these conditions and issues a compile-time error if either one
is violated. This is not possible with invocations using
``.subroutine()``; the error can only be determined at runtime. So it
is advisable to test the use of ``.subroutine()`` carefully before
using it in production. You can use the ``.check_call()`` method
described below to determine if the subroutine call is legal.
``.subroutine()`` invokes VCL failure if:
* The rules for ``n``, ``element`` and ``select`` indicate failure.
* No subroutine was set with the ``sub`` parameter in ``.add()``.
* The subroutine is invoked with ``call``, but the call is not legal
in the invocation context, for the reasons given above.
Example::
# Due to the use of resp.http.*, this subroutine may only be invoked
# in vcl_deliver or vcl_synth, as documented in vcl-var(7). Note
# that subroutine definitions must appear before vcl_init to
# permitted for the sub parameter in .add().
sub resp_sub {
set resp.http.Call-Me = "but only in deliver or synth";
}
sub vcl_init {
new myset = selector.set();
myset.add("/foo", sub=resp_sub);
myset.add("/foo/bar", sub=some_other_sub);
# ...
}
sub vcl_deliver {
if (resp_sub.hasprefix(req.url)) {
call resp_sub.subroutine(select=LONGEST);
}
}
$Method BOOL .check_call(INT n=0, STRING element=0,
ENUM {UNIQUE, EXACT, FIRST, LAST, SHORTEST, LONGEST}
select=UNIQUE)
XXX ...
Returns ``true`` iff the subroutine returned by ``.subroutine()`` for
the element of the set indicated by ``n``, ``element`` and ``select``
may be invoked legally in the current context. The conditions for
legal invocation are documented for ``.subroutine()`` above.
``.check_call()`` never invokes VCL failure, but rather returns
``false`` under conditions for which the use of ``.subroutine()``
would invoke VCL failure, as described above. In that case, a message
is emitted to the Vanrish log using the ``Notice`` tag (the same
message that would appear with the ``VCL_Error`` tag if the subroutine
were called).
Example::
sub vcl_deliver {
if (resp_sub.hasprefix(req.url)) {
if (resp_sub.check_call(select=LONGEST)) {
call resp_sub.subroutine(select=LONGEST);
}
else {
call do_if_resp_sub_is_illegal;
}
}
}
$Function STRING version()
......@@ -1037,6 +1168,9 @@ If you need to check against possible failure conditions:
matched previously -- ``.matched()`` returns ``false`` if the
``element`` is not in the set.
* The ``.check_call()`` method may be used to avoid VCL failure if a
subroutine call using ``.subroutine()`` would be illegal.
See `LIMITATIONS`_ for considerations if you encounter conditions such
as workspace exhaustion.
......@@ -1057,6 +1191,16 @@ log with the tag ``Notice``, in this format::
... where ``<obj>`` is the object name and ``<method>`` is either
``match`` or ``hasprefix``.
If ``.check_call()`` returns ``false``, indicating that the use of
``.subroutine()`` would be illegal in that context, then the VMOD
emits a log meesage using ``Notice`` in this format::
vmod_selector: <obj>.check_call(): <errmsg>
... where ``<obj>`` is the object name and ``<errmsg>`` is the message
that would have been logged with ``VCL_Error`` if the subroutine were
invoked.
As noted above, VCL failure during request/response transactions
(after successful VCL load) is logged with an error message using the
``VCL_Error`` tag. These messages begin with the prefix ``vmod
......@@ -1113,6 +1257,8 @@ SEE ALSO
* vcl(7)
* vcl-var(7)
* varnishstat(1)
* varnishlog(1)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment