Skip to content

L
libvmod-re2
Commit 6d20f3a9 authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Document recent changes in the VMOD. 

- Changed error handling -- VCL failure for unrecoverable errors.
- .compile() is now unnecessary and deprecated.
- Addition of .subroutine() and .check_call().
- Update the REQUIREMENTS and compatibility statements.
- Fix some typos noticed along the way.
parent 04b3e9e0
Hide whitespace changes
Inline Side-by-side
Showing with 503 additions and 278 deletions
README.md
View file @ 6d20f3a9
...@@ -34,8 +34,7 @@ ...@@ -34,8 +34,7 @@
# set object interface # set object interface
new OBJECT = re2.set([ENUM anchor] [, <regex options>]) new OBJECT = re2.set([ENUM anchor] [, <regex options>])
VOID <obj>.add(STRING [, BOOL save] [, BOOL never_capture] [, STRING string] VOID <obj>.add(STRING [, BOOL save] [, BOOL never_capture] [, STRING string]
[, BACKEND backend] [, INT integer]) [, BACKEND backend] [, INT integer] [,SUB sub])
VOID <obj>.compile()
BOOL <obj>.match(STRING) BOOL <obj>.match(STRING)
INT <obj>.nmatches() INT <obj>.nmatches()
BOOL <obj>.matched(INT) BOOL <obj>.matched(INT)
...@@ -43,13 +42,15 @@ ...@@ -43,13 +42,15 @@
STRING <obj>.string([INT n,] [ENUM select]) STRING <obj>.string([INT n,] [ENUM select])
BACKEND <obj>.backend([INT n,] [ENUM select]) BACKEND <obj>.backend([INT n,] [ENUM select])
INT <obj>.integer([INT n] [, ENUM select]) INT <obj>.integer([INT n] [, ENUM select])
SUB <obj>.subroutine([INT n] [, ENUM select])
BOOL <obj>.check_call([INT n] [, ENUM select])
STRING <obj>.sub(STRING text, STRING rewrite [, INT n] STRING <obj>.sub(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
STRING <obj>.suball(STRING text, STRING rewrite [, INT n] STRING <obj>.suball(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
STRING <obj>.extract(STRING text, STRING rewrite [, INT n] STRING <obj>.extract(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
BOOL <obj>.saved([ENUM {REGEX, STR, BE, INT} which] [, INT n] BOOL <obj>.saved([ENUM {REGEX, STR, BE, INT, SUB} which] [, INT n]
[, ENUM select]) [, ENUM select])
VOID <obj>.hdr_filter(HTTP [, BOOL]) VOID <obj>.hdr_filter(HTTP [, BOOL])
...@@ -130,7 +131,6 @@ example: ...@@ -130,7 +131,6 @@ example:
myset.add("foo"); # Pattern 1 myset.add("foo"); # Pattern 1
myset.add("bar"); # Pattern 2 myset.add("bar"); # Pattern 2
myset.add("baz"); # Pattern 3 myset.add("baz"); # Pattern 3
myset.compile();
} }
`myset.match(<string>)` can now be used to match a string against the `myset.match(<string>)` can now be used to match a string against the
...@@ -182,14 +182,12 @@ method: ...@@ -182,14 +182,12 @@ method:
prefix.add("/bar", string="www.domain2.com"); prefix.add("/bar", string="www.domain2.com");
prefix.add("/baz", string="www.domain3.com"); prefix.add("/baz", string="www.domain3.com");
prefix.add("/quux", string="www.domain4.com"); prefix.add("/quux", string="www.domain4.com");
prefix.compile();
new appmatcher = re2.set(anchor=start); new appmatcher = re2.set(anchor=start);
appmatcher.add("/foo", backend=app1); appmatcher.add("/foo", backend=app1);
appmatcher.add("/bar", backend=app2); appmatcher.add("/bar", backend=app2);
appmatcher.add("/baz", backend=app3); appmatcher.add("/baz", backend=app3);
appmatcher.add("/quux", backend=app4); appmatcher.add("/quux", backend=app4);
appmatcher.compile();
} }
After a successful match, the string or backend associated with the After a successful match, the string or backend associated with the
...@@ -280,10 +278,11 @@ the resulting alternation. ...@@ -280,10 +278,11 @@ the resulting alternation.
- `never_capture` - `never_capture`
If true, parentheses in a pattern are interpreted as non-capturing, If true, parentheses in a pattern are interpreted as non-capturing,
and all invocations of the `backref` and `namedref` methods or and all invocations of the `backref` and `namedref` methods or
functions will fail, including `backref(0)` after a successful functions will lead to VCL faillure (see [ERRORS](#errors)),
match. Default is **false**, except for set objects, for which including `backref(0)` after a successful match. Default is
`never_capture` is always true (and cannot be changed), since back **false**, except for set objects, for which `never_capture` is
references are not possible with sets. always true (and cannot be changed), since back references are not
possible with sets.
- `case_sensitive` - `case_sensitive`
If true, matches are case-sensitive. A pattern can override this If true, matches are case-sensitive. A pattern can override this
...@@ -362,10 +361,10 @@ Example: ...@@ -362,10 +361,10 @@ Example:
Returns the nth captured subexpression from the most recent successful Returns the nth captured subexpression from the most recent successful
call of the `.match()` method for this object in the same client or call of the `.match()` method for this object in the same client or
backend, context, or a fallback string in case the capture fails. backend context, or a fallback string in case the capture fails. Backref
Backref 0 indicates the entire matched string. Thus this function 0 indicates the entire matched string. Thus this function behaves like
behaves like the `\n` in the native VCL functions `regsub` and the `\n` notation in the native VCL functions `regsub` and `regsuball`,
`regsuball`, and the `$1`, `$2` ... variables in Perl. and the `$1`, `$2` ... variables in Perl.
Since Varnish client and backend operations run in different threads, Since Varnish client and backend operations run in different threads,
`.backref()` can only refer back to a `.match()` call in the same `.backref()` can only refer back to a `.match()` call in the same
...@@ -375,15 +374,16 @@ subroutines -- the backend context -- refers back to a previous ...@@ -375,15 +374,16 @@ subroutines -- the backend context -- refers back to a previous
other VCL subroutines -- the client context -- refers back to a other VCL subroutines -- the client context -- refers back to a
`.match()` in the same client context. `.match()` in the same client context.
After unsuccessful matches, the `fallback` string is returned for any `.backref()` may return `fallback` after a successful match, if no
call to `.backref()`. The default value of `fallback` is `"**BACKREF
METHOD FAILED**"`. `.backref()` always fails after a failed match, even
if `.match()` had been called successfully before the failure.
`.backref()` may also return `fallback` after a successful match, if no
captured group in the matching string corresponds to the backref number. captured group in the matching string corresponds to the backref number.
For example, when the pattern `(a|(b))c` matches the string `ac`, there For example, when the pattern `(a|(b))c` matches the string `ac`, there
is no backref 2, since nothing matches `b` in the string. is no backref 2, since nothing matches `b` in the string. The default
value of `fallback` is `"**BACKREF METHOD FAILED**"`, but you may set
another value (such as the empty string).
After unsuccessful matches, `.backref()` invokes VCL failure (see
[ERRORS](#errors)). `.backref()` always fails after a failed match, even
if `.match()` had been called successfully before the failure.
The VCL infix operators `~` and `!~` do not affect this method, nor do The VCL infix operators `~` and `!~` do not affect this method, nor do
the functions `regsub` or `regsuball`. Nor is it affected by the matches the functions `regsub` or `regsuball`. Nor is it affected by the matches
...@@ -391,10 +391,9 @@ performed by any other method or function in this VMOD (such as the ...@@ -391,10 +391,9 @@ performed by any other method or function in this VMOD (such as the
`sub()`, `suball()` or `extract()` methods or functions, or the `set` `sub()`, `suball()` or `extract()` methods or functions, or the `set`
object's `.match()` method). object's `.match()` method).
`.backref()` fails, returning `fallback` and writing an error message to `.backref()` invokes VCL failure under the following conditions, even if
the Varnish log with the `VCL_Error` tag, under the following conditions a previous match was successful and a substring could have been captured
(even if a previous match was successful and a substring could have been (see [ERRORS](#errors)):
captured):
- The `fallback` string is undefined, for example if set from an unset - The `fallback` string is undefined, for example if set from an unset
header variable. header variable.
...@@ -422,7 +421,7 @@ Example: ...@@ -422,7 +421,7 @@ Example:
Returns the captured subexpression designated by `name` from the most Returns the captured subexpression designated by `name` from the most
recent successful call to `.match()` in the current context (client or recent successful call to `.match()` in the current context (client or
backend), or `fallback` in case of failure. backend).
Named capturing groups are written in RE2 as: `(?P<name>re)`. (Note that Named capturing groups are written in RE2 as: `(?P<name>re)`. (Note that
this syntax with `P`, inspired by Python, differs from the notation for this syntax with `P`, inspired by Python, differs from the notation for
...@@ -432,16 +431,14 @@ named capturing groups in PCRE.) Thus when `(?P<foo>.+)bar$` matches ...@@ -432,16 +431,14 @@ named capturing groups in PCRE.) Thus when `(?P<foo>.+)bar$` matches
Note that a named capturing group can also be referenced as a numbered Note that a named capturing group can also be referenced as a numbered
group. So in the previous example, `.backref(1)` also returns `baz`. group. So in the previous example, `.backref(1)` also returns `baz`.
`fallback` is returned when `.namedref()` is called after an `fallback` is returned when the named reference did not match. The
unsuccessful match. The default fallback is `"**NAMEDREF METHOD default fallback is `"**NAMEDREF METHOD FAILED**"`.
FAILED**"`.
Like `.backref()`, `.namedref()` is not affected by native VCL regex Like `.backref()`, `.namedref()` is not affected by native VCL regex
operations, nor by any other matches performed by methods or functions operations, nor by any other matches performed by methods or functions
of the VMOD, except for a prior `.match()` for the same object. of the VMOD, except for a prior `.match()` for the same object.
`.namedref()` fails, returning `fallback` and logging a `VCL_Error` `.namedref()` invokes VCL failure (see [ERRORS](#errors)) if:
message, if:
- The `fallback` string is undefined. - The `fallback` string is undefined.
- `name` is undefined or the empty string. - `name` is undefined or the empty string.
...@@ -480,8 +477,7 @@ matching text. This method corresponds to the VCL native function ...@@ -480,8 +477,7 @@ matching text. This method corresponds to the VCL native function
`fallback` is returned if the pattern does not match `text`. The default `fallback` is returned if the pattern does not match `text`. The default
fallback is `"**SUB METHOD FAILED**"`. fallback is `"**SUB METHOD FAILED**"`.
`.sub()` fails, returning `fallback` and logging a `VCL_Error` message, `.sub()` invokes VCL failure (see [ERRORS](#errors)) if:
if:
- Any of `text`, `rewrite` or `fallback` are undefined. - Any of `text`, `rewrite` or `fallback` are undefined.
- There is insufficient workspace for the rewritten string. - There is insufficient workspace for the rewritten string.
...@@ -600,8 +596,8 @@ Like the `regex.match()` method, return `true` if `pattern` matches ...@@ -600,8 +596,8 @@ Like the `regex.match()` method, return `true` if `pattern` matches
`subject`, where `pattern` is compiled with the given options (or `subject`, where `pattern` is compiled with the given options (or
default options) on each invocation. default options) on each invocation.
If `pattern` fails to compile, then an error message is logged with the If `pattern` fails to compile, then VCL failure is invoked (see
`VCL_Error` tag, and `false` is returned. [ERRORS](#errors)).
Example: Example:
...@@ -622,20 +618,20 @@ call of the `match()` function in the current client or backend context, ...@@ -622,20 +618,20 @@ call of the `match()` function in the current client or backend context,
or a fallback string if the capture fails. The default `fallback` is or a fallback string if the capture fails. The default `fallback` is
`"**BACKREF FUNCTION FAILED**"`. `"**BACKREF FUNCTION FAILED**"`.
Similarly to the `regex.backref()` method, `fallback` is returned after Similarly to the `regex.backref()` method, `fallback` is returned if
any failed invocation of the `match()` function, or if there is no there is no captured group corresponding to the backref number. The
captured group corresponding to the backref number. The function is not function is not affected by native VCL regex operations, or any other
affected by native VCL regex operations, or any other method or function method or function of the VMOD except for the `match()` function.
of the VMOD except for the `match()` function.
The function fails, returning `fallback` and logging a `VCL_Error` The function invokes VCL failure under the same conditions as the
message, under the same conditions as the corresponding method: corresponding method (see [ERRORS](#errors)):
- `fallback` is undefined. - `fallback` is undefined.
- `never_capture` was true in the previous invocation of the `match()` - `never_capture` was true in the previous invocation of the `match()`
function. function.
- `ref` is out of range. - `ref` is out of range.
- The `match()` function was never called in this context. - The `match()` function was never called in this context, or if the
previous `match()` call failed (returned `false`).
- The pattern failed to compile for the previous `match()` call. - The pattern failed to compile for the previous `match()` call.
- There is insufficient workspace for the captured subexpression. - There is insufficient workspace for the captured subexpression.
...@@ -656,20 +652,18 @@ Example: ...@@ -656,20 +652,18 @@ Example:
Returns the captured subexpression designated by `name` from the most Returns the captured subexpression designated by `name` from the most
recent successful call to the `match()` function in the current context, recent successful call to the `match()` function in the current context,
or `fallback` in case of failure. The default fallback is `"**NAMEDREF or `fallback` if the corresponding group did not match. The default
FUNCTION FAILED**"`. fallback is `"**NAMEDREF FUNCTION FAILED**"`.
The function returns `fallback` when the previous invocation of the The function invokes VCL failure under the same conditions as the
`match()` function failed, and is only affected by use of the `match()` corresponding method (see [ERRORS](#errors)):
function. The function fails, returning `fallback` and logging a
`VCL_Error` message, under the same conditions as the corresponding
method:
- `fallback` is undefined. - `fallback` is undefined.
- `name` is undefined or the empty string. - `name` is undefined or the empty string.
- The `never_capture` option was set to `true`. - The `never_capture` option was set to `true`.
- There is no such named group. - There is no such named group.
- `match()` was not called in this context. - `match()` was not called in this context, or the previous call
failed.
- The pattern failed to compile for the previous `match()` call. - The pattern failed to compile for the previous `match()` call.
- There is insufficient workspace for the captured expression. - There is insufficient workspace for the captured expression.
...@@ -708,8 +702,7 @@ used in `rewrite` to substitute captured groups from the pattern. ...@@ -708,8 +702,7 @@ used in `rewrite` to substitute captured groups from the pattern.
`fallback` is returned if the pattern does not match `text`. The default `fallback` is returned if the pattern does not match `text`. The default
fallback is `"**SUB FUNCTION FAILED**"`. fallback is `"**SUB FUNCTION FAILED**"`.
`sub()` fails, returning `fallback` and logging a `VCL_Error` message, `sub()` invokes VCL failure (see [ERRORS](#errors)) if:
if:
- `pattern` cannot be compiled. - `pattern` cannot be compiled.
- Any of `text`, `rewrite` or `fallback` are undefined. - Any of `text`, `rewrite` or `fallback` are undefined.
...@@ -817,7 +810,8 @@ Like the `.cost()` method above, return a numeric measurement \> 0 from ...@@ -817,7 +810,8 @@ Like the `.cost()` method above, return a numeric measurement \> 0 from
the RE2 library for `pattern` with the given options. More complex the RE2 library for `pattern` with the given options. More complex
regexen have a higher cost than less complex regexen. regexen have a higher cost than less complex regexen.
Fails and returns -1 if `pattern` cannot be compiled. Invokes VCL failure if `pattern` cannot be compiled (see
[ERRORS](#errors)).
Example: Example:
...@@ -862,18 +856,26 @@ the resulting composed pattern. However, the `never_capture` option ...@@ -862,18 +856,26 @@ the resulting composed pattern. However, the `never_capture` option
cannot be set, and is always implicitly true, since backrefs and cannot be set, and is always implicitly true, since backrefs and
namedrefs are not possible with sets. namedrefs are not possible with sets.
Sets are compiled automatically when `vcl_init` finishes (or when the
deprecated `.compile()` method is called). Compilation fails if any of
the added patterns cannot be compiled, or if no patterns were added to
the set. It may also fail if the `max_mem` setting is not large enough
for the composed pattern. In that case, the VCL load will fail with an
error message (then consider a larger value for `max_mem` in the set
constructor).
Example: Example:
sub vcl_init { sub vcl_init {
# Initialize a regex set for partial matches # Initialize a regex set for partial matches
# with default options # with default options
new foo = re2.set(); new foo = re2.set();
# Initialize a regex set for case insensitive matches # Initialize a regex set for case insensitive matches
# with anchors on both ends (^ and $). # with anchors on both ends (^ and $).
new bar = re2.set(anchor=both, case_sensitive=false); new bar = re2.set(anchor=both, case_sensitive=false);
# Initialize a regex set using POSIX syntax, but allowing # Initialize a regex set using POSIX syntax, but allowing
# Perl character classes, and anchoring at the left (^). # Perl character classes, and anchoring at the left (^).
new baz = re2.set(anchor=start, posix_syntax=true, new baz = re2.set(anchor=start, posix_syntax=true,
perl_classes=true); perl_classes=true);
...@@ -895,12 +897,12 @@ Add the given pattern to the set. If the pattern is invalid, `.add()` ...@@ -895,12 +897,12 @@ Add the given pattern to the set. If the pattern is invalid, `.add()`
fails, and the VCL will fail to load, with an error message describing fails, and the VCL will fail to load, with an error message describing
the problem. the problem.
If values for the `string`, `backend` and/or `integer` parameters are If values for the `string`, `backend`, `integer` and/or `sub` parameters
provided, then these values can be retrieved with the `.string()`, are provided, then these values can be retrieved with the `.string()`,
`.backend()` and `.integer()` methods, respectively, as described below. `.backend()`, `.integer()` and `.subroutine()` methods, respectively, as
This makes it possible to associate data with the added pattern after it described below. This makes it possible to associate data with the added
matches successfully. By default the pattern is not associated with any pattern after it matches successfully. By default the pattern is not
such value. associated with any such value.
If `save` is true, then the given pattern is compiled and saved as a If `save` is true, then the given pattern is compiled and saved as a
`regex` object, just as if the `regex` constructor described above is `regex` object, just as if the `regex` constructor described above is
...@@ -922,13 +924,10 @@ for the individual regex, even though it is implicitly set to true for ...@@ -922,13 +924,10 @@ for the individual regex, even though it is implicitly set to true for
the full set object (default is false). the full set object (default is false).
`.add()` MUST be called in `vcl_init`, and MAY NOT be called after `.add()` MUST be called in `vcl_init`, and MAY NOT be called after
`.compile()`. If `.add()` is called in any other subroutine, an error `.compile()`. VCL failure is invoked if `.add()` is called in any other
message with `VCL_Error` is logged, and the call has no effect. If it is subroutine (see [ERRORS](#errors)). If it is called in `vcl_init` after
called in `vcl_init` after `.compile()`, then the VCL load will fail `.compile()`, then the VCL load will fail with an error message. Note
with an error message. that `.compile()` is now unneccessary and deprecated.
In other words, add all patterns to the set in `vcl_init`, and finally
call `.compile()` when you're done.
When the `.matched(INT)` method is called after a successful match, the When the `.matched(INT)` method is called after a successful match, the
numbering corresponds to the order in which patterns were added. The numbering corresponds to the order in which patterns were added. The
...@@ -945,7 +944,6 @@ Example: ...@@ -945,7 +944,6 @@ Example:
hostmatcher.add("www.domain1.com"); hostmatcher.add("www.domain1.com");
hostmatcher.add("www.domain2.com"); hostmatcher.add("www.domain2.com");
hostmatcher.add("www.domain3.com"); hostmatcher.add("www.domain3.com");
hostmatcher.compile();
} }
# See the documentation of the .string() and .backend() methods # See the documentation of the .string() and .backend() methods
...@@ -953,20 +951,20 @@ Example: ...@@ -953,20 +951,20 @@ Example:
#### VOID xset.compile() #### VOID xset.compile()
**This method is deprecated**, and will be removed in a future version.
`.compile()` may be omitted, since compilation now happens automatically
when `vcl_init` finishes.
Compile the compound pattern represented by the set -- an alternation of Compile the compound pattern represented by the set -- an alternation of
all patterns added by `.add()`. all patterns added by `.add()`.
`.compile()` fails if no patterns were added to the set. It may also Compilation may fail for any of the reasons described for automatic
fail if the `max_mem` setting is not large enough for the composed compilation of set objects as described above.
pattern. In that case, the VCL load will fail with an error message
(then consider a larger value for `max_mem` in the set constructor).
`.compile()` MUST be called in `vcl_init`, and MAY NOT be called more `.compile()` MUST be called in `vcl_init`, and MAY NOT be called more
than once for a set object. If it is called in any other subroutine, a than once for a set object. VCL failure is invoked if it is called in
`VCL_Error` message is logged, and the call has no effect. If it is any other subroutine. If it is called a second time in `vcl_init`, the
called a second time in `vcl_init`, the VCL load will fail. VCL load will fail.
See above for examples.
#### BOOL xset.match(STRING) #### BOOL xset.match(STRING)
...@@ -979,17 +977,12 @@ and match the given string. These can be determined after a successful ...@@ -979,17 +977,12 @@ and match the given string. These can be determined after a successful
match using the `.matched(INT)` and `.nmatches()` methods described match using the `.matched(INT)` and `.nmatches()` methods described
below. below.
`.match()` MUST be called after `.compile()`; otherwise the match always A match may also fail (leading to VCL failure) if the internal memory
fails. limit imposed by the `max_mem` parameter in the constructor is exceeded.
(With the default value of `max_mem`, this ordinarily requires very
A match may also fail (returning `false`) if the internal memory limit large patterns and/or a very large string to be matched.) Since about
imposed by the `max_mem` parameter in the constructor is exceeded. (With version 2017-12-01, the RE2 library reports this condition. If matches
the default value of `max_mem`, this ordinarily requires very large fail due to the out-of-memory condition, increase the `max_mem`
patterns and/or a very large string to be matched.) Since about version
2017-12-01, the RE2 library reports this condition; if so, the VMOD
writes a `VCL_Error` message in the log if it happens, except during
`vcl_init`, in which case the VCL load fails with the error message. If
matches fail due to the out-of-memory condition, increase the `max_mem`
parameter in the constructor. parameter in the constructor.
Example: Example:
...@@ -1010,20 +1003,13 @@ the same object in the same client or backend context. It always returns ...@@ -1010,20 +1003,13 @@ the same object in the same client or backend context. It always returns
`false`, for every value of the parameter, if it is called after an `false`, for every value of the parameter, if it is called after an
unsuccessful match (`.match()` returned `false`). unsuccessful match (`.match()` returned `false`).
`.matched()` fails and returns `false` if: `.matched()` invokes VCL failure (see [ERRORS](#errors)) if:
- The `.match()` method was not called for this object in the same - The `.match()` method was not called for this object in the same
client or backend scope. client or backend scope.
- The integer parameter is out of range; that is, if it is less than 1 - The integer parameter is out of range; that is, if it is less than 1
or greater than the number of patterns added to the set. or greater than the number of patterns added to the set.
On failure, the method writes an error message to the log with the tag
`VCL_Error`; if it fails during `vcl_init`, then the VCL load fails with
the error message. In any other VCL subroutine, the method returns
`false` on failure and processing continues; since `false` is a
legitimate return value, you should consider monitoring the log for the
error messages.
Example: Example:
if (hostmatcher.match(req.http.Host)) { if (hostmatcher.match(req.http.Host)) {
...@@ -1046,12 +1032,8 @@ backend context. The method always returns 0 after an unsuccessful match ...@@ -1046,12 +1032,8 @@ backend context. The method always returns 0 after an unsuccessful match
(`.match()` returned `false`). (`.match()` returned `false`).
If `.match()` was not called for this object in the same client or If `.match()` was not called for this object in the same client or
backend scope, `.nmatches()` fails and returns 0, writing an error backend scope, `.nmatches()` invokes VCL failure (see
message with `VCL_Error` to the log. If this happens in `vcl_init`, the [ERRORS](#errors)).
VCL load fails with the error message. As with `.matched()`,
`.nmatches()` returns a legitimate value and VCL processing continues
when it fails in any other subroutine, so you should monitor the log for
the error messages.
Example: Example:
...@@ -1078,7 +1060,7 @@ is returned. The values `FIRST` and `LAST` specify that, of the patterns ...@@ -1078,7 +1060,7 @@ is returned. The values `FIRST` and `LAST` specify that, of the patterns
that matched, the first or last one added via the `.add()` method is that matched, the first or last one added via the `.add()` method is
chosen, and the number for that pattern is returned. chosen, and the number for that pattern is returned.
`.which()` fails, returning 0 with a `VCL_Error` message in the log, if: `.which()` invokes VCL failure (see [ERRORS](#errors)) if:
- `.match()` was not called for the set in the current client or - `.match()` was not called for the set in the current client or
backend transaction, or if the previous call returned `false`. backend transaction, or if the previous call returned `false`.
...@@ -1158,8 +1140,7 @@ For the pattern selected by these rules, return the string that was set ...@@ -1158,8 +1140,7 @@ For the pattern selected by these rules, return the string that was set
with the `string` parameter in the `.add()` method that added the with the `string` parameter in the `.add()` method that added the
pattern to the set. pattern to the set.
`.string()` fails, returning NULL with an a `VCL_Error` message in the `.string()` invokes VCL failure (see [ERRORS](#errors)) if:
log, if:
- The values of `n` and `select` are invalid: - The values of `n` and `select` are invalid:
- `n` is greater than the number of patterns in the set. - `n` is greater than the number of patterns in the set.
...@@ -1169,9 +1150,11 @@ log, if: ...@@ -1169,9 +1150,11 @@ log, if:
- `n` \<= 0, but the previous `.match()` call returned `false`. - `n` \<= 0, but the previous `.match()` call returned `false`.
- `n` \<= 0 and the `select` ENUM is `UNIQUE` (or default), but - `n` \<= 0 and the `select` ENUM is `UNIQUE` (or default), but
more than one pattern matched in the previous `.match()` call. more than one pattern matched in the previous `.match()` call.
This can be avoided by checking for `.nmatches() == 1`.
- No string was associated with the pattern selected by `n` and - No string was associated with the pattern selected by `n` and
`select`; that is, the `string` parameter was not set in the `select`; that is, the `string` parameter was not set in the
`.add()` call that added the pattern. `.add()` call that added the pattern. This can be avoided by
checking the `.saved()` method (see below).
Examples: Examples:
...@@ -1255,10 +1238,10 @@ The rules for selecting a pattern from the set and its associated ...@@ -1255,10 +1238,10 @@ The rules for selecting a pattern from the set and its associated
backend based on `n` and `select` are the same as described above for backend based on `n` and `select` are the same as described above for
`.string()`. `.string()`.
`.backend()` fails, returning NULL with an a `VCL_Error` message in the `.backend()` invokes VCL failure under the same conditions described for
log, under the same conditions described for `.string()` above -- `n` `.string()` above -- `n` and `select` are invalid, or no backend was
and `select` are invalid, or no backend was associated with the selected associated with the selected pattern with the `.add()` method (see
pattern with the `.add()` method. [ERRORS](#errors)).
Example: Example:
...@@ -1308,18 +1291,7 @@ integer based on `n` and `select` are the same as described above for ...@@ -1308,18 +1291,7 @@ integer based on `n` and `select` are the same as described above for
`.integer()` invokes VCL failure under the same error conditions `.integer()` invokes VCL failure under the same error conditions
described for `.string()` above -- `n` and `select` are invalid, or no described for `.string()` above -- `n` and `select` are invalid, or no
integer was associated with the selected pattern with the `.add()` integer was associated with the selected pattern with the `.add()`
method. method (see [ERRORS](#errors)).
Note that VCL failure differs from the failure mode for `.string()` and
`.backend()`, since there is no distinguished "error" value that could
be returned as the INT. VCL failure has the same effect as if
`return(fail)` were called from a VCL subroutine; usually, control
directs immediately to `vcl_synth`, with the response status set to 503,
and the response reason set to "VCL failed".
You can avoid that, for example, by testing if `.nmatches()==1` after
calling `.match()`, if you need to ensure that calling
`.integer(select=UNIQUE)` will not fail.
Example: Example:
...@@ -1383,8 +1355,7 @@ the rules given above. If an internal `regex` object was saved for that ...@@ -1383,8 +1355,7 @@ the rules given above. If an internal `regex` object was saved for that
pattern, then the result of the `.sub()` method invoked on that object pattern, then the result of the `.sub()` method invoked on that object
is returned. is returned.
`.sub()` fails, returning NULL with a `VCL_Error` message in the log, `.sub()` invokes VCL failkure (see [ERRORS](#errors)) if:
if:
- The values of `n` and `select` are invalid, according to the rules - The values of `n` and `select` are invalid, according to the rules
given above. given above.
...@@ -1531,7 +1502,59 @@ Example: ...@@ -1531,7 +1502,59 @@ Example:
ENUM {FIRST, LAST, UNIQUE} select=UNIQUE ENUM {FIRST, LAST, UNIQUE} select=UNIQUE
) )
XXX ... Returns the subroutine set by the `sub` parameter for the element of the
set indicated by `n` and `select`, according to the rules given above.
The subroutine may be invoked with VCL `call`.
**Note**: you must ensure that the subroutine may invoked legally in the
context in which it is called. This means that:
- The subroutine may only refer to VCL elements that are legal in the
invocation context. For example, if the subroutine only refers to
headers in `req.http.*`, then it may be called in `vcl_recv`, but
not if it refers to any header in `resp.http.*`. See `vcl-var(7)`
for the specification of which VCL variables may be used in which
contexts.
- Recursive subroutine calls are not permitted in VCL. The subroutine
invocation may not appear anywhere in its own call stack.
For standard subroutine invocations with `call`, the VCL compiler checks
these conditions and issues a compile-time error if either one is
violated. This is not possible with invocations using `.subroutine()`;
the error can only be determined at runtime. So it is advisable to test
the use of `.subroutine()` carefully before using it in production. You
can use the `.check_call()` method described below to determine if the
subroutine call is legal.
`.subroutine()` invokes VCL failure (See [ERRORS](#errors)) if:
- The rules for `n` and `select` indicate failure.
- No subroutine was set with the `sub` parameter in `.add()`.
- The subroutine is invoked with `call`, but the call is not legal in
the invocation context, for the reasons given above.
Example:
# Due to the use of resp.http.*, this subroutine may only be invoked
# in vcl_deliver or vcl_synth, as documented in vcl-var(7). Note
# that subroutine definitions must appear before vcl_init to
# permitted for the sub parameter in .add().
sub resp_sub {
set resp.http.Call-Me = "but only in deliver or synth";
}
sub vcl_init {
new myset = re2.set();
myset.add("/foo", sub=resp_sub);
myset.add("/foo/bar", sub=some_other_sub);
# ...
}
sub vcl_deliver {
if (myset.match(req.url)) {
call myset.subroutine(select=FIRST);
}
}
#### BOOL xset.check\_call(INT n, ENUM select) #### BOOL xset.check\_call(INT n, ENUM select)
...@@ -1540,7 +1563,33 @@ XXX ... ...@@ -1540,7 +1563,33 @@ XXX ...
ENUM {FIRST, LAST, UNIQUE} select=UNIQUE ENUM {FIRST, LAST, UNIQUE} select=UNIQUE
) )
XXX ... Returns `true` iff the subroutine returned by `.subroutine()` for the
element of the set indicated by `n` and `select` may be invoked legally
in the current context. The conditions for legal invocation are
documented for `.subroutine()` above.
`.check_call()` never invokes VCL failure, but rather returns `false`
under conditions for which the use of `.subroutine()` would invoke VCL
failure. In that case, a message is emitted to the Vanrish log using the
`Notice` tag (the same message that would appear with the `VCL_Error`
tag if the subroutine were called).
`Notice` messages in the log produced by this VMOD are always prefixed
with the string `vmod_re2:`.
Example:
# Assume that myset is declared as in the example above.
sub vcl_deliver {
if (myset.match(req.url)) {
if (myset.check_call(select=FIRST)) {
call myset.subroutine(select=FIRST);
}
else {
call do_if_resp_sub_is_illegal;
}
}
}
#### BOOL xset.saved(ENUM which, INT n, ENUM select) #### BOOL xset.saved(ENUM which, INT n, ENUM select)
...@@ -1564,13 +1613,13 @@ In other words, `.saved()` returns true: ...@@ -1564,13 +1613,13 @@ In other words, `.saved()` returns true:
- for `which=BE` if a backend was stored with the `backend` attribute. - for `which=BE` if a backend was stored with the `backend` attribute.
- for `which=INT` if an integer was stored with the `integer` - for `which=INT` if an integer was stored with the `integer`
attribute. attribute.
- for `which=SUB` if an integer was stored with the `sub` attribute.
The default value of `which` is `REGEX`. The default value of `which` is `REGEX`.
The pattern in the set is identified by `n` and `select` according to The pattern in the set is identified by `n` and `select` according to
the rules given above. `.saved()` fails, returning false with a the rules given above. `.saved()` invokes VCL failure if the values of
`VCL_Error` message in the log, if the values of `n` or `select` are `n` or `select` are invalid (see [ERRORS](#errors)).
invalid.
Example: Example:
...@@ -1666,8 +1715,8 @@ special characters. This function has a purpose similar to a `\Q..\E` ...@@ -1666,8 +1715,8 @@ special characters. This function has a purpose similar to a `\Q..\E`
sequence within a regex, or the `literal=true` setting in a regex sequence within a regex, or the `literal=true` setting in a regex
constructor. constructor.
The function fails and returns `fallback` if there is insufficient The function invokes VCL failure if there is insufficient workspace for
workspace for the return string. the return string (see [ERRORS](#errors)).
Example: Example:
...@@ -1683,14 +1732,70 @@ Example: ...@@ -1683,14 +1732,70 @@ Example:
std.log("Using VMOD re2 version: " + re2.version()); std.log("Using VMOD re2 version: " + re2.version());
### ERRORS
Functions and methods of the VMOD may invoke VCL failure under
unrecoverable error conditions. The effects of VCL failure depend on the
VCL subroutine in which it takes place:
- If invoked during `vcl_init`, then the VCL load fails, and an error
message is returned over the CLI (reported by `varnishadm(1)`).
- If invoked during any other subroutine besides `vcl_synth`, then an
error message is recorded in the log with the `VCL_Error` tag,
further processing is aborted immediately, and a response with
status 503 (Service Not Available) is returned with the reason
string "VCL failed".
- If invoked during `vcl_synth`, then further processing is aborted,
the error message is logged with `VCL_Error`, and the client
connection is immediately closed -- the client receives no response.
Errors that lead to VCL failure include:
- Any regex compilation failure.
- Out of workspace errors (see [LIMITATIONS](#limitations)).
- Failures reported by the RE2 library for: matches, backrefs,
namedrefs, the rewrite operations (sub, suball and extract), the
`.cost()` function or method, and the `.quotemeta()` function. The
VMOD detects most common errors that would lead to library errors,
and invokes VCL failure in such cases without calling the library.
But library errors may happen under conditions such as out of
memory.
- Functions and methods that require a previous successful match when
there was no prior match, or when the previous match was
unsuccessful. These include backrefs, namedrefs, and the data
retrieval methods for set objects.
- Any of the following parameters are undefined, for example when set
from an unset header: fallbacks; patterns for the regex functions
(which are compiled at runtime); the text and rewrite parameters for
rewrite operations; the name parameter for namedrefs.
- The name parameter for namedrefs is the empty string.
- Backref number is out of range (greater than the number of backrefs
in the pattern).
- Backref or namedref attempted when the `never_capture` option was
set to `true` for the pattern.
- For set objects:
- Numeric index (parameter `n`) is out of range (greater than the
number of patterns in the set).
- Use of `select=UNIQUE` after more than one pattern was matched.
The `.nmatches()` can be used to check for this condition, to
avoid VCL failure -- `UNIQUE` will fail in `.namtches()` \> 1.
- Retrieval of data from a set (such as a string, backend etc) by
numeric index (`n`) or "associatively" (after a match) when no
such object was saved for the corresponding pattern. Use the
`.saved()` and `.check_call()` methods to check for this.
- Calling the subroutine returned by `.subrooutine()` may be
illegal, if it is not permitted in the subroutine from which it
is called, or if it would lead to recursive calls. Use the
`.check_call()` method to check for this.
### REQUIREMENTS ### REQUIREMENTS
The VMOD requires the Varnish master branch. See the source repository The VMOD requires Varnish since version 6.6, or the master branch. See
for versions of the VMOD that are compatible with other Varnish the source repository for versions of the VMOD that are compatible with
versions. other Varnish versions.
It requires the RE2 library, and has been tested against RE2 versions It requires the RE2 library, and has been tested against RE2 versions
since 2015-06-01 (through 2020-11-01 at the time of writing). since 2015-06-01 (through 2021-04-01 at the time of writing).
If the VMOD is built against versions of RE2 since 2017-12-01, it uses a If the VMOD is built against versions of RE2 since 2017-12-01, it uses a
version of the set match operation that reports out-of-memory conditions version of the set match operation that reports out-of-memory conditions
......
src/vmod_re2.vcc
View file @ 6d20f3a9
...@@ -46,8 +46,7 @@ SYNOPSIS ...@@ -46,8 +46,7 @@ SYNOPSIS
# set object interface # set object interface
new OBJECT = re2.set([ENUM anchor] [, <regex options>]) new OBJECT = re2.set([ENUM anchor] [, <regex options>])
VOID <obj>.add(STRING [, BOOL save] [, BOOL never_capture] [, STRING string] VOID <obj>.add(STRING [, BOOL save] [, BOOL never_capture] [, STRING string]
[, BACKEND backend] [, INT integer]) [, BACKEND backend] [, INT integer] [,SUB sub])
VOID <obj>.compile()
BOOL <obj>.match(STRING) BOOL <obj>.match(STRING)
INT <obj>.nmatches() INT <obj>.nmatches()
BOOL <obj>.matched(INT) BOOL <obj>.matched(INT)
...@@ -55,13 +54,15 @@ SYNOPSIS ...@@ -55,13 +54,15 @@ SYNOPSIS
STRING <obj>.string([INT n,] [ENUM select]) STRING <obj>.string([INT n,] [ENUM select])
BACKEND <obj>.backend([INT n,] [ENUM select]) BACKEND <obj>.backend([INT n,] [ENUM select])
INT <obj>.integer([INT n] [, ENUM select]) INT <obj>.integer([INT n] [, ENUM select])
SUB <obj>.subroutine([INT n] [, ENUM select])
BOOL <obj>.check_call([INT n] [, ENUM select])
STRING <obj>.sub(STRING text, STRING rewrite [, INT n] STRING <obj>.sub(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
STRING <obj>.suball(STRING text, STRING rewrite [, INT n] STRING <obj>.suball(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
STRING <obj>.extract(STRING text, STRING rewrite [, INT n] STRING <obj>.extract(STRING text, STRING rewrite [, INT n]
[, ENUM select]) [, ENUM select])
BOOL <obj>.saved([ENUM {REGEX, STR, BE, INT} which] [, INT n] BOOL <obj>.saved([ENUM {REGEX, STR, BE, INT, SUB} which] [, INT n]
[, ENUM select]) [, ENUM select])
VOID <obj>.hdr_filter(HTTP [, BOOL]) VOID <obj>.hdr_filter(HTTP [, BOOL])
...@@ -141,7 +142,6 @@ example:: ...@@ -141,7 +142,6 @@ example::
myset.add("foo"); # Pattern 1 myset.add("foo"); # Pattern 1
myset.add("bar"); # Pattern 2 myset.add("bar"); # Pattern 2
myset.add("baz"); # Pattern 3 myset.add("baz"); # Pattern 3
myset.compile();
} }
``myset.match(<string>)`` can now be used to match a string against ``myset.match(<string>)`` can now be used to match a string against
...@@ -195,14 +195,12 @@ strings or backends with the patterns added to the set with the ...@@ -195,14 +195,12 @@ strings or backends with the patterns added to the set with the
prefix.add("/bar", string="www.domain2.com"); prefix.add("/bar", string="www.domain2.com");
prefix.add("/baz", string="www.domain3.com"); prefix.add("/baz", string="www.domain3.com");
prefix.add("/quux", string="www.domain4.com"); prefix.add("/quux", string="www.domain4.com");
prefix.compile();
new appmatcher = re2.set(anchor=start); new appmatcher = re2.set(anchor=start);
appmatcher.add("/foo", backend=app1); appmatcher.add("/foo", backend=app1);
appmatcher.add("/bar", backend=app2); appmatcher.add("/bar", backend=app2);
appmatcher.add("/baz", backend=app3); appmatcher.add("/baz", backend=app3);
appmatcher.add("/quux", backend=app4); appmatcher.add("/quux", backend=app4);
appmatcher.compile();
} }
After a successful match, the string or backend associated with the After a successful match, the string or backend associated with the
...@@ -285,11 +283,13 @@ constructor apply to all of the patterns in the resulting alternation. ...@@ -285,11 +283,13 @@ constructor apply to all of the patterns in the resulting alternation.
including newline. Otherwise, ``.`` never matches newline. Default is including newline. Otherwise, ``.`` never matches newline. Default is
**false**. **false**.
``never_capture`` ``never_capture``
If true, parentheses in a pattern are interpreted as non-capturing, and all If true, parentheses in a pattern are interpreted as non-capturing,
invocations of the ``backref`` and ``namedref`` methods or functions will and all invocations of the ``backref`` and ``namedref`` methods or
fail, including ``backref(0)`` after a successful match. Default is **false**, functions will lead to VCL faillure (see `ERRORS`_), including
except for set objects, for which ``never_capture`` is always true (and cannot ``backref(0)`` after a successful match. Default is **false**,
be changed), since back references are not possible with sets. except for set objects, for which ``never_capture`` is always true
(and cannot be changed), since back references are not possible with
sets.
``case_sensitive`` ``case_sensitive``
If true, matches are case-sensitive. A pattern can override this option with If true, matches are case-sensitive. A pattern can override this option with
the ``(?i)`` flag, unless ``posix_syntax`` is true. Default is **true**. the ``(?i)`` flag, unless ``posix_syntax`` is true. Default is **true**.
...@@ -346,9 +346,9 @@ $Method STRING .backref(INT ref, STRING fallback = "**BACKREF METHOD FAILED**") ...@@ -346,9 +346,9 @@ $Method STRING .backref(INT ref, STRING fallback = "**BACKREF METHOD FAILED**")
Returns the `nth` captured subexpression from the most recent Returns the `nth` captured subexpression from the most recent
successful call of the ``.match()`` method for this object in the same successful call of the ``.match()`` method for this object in the same
client or backend, context, or a fallback string in case the capture client or backend context, or a fallback string in case the capture
fails. Backref 0 indicates the entire matched string. Thus this fails. Backref 0 indicates the entire matched string. Thus this
function behaves like the ``\n`` in the native VCL functions function behaves like the ``\n`` notation in the native VCL functions
``regsub`` and ``regsuball``, and the ``$1``, ``$2`` ... variables in ``regsub`` and ``regsuball``, and the ``$1``, ``$2`` ... variables in
Perl. Perl.
...@@ -360,17 +360,16 @@ subroutines -- the backend context -- refers back to a previous ...@@ -360,17 +360,16 @@ subroutines -- the backend context -- refers back to a previous
the other VCL subroutines -- the client context -- refers back to a the other VCL subroutines -- the client context -- refers back to a
``.match()`` in the same client context. ``.match()`` in the same client context.
After unsuccessful matches, the ``fallback`` string is returned for ``.backref()`` may return ``fallback`` after a successful match, if no
any call to ``.backref()``. The default value of ``fallback`` is captured group in the matching string corresponds to the backref
``"**BACKREF METHOD FAILED**"``. ``.backref()`` always fails after a
failed match, even if ``.match()`` had been called successfully before
the failure.
``.backref()`` may also return ``fallback`` after a successful match,
if no captured group in the matching string corresponds to the backref
number. For example, when the pattern ``(a|(b))c`` matches the string number. For example, when the pattern ``(a|(b))c`` matches the string
``ac``, there is no backref 2, since nothing matches ``b`` in the ``ac``, there is no backref 2, since nothing matches ``b`` in the
string. string. The default value of ``fallback`` is ``"**BACKREF METHOD
FAILED**"``, but you may set another value (such as the empty string).
After unsuccessful matches, ``.backref()`` invokes VCL failure (see
`ERRORS`_). ``.backref()`` always fails after a failed match, even if
``.match()`` had been called successfully before the failure.
The VCL infix operators ``~`` and ``!~`` do not affect this method, The VCL infix operators ``~`` and ``!~`` do not affect this method,
nor do the functions ``regsub`` or ``regsuball``. Nor is it affected nor do the functions ``regsub`` or ``regsuball``. Nor is it affected
...@@ -378,10 +377,9 @@ by the matches performed by any other method or function in this VMOD ...@@ -378,10 +377,9 @@ by the matches performed by any other method or function in this VMOD
(such as the ``sub()``, ``suball()`` or ``extract()`` methods or (such as the ``sub()``, ``suball()`` or ``extract()`` methods or
functions, or the ``set`` object's ``.match()`` method). functions, or the ``set`` object's ``.match()`` method).
``.backref()`` fails, returning ``fallback`` and writing an error ``.backref()`` invokes VCL failure under the following conditions,
message to the Varnish log with the ``VCL_Error`` tag, under the even if a previous match was successful and a substring could have
following conditions (even if a previous match was successful and a been captured (see `ERRORS`_):
substring could have been captured):
* The ``fallback`` string is undefined, for example if set from an unset * The ``fallback`` string is undefined, for example if set from an unset
header variable. header variable.
...@@ -404,7 +402,7 @@ $Method STRING .namedref(STRING name, ...@@ -404,7 +402,7 @@ $Method STRING .namedref(STRING name,
Returns the captured subexpression designated by ``name`` from the Returns the captured subexpression designated by ``name`` from the
most recent successful call to ``.match()`` in the current context most recent successful call to ``.match()`` in the current context
(client or backend), or ``fallback`` in case of failure. (client or backend).
Named capturing groups are written in RE2 as: ``(?P<name>re)``. (Note Named capturing groups are written in RE2 as: ``(?P<name>re)``. (Note
that this syntax with ``P``, inspired by Python, differs from the that this syntax with ``P``, inspired by Python, differs from the
...@@ -416,17 +414,15 @@ Note that a named capturing group can also be referenced as a numbered ...@@ -416,17 +414,15 @@ Note that a named capturing group can also be referenced as a numbered
group. So in the previous example, ``.backref(1)`` also returns group. So in the previous example, ``.backref(1)`` also returns
``baz``. ``baz``.
``fallback`` is returned when ``.namedref()`` is called after an ``fallback`` is returned when the named reference did not match. The
unsuccessful match. The default fallback is ``"**NAMEDREF METHOD default fallback is ``"**NAMEDREF METHOD FAILED**"``.
FAILED**"``.
Like ``.backref()``, ``.namedref()`` is not affected by native VCL Like ``.backref()``, ``.namedref()`` is not affected by native VCL
regex operations, nor by any other matches performed by methods or regex operations, nor by any other matches performed by methods or
functions of the VMOD, except for a prior ``.match()`` for the same functions of the VMOD, except for a prior ``.match()`` for the same
object. object.
``.namedref()`` fails, returning ``fallback`` and logging a ``.namedref()`` invokes VCL failure (see `ERRORS`_) if:
``VCL_Error`` message, if:
* The ``fallback`` string is undefined. * The ``fallback`` string is undefined.
* ``name`` is undefined or the empty string. * ``name`` is undefined or the empty string.
...@@ -460,8 +456,7 @@ native function ``regsub()``. ...@@ -460,8 +456,7 @@ native function ``regsub()``.
``fallback`` is returned if the pattern does not match ``text``. The ``fallback`` is returned if the pattern does not match ``text``. The
default fallback is ``"**SUB METHOD FAILED**"``. default fallback is ``"**SUB METHOD FAILED**"``.
``.sub()`` fails, returning ``fallback`` and logging a ``VCL_Error`` ``.sub()`` invokes VCL failure (see `ERRORS`_) if:
message, if:
* Any of ``text``, ``rewrite`` or ``fallback`` are undefined. * Any of ``text``, ``rewrite`` or ``fallback`` are undefined.
* There is insufficient workspace for the rewritten string. * There is insufficient workspace for the rewritten string.
...@@ -559,8 +554,8 @@ Like the ``regex.match()`` method, return ``true`` if ``pattern`` ...@@ -559,8 +554,8 @@ Like the ``regex.match()`` method, return ``true`` if ``pattern``
matches ``subject``, where ``pattern`` is compiled with the given matches ``subject``, where ``pattern`` is compiled with the given
options (or default options) on each invocation. options (or default options) on each invocation.
If ``pattern`` fails to compile, then an error message is logged with If ``pattern`` fails to compile, then VCL failure is invoked (see
the ``VCL_Error`` tag, and ``false`` is returned. `ERRORS`_).
Example:: Example::
...@@ -578,19 +573,19 @@ backend context, or a fallback string if the capture fails. The ...@@ -578,19 +573,19 @@ backend context, or a fallback string if the capture fails. The
default ``fallback`` is ``"**BACKREF FUNCTION FAILED**"``. default ``fallback`` is ``"**BACKREF FUNCTION FAILED**"``.
Similarly to the ``regex.backref()`` method, ``fallback`` is returned Similarly to the ``regex.backref()`` method, ``fallback`` is returned
after any failed invocation of the ``match()`` function, or if there if there is no captured group corresponding to the backref number. The
is no captured group corresponding to the backref number. The function function is not affected by native VCL regex operations, or any other
is not affected by native VCL regex operations, or any other method or method or function of the VMOD except for the ``match()`` function.
function of the VMOD except for the ``match()`` function.
The function fails, returning ``fallback`` and logging a ``VCL_Error`` The function invokes VCL failure under the same conditions as the
message, under the same conditions as the corresponding method: corresponding method (see `ERRORS`_):
* ``fallback`` is undefined. * ``fallback`` is undefined.
* ``never_capture`` was true in the previous invocation of the ``match()`` * ``never_capture`` was true in the previous invocation of the ``match()``
function. function.
* ``ref`` is out of range. * ``ref`` is out of range.
* The ``match()`` function was never called in this context. * The ``match()`` function was never called in this context, or if the
previous ``match()`` call failed (returned ``false``).
* The pattern failed to compile for the previous ``match()`` call. * The pattern failed to compile for the previous ``match()`` call.
* There is insufficient workspace for the captured subexpression. * There is insufficient workspace for the captured subexpression.
...@@ -607,20 +602,17 @@ $Function STRING namedref(PRIV_TASK, STRING name, ...@@ -607,20 +602,17 @@ $Function STRING namedref(PRIV_TASK, STRING name,
Returns the captured subexpression designated by ``name`` from the Returns the captured subexpression designated by ``name`` from the
most recent successful call to the ``match()`` function in the current most recent successful call to the ``match()`` function in the current
context, or ``fallback`` in case of failure. The default fallback is context, or ``fallback`` if the corresponding group did not match. The
``"**NAMEDREF FUNCTION FAILED**"``. default fallback is ``"**NAMEDREF FUNCTION FAILED**"``.
The function returns ``fallback`` when the previous invocation of the The function invokes VCL failure under the same conditions as the
``match()`` function failed, and is only affected by use of the corresponding method (see `ERRORS`_):
``match()`` function. The function fails, returning ``fallback`` and
logging a ``VCL_Error`` message, under the same conditions as the
corresponding method:
* ``fallback`` is undefined. * ``fallback`` is undefined.
* ``name`` is undefined or the empty string. * ``name`` is undefined or the empty string.
* The ``never_capture`` option was set to ``true``. * The ``never_capture`` option was set to ``true``.
* There is no such named group. * There is no such named group.
* ``match()`` was not called in this context. * ``match()`` was not called in this context, or the previous call failed.
* The pattern failed to compile for the previous ``match()`` call. * The pattern failed to compile for the previous ``match()`` call.
* There is insufficient workspace for the captured expression. * There is insufficient workspace for the captured expression.
...@@ -646,8 +638,7 @@ groups from the pattern. ...@@ -646,8 +638,7 @@ groups from the pattern.
``fallback`` is returned if the pattern does not match ``text``. The ``fallback`` is returned if the pattern does not match ``text``. The
default fallback is ``"**SUB FUNCTION FAILED**"``. default fallback is ``"**SUB FUNCTION FAILED**"``.
``sub()`` fails, returning ``fallback`` and logging a ``VCL_Error`` ``sub()`` invokes VCL failure (see `ERRORS`_) if:
message, if:
* ``pattern`` cannot be compiled. * ``pattern`` cannot be compiled.
* Any of ``text``, ``rewrite`` or ``fallback`` are undefined. * Any of ``text``, ``rewrite`` or ``fallback`` are undefined.
...@@ -716,7 +707,7 @@ Like the ``.cost()`` method above, return a numeric measurement > 0 ...@@ -716,7 +707,7 @@ Like the ``.cost()`` method above, return a numeric measurement > 0
from the RE2 library for ``pattern`` with the given options. More from the RE2 library for ``pattern`` with the given options. More
complex regexen have a higher cost than less complex regexen. complex regexen have a higher cost than less complex regexen.
Fails and returns -1 if ``pattern`` cannot be compiled. Invokes VCL failure if ``pattern`` cannot be compiled (see `ERRORS`_).
Example:: Example::
...@@ -752,20 +743,28 @@ against the resulting composed pattern. However, the ``never_capture`` ...@@ -752,20 +743,28 @@ against the resulting composed pattern. However, the ``never_capture``
option cannot be set, and is always implicitly true, since backrefs option cannot be set, and is always implicitly true, since backrefs
and namedrefs are not possible with sets. and namedrefs are not possible with sets.
Sets are compiled automatically when ``vcl_init`` finishes (or when
the deprecated ``.compile()`` method is called). Compilation fails if
any of the added patterns cannot be compiled, or if no patterns were
added to the set. It may also fail if the ``max_mem`` setting is not
large enough for the composed pattern. In that case, the VCL load will
fail with an error message (then consider a larger value for
``max_mem`` in the set constructor).
Example:: Example::
sub vcl_init { sub vcl_init {
# Initialize a regex set for partial matches # Initialize a regex set for partial matches
# with default options # with default options
new foo = re2.set(); new foo = re2.set();
# Initialize a regex set for case insensitive matches # Initialize a regex set for case insensitive matches
# with anchors on both ends (^ and $). # with anchors on both ends (^ and $).
new bar = re2.set(anchor=both, case_sensitive=false); new bar = re2.set(anchor=both, case_sensitive=false);
# Initialize a regex set using POSIX syntax, but allowing # Initialize a regex set using POSIX syntax, but allowing
# Perl character classes, and anchoring at the left (^). # Perl character classes, and anchoring at the left (^).
new baz = re2.set(anchor=start, posix_syntax=true, new baz = re2.set(anchor=start, posix_syntax=true,
perl_classes=true); perl_classes=true);
} }
...@@ -776,12 +775,13 @@ Add the given pattern to the set. If the pattern is invalid, ...@@ -776,12 +775,13 @@ Add the given pattern to the set. If the pattern is invalid,
``.add()`` fails, and the VCL will fail to load, with an error message ``.add()`` fails, and the VCL will fail to load, with an error message
describing the problem. describing the problem.
If values for the ``string``, ``backend`` and/or ``integer`` If values for the ``string``, ``backend``, ``integer`` and/or ``sub``
parameters are provided, then these values can be retrieved with the parameters are provided, then these values can be retrieved with the
``.string()``, ``.backend()`` and ``.integer()`` methods, ``.string()``, ``.backend()``, ``.integer()`` and ``.subroutine()``
respectively, as described below. This makes it possible to associate methods, respectively, as described below. This makes it possible to
data with the added pattern after it matches successfully. By default associate data with the added pattern after it matches
the pattern is not associated with any such value. successfully. By default the pattern is not associated with any such
value.
If ``save`` is true, then the given pattern is compiled and saved as a If ``save`` is true, then the given pattern is compiled and saved as a
``regex`` object, just as if the ``regex`` constructor described above ``regex`` object, just as if the ``regex`` constructor described above
...@@ -803,13 +803,10 @@ be set to false for the individual regex, even though it is implicitly ...@@ -803,13 +803,10 @@ be set to false for the individual regex, even though it is implicitly
set to true for the full set object (default is false). set to true for the full set object (default is false).
``.add()`` MUST be called in ``vcl_init``, and MAY NOT be called after ``.add()`` MUST be called in ``vcl_init``, and MAY NOT be called after
``.compile()``. If ``.add()`` is called in any other subroutine, an ``.compile()``. VCL failure is invoked if ``.add()`` is called in any
error message with ``VCL_Error`` is logged, and the call has no other subroutine (see `ERRORS`_). If it is called in ``vcl_init``
effect. If it is called in ``vcl_init`` after ``.compile()``, then the after ``.compile()``, then the VCL load will fail with an error
VCL load will fail with an error message. message. Note that ``.compile()`` is now unneccessary and deprecated.
In other words, add all patterns to the set in ``vcl_init``, and
finally call ``.compile()`` when you're done.
When the ``.matched(INT)`` method is called after a successful match, When the ``.matched(INT)`` method is called after a successful match,
the numbering corresponds to the order in which patterns were added. the numbering corresponds to the order in which patterns were added.
...@@ -827,7 +824,6 @@ Example:: ...@@ -827,7 +824,6 @@ Example::
hostmatcher.add("www.domain1.com"); hostmatcher.add("www.domain1.com");
hostmatcher.add("www.domain2.com"); hostmatcher.add("www.domain2.com");
hostmatcher.add("www.domain3.com"); hostmatcher.add("www.domain3.com");
hostmatcher.compile();
} }
# See the documentation of the .string() and .backend() methods # See the documentation of the .string() and .backend() methods
...@@ -835,21 +831,20 @@ Example:: ...@@ -835,21 +831,20 @@ Example::
$Method VOID .compile() $Method VOID .compile()
**This method is deprecated**, and will be removed in a future
version. ``.compile()`` may be omitted, since compilation now happens
automatically when ``vcl_init`` finishes.
Compile the compound pattern represented by the set -- an alternation Compile the compound pattern represented by the set -- an alternation
of all patterns added by ``.add()``. of all patterns added by ``.add()``.
``.compile()`` fails if no patterns were added to the set. It may also Compilation may fail for any of the reasons described for automatic
fail if the ``max_mem`` setting is not large enough for the composed compilation of set objects as described above.
pattern. In that case, the VCL load will fail with an error message
(then consider a larger value for ``max_mem`` in the set constructor).
``.compile()`` MUST be called in ``vcl_init``, and MAY NOT be called ``.compile()`` MUST be called in ``vcl_init``, and MAY NOT be called
more than once for a set object. If it is called in any other more than once for a set object. VCL failure is invoked if it is
subroutine, a ``VCL_Error`` message is logged, and the call has no called in any other subroutine. If it is called a second time in
effect. If it is called a second time in ``vcl_init``, the VCL load ``vcl_init``, the VCL load will fail.
will fail.
See above for examples.
$Method BOOL .match(STRING) $Method BOOL .match(STRING)
...@@ -862,19 +857,13 @@ and match the given string. These can be determined after a successful ...@@ -862,19 +857,13 @@ and match the given string. These can be determined after a successful
match using the ``.matched(INT)`` and ``.nmatches()`` methods match using the ``.matched(INT)`` and ``.nmatches()`` methods
described below. described below.
``.match()`` MUST be called after ``.compile()``; otherwise the match A match may also fail (leading to VCL failure) if the internal memory
always fails.
A match may also fail (returning ``false``) if the internal memory
limit imposed by the ``max_mem`` parameter in the constructor is limit imposed by the ``max_mem`` parameter in the constructor is
exceeded. (With the default value of ``max_mem``, this ordinarily exceeded. (With the default value of ``max_mem``, this ordinarily
requires very large patterns and/or a very large string to be requires very large patterns and/or a very large string to be
matched.) Since about version 2017-12-01, the RE2 library reports matched.) Since about version 2017-12-01, the RE2 library reports
this condition; if so, the VMOD writes a ``VCL_Error`` message in the this condition. If matches fail due to the out-of-memory condition,
log if it happens, except during ``vcl_init``, in which case the VCL increase the ``max_mem`` parameter in the constructor.
load fails with the error message. If matches fail due to the
out-of-memory condition, increase the ``max_mem`` parameter in the
constructor.
Example:: Example::
...@@ -894,7 +883,7 @@ for the same object in the same client or backend context. It always ...@@ -894,7 +883,7 @@ for the same object in the same client or backend context. It always
returns ``false``, for every value of the parameter, if it is called returns ``false``, for every value of the parameter, if it is called
after an unsuccessful match (``.match()`` returned ``false``). after an unsuccessful match (``.match()`` returned ``false``).
``.matched()`` fails and returns ``false`` if: ``.matched()`` invokes VCL failure (see `ERRORS`_) if:
* The ``.match()`` method was not called for this object in the same * The ``.match()`` method was not called for this object in the same
client or backend scope. client or backend scope.
...@@ -902,13 +891,6 @@ after an unsuccessful match (``.match()`` returned ``false``). ...@@ -902,13 +891,6 @@ after an unsuccessful match (``.match()`` returned ``false``).
* The integer parameter is out of range; that is, if it is less than 1 * The integer parameter is out of range; that is, if it is less than 1
or greater than the number of patterns added to the set. or greater than the number of patterns added to the set.
On failure, the method writes an error message to the log with the tag
``VCL_Error``; if it fails during ``vcl_init``, then the VCL load
fails with the error message. In any other VCL subroutine, the method
returns ``false`` on failure and processing continues; since ``false``
is a legitimate return value, you should consider monitoring the log
for the error messages.
Example:: Example::
if (hostmatcher.match(req.http.Host)) { if (hostmatcher.match(req.http.Host)) {
...@@ -931,12 +913,7 @@ backend context. The method always returns 0 after an unsuccessful ...@@ -931,12 +913,7 @@ backend context. The method always returns 0 after an unsuccessful
match (``.match()`` returned ``false``). match (``.match()`` returned ``false``).
If ``.match()`` was not called for this object in the same client or If ``.match()`` was not called for this object in the same client or
backend scope, ``.nmatches()`` fails and returns 0, writing an error backend scope, ``.nmatches()`` invokes VCL failure (see `ERRORS`_).
message with ``VCL_Error`` to the log. If this happens in
``vcl_init``, the VCL load fails with the error message. As with
``.matched()``, ``.nmatches()`` returns a legitimate value and VCL
processing continues when it fails in any other subroutine, so you
should monitor the log for the error messages.
Example:: Example::
...@@ -964,8 +941,7 @@ the patterns that matched, the first or last one added via the ...@@ -964,8 +941,7 @@ the patterns that matched, the first or last one added via the
``.add()`` method is chosen, and the number for that pattern is ``.add()`` method is chosen, and the number for that pattern is
returned. returned.
``.which()`` fails, returning 0 with a ``VCL_Error`` message in the log, ``.which()`` invokes VCL failure (see `ERRORS`_) if:
if:
* ``.match()`` was not called for the set in the current client or * ``.match()`` was not called for the set in the current client or
backend transaction, or if the previous call returned ``false``. backend transaction, or if the previous call returned ``false``.
...@@ -1045,8 +1021,7 @@ For the pattern selected by these rules, return the string that was ...@@ -1045,8 +1021,7 @@ For the pattern selected by these rules, return the string that was
set with the ``string`` parameter in the ``.add()`` method that added set with the ``string`` parameter in the ``.add()`` method that added
the pattern to the set. the pattern to the set.
``.string()`` fails, returning NULL with an a ``VCL_Error`` message in ``.string()`` invokes VCL failure (see `ERRORS`_) if:
the log, if:
* The values of ``n`` and ``select`` are invalid: * The values of ``n`` and ``select`` are invalid:
...@@ -1059,10 +1034,12 @@ the log, if: ...@@ -1059,10 +1034,12 @@ the log, if:
* ``n`` <= 0 and the ``select`` ENUM is ``UNIQUE`` (or default), but * ``n`` <= 0 and the ``select`` ENUM is ``UNIQUE`` (or default), but
more than one pattern matched in the previous ``.match()`` call. more than one pattern matched in the previous ``.match()`` call.
This can be avoided by checking for ``.nmatches() == 1``.
* No string was associated with the pattern selected by ``n`` and * No string was associated with the pattern selected by ``n`` and
``select``; that is, the ``string`` parameter was not set in the ``select``; that is, the ``string`` parameter was not set in the
``.add()`` call that added the pattern. ``.add()`` call that added the pattern. This can be avoided by
checking the ``.saved()`` method (see below).
Examples:: Examples::
...@@ -1142,10 +1119,10 @@ The rules for selecting a pattern from the set and its associated ...@@ -1142,10 +1119,10 @@ The rules for selecting a pattern from the set and its associated
backend based on ``n`` and ``select`` are the same as described above backend based on ``n`` and ``select`` are the same as described above
for ``.string()``. for ``.string()``.
``.backend()`` fails, returning NULL with an a ``VCL_Error`` message ``.backend()`` invokes VCL failure under the same conditions described
in the log, under the same conditions described for ``.string()`` for ``.string()`` above -- ``n`` and ``select`` are invalid, or no
above -- ``n`` and ``select`` are invalid, or no backend was backend was associated with the selected pattern with the ``.add()``
associated with the selected pattern with the ``.add()`` method. method (see `ERRORS`_).
Example:: Example::
...@@ -1190,18 +1167,7 @@ for ``.string()``. ...@@ -1190,18 +1167,7 @@ for ``.string()``.
``.integer()`` invokes VCL failure under the same error conditions ``.integer()`` invokes VCL failure under the same error conditions
described for ``.string()`` above -- ``n`` and ``select`` are invalid, described for ``.string()`` above -- ``n`` and ``select`` are invalid,
or no integer was associated with the selected pattern with the or no integer was associated with the selected pattern with the
``.add()`` method. ``.add()`` method (see `ERRORS`_).
Note that VCL failure differs from the failure mode for ``.string()``
and ``.backend()``, since there is no distinguished "error" value that
could be returned as the INT. VCL failure has the same effect as if
``return(fail)`` were called from a VCL subroutine; usually, control
directs immediately to ``vcl_synth``, with the response status set to
503, and the response reason set to "VCL failed".
You can avoid that, for example, by testing if ``.nmatches()==1``
after calling ``.match()``, if you need to ensure that calling
``.integer(select=UNIQUE)`` will not fail.
Example:: Example::
...@@ -1259,8 +1225,7 @@ to the rules given above. If an internal ``regex`` object was saved ...@@ -1259,8 +1225,7 @@ to the rules given above. If an internal ``regex`` object was saved
for that pattern, then the result of the ``.sub()`` method invoked on for that pattern, then the result of the ``.sub()`` method invoked on
that object is returned. that object is returned.
``.sub()`` fails, returning NULL with a ``VCL_Error`` message in the ``.sub()`` invokes VCL failkure (see `ERRORS`_) if:
log, if:
* The values of ``n`` and ``select`` are invalid, according to the * The values of ``n`` and ``select`` are invalid, according to the
rules given above. rules given above.
...@@ -1399,11 +1364,92 @@ Example:: ...@@ -1399,11 +1364,92 @@ Example::
$Method SUB .subroutine(INT n=0, ENUM {FIRST, LAST, UNIQUE} select=UNIQUE) $Method SUB .subroutine(INT n=0, ENUM {FIRST, LAST, UNIQUE} select=UNIQUE)
XXX ... Returns the subroutine set by the ``sub`` parameter for the element of
the set indicated by ``n`` and ``select``, according to the rules
given above. The subroutine may be invoked with VCL ``call``.
**Note**: you must ensure that the subroutine may invoked legally in
the context in which it is called. This means that:
* The subroutine may only refer to VCL elements that are legal in the
invocation context. For example, if the subroutine only refers to
headers in ``req.http.*``, then it may be called in ``vcl_recv``,
but not if it refers to any header in ``resp.http.*``. See
``vcl-var(7)`` for the specification of which VCL variables may be
used in which contexts.
* Recursive subroutine calls are not permitted in VCL. The subroutine
invocation may not appear anywhere in its own call stack.
For standard subroutine invocations with ``call``, the VCL compiler
checks these conditions and issues a compile-time error if either one
is violated. This is not possible with invocations using
``.subroutine()``; the error can only be determined at runtime. So it
is advisable to test the use of ``.subroutine()`` carefully before
using it in production. You can use the ``.check_call()`` method
described below to determine if the subroutine call is legal.
``.subroutine()`` invokes VCL failure (See `ERRORS`_) if:
* The rules for ``n`` and ``select`` indicate failure.
* No subroutine was set with the ``sub`` parameter in ``.add()``.
* The subroutine is invoked with ``call``, but the call is not legal
in the invocation context, for the reasons given above.
Example::
# Due to the use of resp.http.*, this subroutine may only be invoked
# in vcl_deliver or vcl_synth, as documented in vcl-var(7). Note
# that subroutine definitions must appear before vcl_init to
# permitted for the sub parameter in .add().
sub resp_sub {
set resp.http.Call-Me = "but only in deliver or synth";
}
sub vcl_init {
new myset = re2.set();
myset.add("/foo", sub=resp_sub);
myset.add("/foo/bar", sub=some_other_sub);
# ...
}
sub vcl_deliver {
if (myset.match(req.url)) {
call myset.subroutine(select=FIRST);
}
}
$Method BOOL .check_call(INT n=0, ENUM {FIRST, LAST, UNIQUE} select=UNIQUE) $Method BOOL .check_call(INT n=0, ENUM {FIRST, LAST, UNIQUE} select=UNIQUE)
XXX ... Returns ``true`` iff the subroutine returned by ``.subroutine()`` for
the element of the set indicated by ``n`` and ``select`` may be
invoked legally in the current context. The conditions for legal
invocation are documented for ``.subroutine()`` above.
``.check_call()`` never invokes VCL failure, but rather returns
``false`` under conditions for which the use of ``.subroutine()``
would invoke VCL failure. In that case, a message is emitted to the
Vanrish log using the ``Notice`` tag (the same message that would
appear with the ``VCL_Error`` tag if the subroutine were called).
``Notice`` messages in the log produced by this VMOD are always
prefixed with the string ``vmod_re2: ``.
Example::
# Assume that myset is declared as in the example above.
sub vcl_deliver {
if (myset.match(req.url)) {
if (myset.check_call(select=FIRST)) {
call myset.subroutine(select=FIRST);
}
else {
call do_if_resp_sub_is_illegal;
}
}
}
$Method BOOL .saved(ENUM {REGEX, STR, BE, INT, SUB} which=REGEX, INT n=0, $Method BOOL .saved(ENUM {REGEX, STR, BE, INT, SUB} which=REGEX, INT n=0,
ENUM {FIRST, LAST, UNIQUE} select=UNIQUE) ENUM {FIRST, LAST, UNIQUE} select=UNIQUE)
...@@ -1427,12 +1473,14 @@ In other words, ``.saved()`` returns true: ...@@ -1427,12 +1473,14 @@ In other words, ``.saved()`` returns true:
* for ``which=INT`` if an integer was stored with the ``integer`` * for ``which=INT`` if an integer was stored with the ``integer``
attribute. attribute.
* for ``which=SUB`` if an integer was stored with the ``sub``
attribute.
The default value of ``which`` is ``REGEX``. The default value of ``which`` is ``REGEX``.
The pattern in the set is identified by ``n`` and ``select`` according The pattern in the set is identified by ``n`` and ``select`` according
to the rules given above. ``.saved()`` fails, returning false with a to the rules given above. ``.saved()`` invokes VCL failure if the
``VCL_Error`` message in the log, if the values of ``n`` or ``select`` values of ``n`` or ``select`` are invalid (see `ERRORS`_).
are invalid.
Example:: Example::
...@@ -1528,8 +1576,8 @@ any special characters. This function has a purpose similar to a ...@@ -1528,8 +1576,8 @@ any special characters. This function has a purpose similar to a
``\Q..\E`` sequence within a regex, or the ``literal=true`` setting in ``\Q..\E`` sequence within a regex, or the ``literal=true`` setting in
a regex constructor. a regex constructor.
The function fails and returns ``fallback`` if there is insufficient The function invokes VCL failure if there is insufficient workspace
workspace for the return string. for the return string (see `ERRORS`_).
Example:: Example::
...@@ -1545,15 +1593,87 @@ Example:: ...@@ -1545,15 +1593,87 @@ Example::
std.log("Using VMOD re2 version: " + re2.version()); std.log("Using VMOD re2 version: " + re2.version());
ERRORS
======
Functions and methods of the VMOD may invoke VCL failure under
unrecoverable error conditions. The effects of VCL failure depend on
the VCL subroutine in which it takes place:
* If invoked during ``vcl_init``, then the VCL load fails, and an
error message is returned over the CLI (reported by
``varnishadm(1)``).
* If invoked during any other subroutine besides ``vcl_synth``, then
an error message is recorded in the log with the ``VCL_Error`` tag,
further processing is aborted immediately, and a response with
status 503 (Service Not Available) is returned with the reason
string "VCL failed".
* If invoked during ``vcl_synth``, then further processing is aborted,
the error message is logged with ``VCL_Error``, and the client
connection is immediately closed -- the client receives no response.
Errors that lead to VCL failure include:
* Any regex compilation failure.
* Out of workspace errors (see `LIMITATIONS`_).
* Failures reported by the RE2 library for: matches, backrefs,
namedrefs, the rewrite operations (sub, suball and extract), the
``.cost()`` function or method, and the ``.quotemeta()``
function. The VMOD detects most common errors that would lead to
library errors, and invokes VCL failure in such cases without
calling the library. But library errors may happen under conditions
such as out of memory.
* Functions and methods that require a previous successful match when
there was no prior match, or when the previous match was
unsuccessful. These include backrefs, namedrefs, and the data
retrieval methods for set objects.
* Any of the following parameters are undefined, for example when
set from an unset header: fallbacks; patterns for the regex functions
(which are compiled at runtime); the text and rewrite parameters
for rewrite operations; the name parameter for namedrefs.
* The name parameter for namedrefs is the empty string.
* Backref number is out of range (greater than the number of backrefs
in the pattern).
* Backref or namedref attempted when the ``never_capture`` option was
set to ``true`` for the pattern.
* For set objects:
* Numeric index (parameter ``n``) is out of range (greater than the
number of patterns in the set).
* Use of ``select=UNIQUE`` after more than one pattern was matched.
The ``.nmatches()`` can be used to check for this condition, to
avoid VCL failure -- ``UNIQUE`` will fail in ``.namtches()`` > 1.
* Retrieval of data from a set (such as a string, backend etc) by
numeric index (``n``) or "associatively" (after a match) when no
such object was saved for the corresponding pattern. Use the
``.saved()`` and ``.check_call()`` methods to check for this.
* Calling the subroutine returned by ``.subrooutine()`` may be
illegal, if it is not permitted in the subroutine from which it is
called, or if it would lead to recursive calls. Use the
``.check_call()`` method to check for this.
REQUIREMENTS REQUIREMENTS
============ ============
The VMOD requires the Varnish master branch. See the source repository The VMOD requires Varnish since version 6.6, or the master branch. See
for versions of the VMOD that are compatible with other Varnish the source repository for versions of the VMOD that are compatible
versions. with other Varnish versions.
It requires the RE2 library, and has been tested against RE2 versions It requires the RE2 library, and has been tested against RE2 versions
since 2015-06-01 (through 2020-11-01 at the time of writing). since 2015-06-01 (through 2021-04-01 at the time of writing).
If the VMOD is built against versions of RE2 since 2017-12-01, it uses If the VMOD is built against versions of RE2 since 2017-12-01, it uses
a version of the set match operation that reports out-of-memory a version of the set match operation that reports out-of-memory
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment