Skip to content

L
libvmod-re
Commit 26f3394e authored by Geoff Simmons's avatar Geoff Simmons
1 Browse files
Options

Use PRIV_TASK instead of pthread keys; THIS CHANGES THE INTERFACE. 

The .match_dyn method has been removed, in its place we have the
functions match_dyn and backref_dyn.

Closes #1
parent b01518a1
Pipeline #200 skipped
Hide whitespace changes
Inline Side-by-side
Showing with 310 additions and 271 deletions
README.rst
View file @ 26f3394e
...@@ -59,7 +59,7 @@ The equivalent solution with the VMOD looks like this:: ...@@ -59,7 +59,7 @@ The equivalent solution with the VMOD looks like this::
sub vcl_recv { sub vcl_recv {
if (myregex.match(req.http.Foo)) { if (myregex.match(req.http.Foo)) {
set req.http.Baz = myregex.backref(1, ""); set req.http.Baz = myregex.backref(1);
} }
} }
...@@ -68,29 +68,34 @@ the capture expression, only describing the substring to be ...@@ -68,29 +68,34 @@ the capture expression, only describing the substring to be
matched. When a match with the ``match`` method succeeds, then a matched. When a match with the ``match`` method succeeds, then a
captured string can be obtained from the ``backref`` method. captured string can be obtained from the ``backref`` method.
Calls to the ``backref`` method refer back to the most recent
successful call to ``match`` for the same object in the same task
scope; that is, in the same client or backend context. For example if
``match`` is called for an object in one of the ``vcl_backend_*``
subroutines and returns ``true``, then subsequent calls to ``backref``
in the same backend scope extract substrings from the matched
substring.
The VMOD also supports dynamic regex matching with the ``match_dyn`` The VMOD also supports dynamic regex matching with the ``match_dyn``
method:: and ``backref_dyn`` functions::
import re; import re;
sub vcl_init {
new myregex = re.regex("");
}
sub vcl_backend_response { sub vcl_backend_response {
if (myregex.match_dyn(beresp.http.Bar + "(\d+)", if (re.match_dyn(beresp.http.Bar + "(\d+)",
req.http.Foo)) { req.http.Foo)) {
set beresp.http.Baz = myregex.backref(1, ""); set beresp.http.Baz = re.backref_dyn(1);
} }
} }
In ``match_dyn``, the regex in the first argument is compiled when it In ``match_dyn``, the regex in the first argument is compiled when it
is called, and matched against the string in the second argument; the is called, and matched against the string in the second
regex provided in ``vcl_init`` is ignored. Subsequent calls to argument. Subsequent calls to ``backref_dyn`` extract substrings from
``backref`` extract substrings from the matched string. the matched string for the most recent successful call to
``match_dyn`` in the same task scope.
As with the constructor, the regex argument to ``match_dyn`` should As with the constructor, the regex argument to ``match_dyn`` should
contain any capturing expressions needed for calls to ``backref``. contain any capturing expressions needed for calls to ``backref_dyn``.
``match_dyn`` makes it possible to construct regexen whose contents ``match_dyn`` makes it possible to construct regexen whose contents
are not fully known until runtime, but ``match`` is more efficient, are not fully known until runtime, but ``match`` is more efficient,
...@@ -102,6 +107,8 @@ CONTENTS ...@@ -102,6 +107,8 @@ CONTENTS
======== ========
* regex(STRING) * regex(STRING)
* BOOL match_dyn(PRIV_TASK, STRING, STRING)
* STRING backref_dyn(PRIV_TASK, INT, STRING)
* STRING version() * STRING version()
.. _obj_regex: .. _obj_regex:
...@@ -113,9 +120,6 @@ regex ...@@ -113,9 +120,6 @@ regex
new OBJ = regex(STRING) new OBJ = regex(STRING)
Prototype
new OBJ = re.regex(STRING)
Description Description
Create a regex object with the given regular expression. The Create a regex object with the given regular expression. The
expression is compiled when the constructor is called. It expression is compiled when the constructor is called. It
...@@ -177,24 +181,6 @@ Description ...@@ -177,24 +181,6 @@ Description
Example Example
``if (myregex.match(beresp.http.Surrogate-Control)) { # ...`` ``if (myregex.match(beresp.http.Surrogate-Control)) { # ...``
.. _func_regex.match_dyn:
regex.match_dyn
---------------
::
BOOL regex.match_dyn(STRING, STRING)
Description
Compiles the regular expression given in the first argument,
and determines whether it matches the string in the second
argument. The regex supplied in the constructor is ignored.
Example
``if (myregex.match_dyn(req.http.Foo + "(\d+)",``
``beresp.http.Bar)) { # ...``
.. _func_regex.backref: .. _func_regex.backref:
regex.backref regex.backref
...@@ -202,31 +188,72 @@ regex.backref ...@@ -202,31 +188,72 @@ regex.backref
:: ::
STRING regex.backref(INT, STRING) STRING regex.backref(INT, STRING fallback="**BACKREF METHOD FAILED**")
Description Description
Extracts the `nth` subexpression of the most recent successful Extracts the `nth` subexpression of the most recent successful
call of the ``match`` or ``match_dyn`` method for this object call of the ``match`` method for this object in the same task
in the same VCL subroutine call, or a fallback string in case scope (client or backend context), or a fallback string in
the extraction fails. Backref 0 indicates the entire matched case the extraction fails. Backref 0 indicates the entire
string. Thus this function behaves like the ``\n`` symbols in matched string. Thus this function behaves like the ``\n``
``regsub`` and ``regsuball``, and the ``$1``, ``$2`` ... symbols in ``regsub`` and ``regsuball``, and the ``$1``,
variables in Perl. ``$2`` ... variables in Perl.
After unsuccessful matches, the ``fallback`` string is returned After unsuccessful matches, the ``fallback`` string is returned
for any call to ``backref``. for any call to ``backref``. The default value of ``fallback``
is ``"**BACKREF METHOD FAILED**"``.
The VCL infix operators ``~`` and ``!~`` do not affect this The VCL infix operators ``~`` and ``!~`` do not affect this
method, nor do the functions ``regsub`` or ``regsuball``. method, nor do the functions ``regsub`` or ``regsuball``.
If ``backref`` is called without any prior call to ``match`` If ``backref`` is called without any prior call to ``match``
or ``match_dyn`` for this object in the same VCL context, then for this object in the same task scope, then an error message
an error message is emitted to the Varnish log using the is emitted to the Varnish log using the ``VCL_Error`` tag, and
``VCL_Error`` tag, and the fallback string is returned. the fallback string is returned.
Example Example
``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);`` ``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);``
.. _func_match_dyn:
match_dyn
---------
::
BOOL match_dyn(PRIV_TASK, STRING, STRING)
Description
Compiles the regular expression given in the first argument,
and determines whether it matches the string in the second
argument.
Example
``if (re.match_dyn(req.http.Foo + "(\d+)", beresp.http.Bar)) { # ...``
.. _func_backref_dyn:
backref_dyn
-----------
::
STRING backref_dyn(PRIV_TASK, INT, STRING fallback="**BACKREF FUNCTION FAILED**")
Description
Similar to the ``backref`` method, this function extracts the
`nth` subexpression of the most recent successful call of the
``match_dyn`` function in the same task scope, or a fallback
string in case the extraction fails.
After unsuccessful matches, the ``fallback`` string is returned
for any call to ``backref_dyn``. The default value of ``fallback``
is ``"**BACKREF FUNCTION FAILED**"``.
If ``backref_dyn`` is called without any prior call to ``match_dyn``
in the same task scope, then a ``VCL_Error`` message is logged, and
the fallback string is returned.
.. _func_version: .. _func_version:
version version
......
src/tests/b06.vtc
View file @ 26f3394e
...@@ -29,7 +29,7 @@ varnish v1 -vcl+backend { ...@@ -29,7 +29,7 @@ varnish v1 -vcl+backend {
set beresp.http.static = "nomatch"; set beresp.http.static = "nomatch";
} }
if (baz.match_dyn(beresp.http.foo, beresp.http.bar)) { if (re.match_dyn(beresp.http.foo, beresp.http.bar)) {
set beresp.http.dynamic = "match"; set beresp.http.dynamic = "match";
} }
else { else {
......
src/tests/c01.vtc
View file @ 26f3394e
...@@ -25,7 +25,7 @@ varnish v1 -vcl+backend { ...@@ -25,7 +25,7 @@ varnish v1 -vcl+backend {
set beresp.status = 999; set beresp.status = 999;
} }
if (barbaz.match_dyn("(bar)(baz)", beresp.http.foo)) { if (re.match_dyn("(bar)(baz)", beresp.http.foo)) {
set beresp.http.foo0d = barbaz.backref(0, "error0"); set beresp.http.foo0d = barbaz.backref(0, "error0");
set beresp.http.foo1d = barbaz.backref(1, "error1"); set beresp.http.foo1d = barbaz.backref(1, "error1");
set beresp.http.foo2d = barbaz.backref(2, "error2"); set beresp.http.foo2d = barbaz.backref(2, "error2");
...@@ -43,7 +43,7 @@ varnish v1 -vcl+backend { ...@@ -43,7 +43,7 @@ varnish v1 -vcl+backend {
set beresp.status = 999; set beresp.status = 999;
} }
if (bazplus.match_dyn("(baz)(.+)", beresp.http.bar)) { if (re.match_dyn("(baz)(.+)", beresp.http.bar)) {
set beresp.http.bar0d = bazplus.backref(0, "error0"); set beresp.http.bar0d = bazplus.backref(0, "error0");
set beresp.http.bar1d = bazplus.backref(1, "error1"); set beresp.http.bar1d = bazplus.backref(1, "error1");
set beresp.http.bar2d = bazplus.backref(2, "error2"); set beresp.http.bar2d = bazplus.backref(2, "error2");
...@@ -66,8 +66,8 @@ varnish v1 -vcl+backend { ...@@ -66,8 +66,8 @@ varnish v1 -vcl+backend {
set beresp.status = 999; set beresp.status = 999;
} }
if (barbaz.match_dyn("(bar)(baz)", beresp.http.foo) if (re.match_dyn("(bar)(baz)", beresp.http.foo)
&& bazplus.match_dyn("(baz)(.+)", beresp.http.bar)) { && re.match_dyn("(baz)(.+)", beresp.http.bar)) {
set beresp.http.foo20d = barbaz.backref(0, "error0"); set beresp.http.foo20d = barbaz.backref(0, "error0");
set beresp.http.foo21d = barbaz.backref(1, "error1"); set beresp.http.foo21d = barbaz.backref(1, "error1");
set beresp.http.foo22d = barbaz.backref(2, "error2"); set beresp.http.foo22d = barbaz.backref(2, "error2");
...@@ -91,7 +91,7 @@ varnish v1 -vcl+backend { ...@@ -91,7 +91,7 @@ varnish v1 -vcl+backend {
set beresp.status = 999; set beresp.status = 999;
} }
if (fourdots.match_dyn("(.)(.)(.)(.)", beresp.http.barf)) { if (re.match_dyn("(.)(.)(.)(.)", beresp.http.barf)) {
set beresp.http.frapd set beresp.http.frapd
= "_" + fourdots.backref(0, "error0") + "_" = "_" + fourdots.backref(0, "error0") + "_"
+ fourdots.backref(5, "") + fourdots.backref(5, "")
......
src/tests/c03.vtc
View file @ 26f3394e
...@@ -11,49 +11,44 @@ server s1 { ...@@ -11,49 +11,44 @@ server s1 {
varnish v1 -vcl+backend { varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so"; import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_init {
new frobnitz = re.regex("");
new barbaz = re.regex("");
}
sub vcl_deliver { sub vcl_deliver {
set resp.http.nomatch = barbaz.backref(0, "fallback"); set resp.http.nomatch = re.backref_dyn(0, "fallback");
if (frobnitz.match_dyn("(frob)(nitz)", resp.http.foo)) { if (re.match_dyn("(frob)(nitz)", resp.http.foo)) {
set resp.http.frob = "nitz"; set resp.http.frob = "nitz";
} }
set resp.http.frob0 = frobnitz.backref(0, "fallback0"); set resp.http.frob0 = re.backref_dyn(0, "fallback0");
set resp.http.frob1 = frobnitz.backref(1, "fallback1"); set resp.http.frob1 = re.backref_dyn(1, "fallback1");
set resp.http.frob2 = frobnitz.backref(2, "fallback2"); set resp.http.frob2 = re.backref_dyn(2, "fallback2");
set resp.http.frob3 = frobnitz.backref(3, "fallback3"); set resp.http.frob3 = re.backref_dyn(3, "fallback3");
set resp.http.frob4 = frobnitz.backref(4, "fallback4"); set resp.http.frob4 = re.backref_dyn(4, "fallback4");
set resp.http.frob5 = frobnitz.backref(5, "fallback5"); set resp.http.frob5 = re.backref_dyn(5, "fallback5");
set resp.http.frob6 = frobnitz.backref(6, "fallback6"); set resp.http.frob6 = re.backref_dyn(6, "fallback6");
set resp.http.frob7 = frobnitz.backref(7, "fallback7"); set resp.http.frob7 = re.backref_dyn(7, "fallback7");
set resp.http.frob8 = frobnitz.backref(8, "fallback8"); set resp.http.frob8 = re.backref_dyn(8, "fallback8");
set resp.http.frob9 = frobnitz.backref(9, "fallback9"); set resp.http.frob9 = re.backref_dyn(9, "fallback9");
set resp.http.frob10 = frobnitz.backref(10, "fallback10"); set resp.http.frob10 = re.backref_dyn(10, "fallback10");
if (barbaz.match_dyn("(bar)(baz)", resp.http.foo)) { if (re.match_dyn("(bar)(baz)", resp.http.foo)) {
set resp.http.foo0 = barbaz.backref(0, "error0"); set resp.http.foo0 = re.backref_dyn(0, "error0");
set resp.http.foo1 = barbaz.backref(1, "error1"); set resp.http.foo1 = re.backref_dyn(1, "error1");
set resp.http.foo2 = barbaz.backref(2, "error2"); set resp.http.foo2 = re.backref_dyn(2, "error2");
set resp.http.foo3 = barbaz.backref(3, "foofallback"); set resp.http.foo3 = re.backref_dyn(3, "foofallback");
} }
if (barbaz.match_dyn("(bar)(baz)", resp.http.barf)) { if (re.match_dyn("(bar)(baz)", resp.http.barf)) {
set resp.http.puke = "match"; set resp.http.puke = "match";
} }
set resp.http.barf0 = barbaz.backref(0, "fallback0"); set resp.http.barf0 = re.backref_dyn(0, "fallback0");
set resp.http.barf1 = barbaz.backref(1, "fallback1"); set resp.http.barf1 = re.backref_dyn(1, "fallback1");
set resp.http.barf2 = barbaz.backref(2, "fallback2"); set resp.http.barf2 = re.backref_dyn(2, "fallback2");
set resp.http.barf3 = barbaz.backref(3, "fallback3"); set resp.http.barf3 = re.backref_dyn(3, "fallback3");
set resp.http.barf4 = barbaz.backref(4, "fallback4"); set resp.http.barf4 = re.backref_dyn(4, "fallback4");
set resp.http.barf5 = barbaz.backref(5, "fallback5"); set resp.http.barf5 = re.backref_dyn(5, "fallback5");
set resp.http.barf6 = barbaz.backref(6, "fallback6"); set resp.http.barf6 = re.backref_dyn(6, "fallback6");
set resp.http.barf7 = barbaz.backref(7, "fallback7"); set resp.http.barf7 = re.backref_dyn(7, "fallback7");
set resp.http.barf8 = barbaz.backref(8, "fallback8"); set resp.http.barf8 = re.backref_dyn(8, "fallback8");
set resp.http.barf9 = barbaz.backref(9, "fallback9"); set resp.http.barf9 = re.backref_dyn(9, "fallback9");
set resp.http.barf10 = barbaz.backref(10, "fallback10"); set resp.http.barf10 = re.backref_dyn(10, "fallback10");
} }
} -start } -start
......
src/tests/c04.vtc
View file @ 26f3394e
...@@ -21,10 +21,10 @@ varnish v1 -vcl+backend { ...@@ -21,10 +21,10 @@ varnish v1 -vcl+backend {
set beresp.status = 999; set beresp.status = 999;
} }
if (barbaz.match_dyn("(?:bar)(baz)", beresp.http.foo)) { if (re.match_dyn("(?:bar)(baz)", beresp.http.foo)) {
set beresp.http.foo0d = barbaz.backref(0, "error0"); set beresp.http.foo0d = re.backref_dyn(0, "error0");
set beresp.http.foo1d = barbaz.backref(1, "error1"); set beresp.http.foo1d = re.backref_dyn(1, "error1");
set beresp.http.foo2d = barbaz.backref(2, "fallback"); set beresp.http.foo2d = re.backref_dyn(2, "fallback");
} else { } else {
set beresp.status = 999; set beresp.status = 999;
} }
......
src/tests/c06.vtc
View file @ 26f3394e
...@@ -10,43 +10,38 @@ server s1 { ...@@ -10,43 +10,38 @@ server s1 {
varnish v1 -vcl+backend { varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so"; import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_init {
new tendots = re.regex("");
new moredots = re.regex("");
}
sub vcl_backend_response { sub vcl_backend_response {
if (tendots.match_dyn("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)", if (re.match_dyn("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)",
beresp.http.foo)) { beresp.http.foo)) {
set beresp.http.foo0 = tendots.backref(0, "error0"); set beresp.http.foo0 = re.backref_dyn(0, "error0");
set beresp.http.foo1 = tendots.backref(1, "error1"); set beresp.http.foo1 = re.backref_dyn(1, "error1");
set beresp.http.foo2 = tendots.backref(2, "error2"); set beresp.http.foo2 = re.backref_dyn(2, "error2");
set beresp.http.foo3 = tendots.backref(3, "error3"); set beresp.http.foo3 = re.backref_dyn(3, "error3");
set beresp.http.foo4 = tendots.backref(4, "error4"); set beresp.http.foo4 = re.backref_dyn(4, "error4");
set beresp.http.foo5 = tendots.backref(5, "error5"); set beresp.http.foo5 = re.backref_dyn(5, "error5");
set beresp.http.foo6 = tendots.backref(6, "error6"); set beresp.http.foo6 = re.backref_dyn(6, "error6");
set beresp.http.foo7 = tendots.backref(7, "error7"); set beresp.http.foo7 = re.backref_dyn(7, "error7");
set beresp.http.foo8 = tendots.backref(8, "error8"); set beresp.http.foo8 = re.backref_dyn(8, "error8");
set beresp.http.foo9 = tendots.backref(9, "error9"); set beresp.http.foo9 = re.backref_dyn(9, "error9");
set beresp.http.foo10 = tendots.backref(10, "error10"); set beresp.http.foo10 = re.backref_dyn(10, "error10");
set beresp.http.foo11 = tendots.backref(11, "fallback"); set beresp.http.foo11 = re.backref_dyn(11, "fallback");
} else { } else {
set beresp.status = 999; set beresp.status = 999;
} }
if (moredots.match_dyn("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)", if (re.match_dyn("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)",
beresp.http.bar)) { beresp.http.bar)) {
set beresp.http.bar0 = moredots.backref(0, "error0"); set beresp.http.bar0 = re.backref_dyn(0, "error0");
set beresp.http.bar1 = moredots.backref(1, "error1"); set beresp.http.bar1 = re.backref_dyn(1, "error1");
set beresp.http.bar2 = moredots.backref(2, "error2"); set beresp.http.bar2 = re.backref_dyn(2, "error2");
set beresp.http.bar3 = moredots.backref(3, "error3"); set beresp.http.bar3 = re.backref_dyn(3, "error3");
set beresp.http.bar4 = moredots.backref(4, "error4"); set beresp.http.bar4 = re.backref_dyn(4, "error4");
set beresp.http.bar5 = moredots.backref(5, "error5"); set beresp.http.bar5 = re.backref_dyn(5, "error5");
set beresp.http.bar6 = moredots.backref(6, "error6"); set beresp.http.bar6 = re.backref_dyn(6, "error6");
set beresp.http.bar7 = moredots.backref(7, "error7"); set beresp.http.bar7 = re.backref_dyn(7, "error7");
set beresp.http.bar8 = moredots.backref(8, "error8"); set beresp.http.bar8 = re.backref_dyn(8, "error8");
set beresp.http.bar9 = moredots.backref(9, "error9"); set beresp.http.bar9 = re.backref_dyn(9, "error9");
set beresp.http.bar10 = moredots.backref(10, "error10"); set beresp.http.bar10 = re.backref_dyn(10, "error10");
set beresp.http.bar11 = moredots.backref(11, "fallback"); set beresp.http.bar11 = re.backref_dyn(11, "fallback");
} else { } else {
set beresp.status = 999; set beresp.status = 999;
} }
......
src/tests/c07.vtc
View file @ 26f3394e
...@@ -41,33 +41,33 @@ varnish v1 -vcl+backend { ...@@ -41,33 +41,33 @@ varnish v1 -vcl+backend {
set beresp.http.regsuball0 = barbaz.backref(0, "regsuball0"); set beresp.http.regsuball0 = barbaz.backref(0, "regsuball0");
set beresp.http.regsuball1 = barbaz.backref(1, "regsuball1"); set beresp.http.regsuball1 = barbaz.backref(1, "regsuball1");
if (!barbaz.match_dyn("(bar)baz", beresp.http.foo)) { if (!re.match_dyn("(bar)baz", beresp.http.foo)) {
set beresp.status = 999; set beresp.status = 999;
} }
if (beresp.http.foo ~ "bar(baz)") { if (beresp.http.foo ~ "bar(baz)") {
set beresp.http.tilde0d = barbaz.backref(0, "tilde0d"); set beresp.http.tilde0d = re.backref_dyn(0, "tilde0d");
set beresp.http.tilde1d = barbaz.backref(1, "tilde1d"); set beresp.http.tilde1d = re.backref_dyn(1, "tilde1d");
} else { } else {
set beresp.status = 999; set beresp.status = 999;
} }
if (beresp.http.foo !~ "bar(quux)") { if (beresp.http.foo !~ "bar(quux)") {
set beresp.http.neg0d = barbaz.backref(0, "neg0d"); set beresp.http.neg0d = re.backref_dyn(0, "neg0d");
set beresp.http.neg1d = barbaz.backref(1, "neg1d"); set beresp.http.neg1d = re.backref_dyn(1, "neg1d");
} else { } else {
set beresp.status = 999; set beresp.status = 999;
} }
set beresp.http.regsubd set beresp.http.regsubd
= regsub(beresp.http.foo, "bar(baz)", "\1"); = regsub(beresp.http.foo, "bar(baz)", "\1");
set beresp.http.regsub0d = barbaz.backref(0, "regsub0d"); set beresp.http.regsub0d = re.backref_dyn(0, "regsub0d");
set beresp.http.regsub1d = barbaz.backref(1, "regsub1d"); set beresp.http.regsub1d = re.backref_dyn(1, "regsub1d");
set beresp.http.regsuballd set beresp.http.regsuballd
= regsuball(beresp.http.foo, "(.)", "x"); = regsuball(beresp.http.foo, "(.)", "x");
set beresp.http.regsuball0d = barbaz.backref(0, "regsuball0d"); set beresp.http.regsuball0d = re.backref_dyn(0, "regsuball0d");
set beresp.http.regsuball1d = barbaz.backref(1, "regsuball1d"); set beresp.http.regsuball1d = re.backref_dyn(1, "regsuball1d");
} }
} -start } -start
......
src/tests/c08.vtc
View file @ 26f3394e
...@@ -34,28 +34,27 @@ varnish v1 -vcl+backend { ...@@ -34,28 +34,27 @@ varnish v1 -vcl+backend {
set resp.http.paren9 = paren.backref(9, "fallback9"); set resp.http.paren9 = paren.backref(9, "fallback9");
set resp.http.paren10 = paren.backref(10, "fallback10"); set resp.http.paren10 = paren.backref(10, "fallback10");
if (paren.match_dyn("(bar)", resp.http.foo)) { if (!re.match_dyn("(bar)", resp.http.foo)) {
# dynamic matches do not affect failed() or error() set resp.status = 999;
set resp.http.parend = paren.failed(); return(deliver);
set resp.http.errord = paren.error();
} }
/* compilation fails */ /* compilation fails */
if (parend.match_dyn(")", resp.http.foo)) { if (re.match_dyn(")", resp.http.foo)) {
set resp.http.matchd = "success"; set resp.http.matchd = "success";
} }
/* ... so all backrefs fail */ /* ... so all backrefs fail */
set resp.http.paren0d = parend.backref(0, "fallback0"); set resp.http.paren0d = re.backref_dyn(0, "fallback0");
set resp.http.paren1d = parend.backref(1, "fallback1"); set resp.http.paren1d = re.backref_dyn(1, "fallback1");
set resp.http.paren2d = parend.backref(2, "fallback2"); set resp.http.paren2d = re.backref_dyn(2, "fallback2");
set resp.http.paren3d = parend.backref(3, "fallback3"); set resp.http.paren3d = re.backref_dyn(3, "fallback3");
set resp.http.paren4d = parend.backref(4, "fallback4"); set resp.http.paren4d = re.backref_dyn(4, "fallback4");
set resp.http.paren5d = parend.backref(5, "fallback5"); set resp.http.paren5d = re.backref_dyn(5, "fallback5");
set resp.http.paren6d = parend.backref(6, "fallback6"); set resp.http.paren6d = re.backref_dyn(6, "fallback6");
set resp.http.paren7d = parend.backref(7, "fallback7"); set resp.http.paren7d = re.backref_dyn(7, "fallback7");
set resp.http.paren8d = parend.backref(8, "fallback8"); set resp.http.paren8d = re.backref_dyn(8, "fallback8");
set resp.http.paren9d = parend.backref(9, "fallback9"); set resp.http.paren9d = re.backref_dyn(9, "fallback9");
set resp.http.paren10d = parend.backref(10, "fallback10"); set resp.http.paren10d = re.backref_dyn(10, "fallback10");
} }
} -start } -start
...@@ -80,10 +79,7 @@ client c1 { ...@@ -80,10 +79,7 @@ client c1 {
expect resp.http.paren9 == "fallback9" expect resp.http.paren9 == "fallback9"
expect resp.http.paren10 == "fallback10" expect resp.http.paren10 == "fallback10"
expect resp.http.parend == "true"
expect resp.http.matchd == <undef> expect resp.http.matchd == <undef>
expect resp.http.errord != <undef>
expect resp.http.errord != ""
expect resp.http.paren0d == "fallback0" expect resp.http.paren0d == "fallback0"
expect resp.http.paren1d == "fallback1" expect resp.http.paren1d == "fallback1"
expect resp.http.paren2d == "fallback2" expect resp.http.paren2d == "fallback2"
...@@ -106,18 +102,5 @@ logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" { ...@@ -106,18 +102,5 @@ logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
# from match_dyn() # from match_dyn()
expect * = VCL_Error "^vmod re: error compiling regex" expect * = VCL_Error "^vmod re: error compiling regex"
# from the backref() calls after match_dyn()
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = VCL_Error "^vmod re: backref called without prior match$"
expect * = End expect * = End
} -run } -run
src/vmod_re.c
View file @ 26f3394e
...@@ -30,7 +30,6 @@ ...@@ -30,7 +30,6 @@
#include "config.h" #include "config.h"
#include <stdlib.h> #include <stdlib.h>
#include <pthread.h>
#include "vre.h" #include "vre.h"
#include "cache/cache.h" #include "cache/cache.h"
...@@ -60,7 +59,6 @@ struct vmod_re_regex { ...@@ -60,7 +59,6 @@ struct vmod_re_regex {
unsigned magic; unsigned magic;
#define VMOD_RE_REGEX_MAGIC 0x955706ee #define VMOD_RE_REGEX_MAGIC 0x955706ee
vre_t *vre; vre_t *vre;
pthread_key_t ovk;
int erroffset; int erroffset;
const char *error; const char *error;
}; };
...@@ -76,8 +74,8 @@ static char c; ...@@ -76,8 +74,8 @@ static char c;
static const void *match_failed = (void *) &c; static const void *match_failed = (void *) &c;
VCL_VOID VCL_VOID
vmod_regex__init(const struct vrt_ctx *ctx, struct vmod_re_regex **rep, vmod_regex__init(VRT_CTX, struct vmod_re_regex **rep, const char *vcl_name,
const char *vcl_name, VCL_STRING pattern) VCL_STRING pattern)
{ {
struct vmod_re_regex *re; struct vmod_re_regex *re;
...@@ -93,7 +91,6 @@ vmod_regex__init(const struct vrt_ctx *ctx, struct vmod_re_regex **rep, ...@@ -93,7 +91,6 @@ vmod_regex__init(const struct vrt_ctx *ctx, struct vmod_re_regex **rep,
AN(re); AN(re);
*rep = re; *rep = re;
AZ(pthread_key_create(&re->ovk, NULL));
re->erroffset = 0; re->erroffset = 0;
re->error = NULL; re->error = NULL;
...@@ -107,7 +104,7 @@ vmod_regex__init(const struct vrt_ctx *ctx, struct vmod_re_regex **rep, ...@@ -107,7 +104,7 @@ vmod_regex__init(const struct vrt_ctx *ctx, struct vmod_re_regex **rep,
} }
VCL_BOOL VCL_BOOL
vmod_regex_failed(const struct vrt_ctx *ctx, struct vmod_re_regex *re) vmod_regex_failed(VRT_CTX, struct vmod_re_regex *re)
{ {
(void) ctx; (void) ctx;
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC); CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
...@@ -115,7 +112,7 @@ vmod_regex_failed(const struct vrt_ctx *ctx, struct vmod_re_regex *re) ...@@ -115,7 +112,7 @@ vmod_regex_failed(const struct vrt_ctx *ctx, struct vmod_re_regex *re)
} }
VCL_STRING VCL_STRING
vmod_regex_error(const struct vrt_ctx *ctx, struct vmod_re_regex *re) vmod_regex_error(VRT_CTX, struct vmod_re_regex *re)
{ {
VCL_STRING error; VCL_STRING error;
...@@ -142,7 +139,6 @@ vmod_regex__fini(struct vmod_re_regex **rep) ...@@ -142,7 +139,6 @@ vmod_regex__fini(struct vmod_re_regex **rep)
return; return;
re = *rep; re = *rep;
*rep = NULL; *rep = NULL;
AZ(pthread_key_delete(re->ovk));
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC); CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
if (re->vre != NULL) if (re->vre != NULL)
VRE_free(&re->vre); VRE_free(&re->vre);
...@@ -150,34 +146,31 @@ vmod_regex__fini(struct vmod_re_regex **rep) ...@@ -150,34 +146,31 @@ vmod_regex__fini(struct vmod_re_regex **rep)
} }
static VCL_BOOL static VCL_BOOL
match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, vre_t *vre, match(VRT_CTX, vre_t *vre, VCL_STRING subject, struct vmod_priv *task)
VCL_STRING subject)
{ {
ov_t *ov; ov_t *ov;
int s, nov[MAX_OV]; int s, nov[MAX_OV];
uintptr_t snap; uintptr_t snap;
size_t cp; size_t cp;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); AN(vre);
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
if (vre == NULL)
vre = re->vre;
AZ(pthread_setspecific(re->ovk, match_failed));
/* compilation error at init time */
if (vre == NULL) {
AN(re->error);
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: error compiling regex: %s (position %d)",
re->error, re->erroffset);
return 0;
}
if (subject == NULL) if (subject == NULL)
subject = ""; subject = "";
if (task->priv == NULL) {
if ((task->priv = WS_Alloc(ctx->ws, sizeof(*ov))) == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error, "vmod re error: "
"insufficient workspace for backref data");
return 0;
}
task->len = -1;
task->free = NULL;
ov = (ov_t *) task->priv;
ov->magic = OV_MAGIC;
}
else
CAST_OBJ(ov, task->priv, OV_MAGIC);
/* XXX: cache_param->vre_limits incorrect?! */ /* XXX: cache_param->vre_limits incorrect?! */
s = VRE_exec(vre, subject, strlen(subject), 0, 0, nov, MAX_OV, s = VRE_exec(vre, subject, strlen(subject), 0, 0, nov, MAX_OV,
NULL); NULL);
...@@ -197,13 +190,8 @@ match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, vre_t *vre, ...@@ -197,13 +190,8 @@ match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, vre_t *vre,
s = MAX_MATCHES; s = MAX_MATCHES;
} }
task->len = sizeof(*ov);
snap = WS_Snapshot(ctx->ws); snap = WS_Snapshot(ctx->ws);
ov = (ov_t *) WS_Alloc(ctx->ws, sizeof(ov_t));
if (ov == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: insufficient workspace");
return 0;
}
ov->subject = WS_Copy(ctx->ws, (const void *) subject, -1); ov->subject = WS_Copy(ctx->ws, (const void *) subject, -1);
if (ov->subject == NULL) { if (ov->subject == NULL) {
WS_Reset(ctx->ws, snap); WS_Reset(ctx->ws, snap);
...@@ -211,71 +199,39 @@ match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, vre_t *vre, ...@@ -211,71 +199,39 @@ match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, vre_t *vre,
"vmod re: insufficient workspace"); "vmod re: insufficient workspace");
return 0; return 0;
} }
ov->magic = OV_MAGIC;
memset(ov->ovector, -1, sizeof(ov->ovector)); memset(ov->ovector, -1, sizeof(ov->ovector));
cp = s * 2 * sizeof(*nov); cp = s * 2 * sizeof(*nov);
assert(cp <= sizeof(nov)); assert(cp <= sizeof(nov));
memcpy(ov->ovector, nov, cp); memcpy(ov->ovector, nov, cp);
AZ(pthread_setspecific(re->ovk, (const void *) ov));
return 1; return 1;
} }
VCL_BOOL static VCL_STRING
vmod_regex_match(const struct vrt_ctx *ctx, struct vmod_re_regex *re, backref(VRT_CTX, VCL_INT refnum, VCL_STRING fallback, struct vmod_priv *task)
VCL_STRING subject)
{ {
return match(ctx, re, NULL, subject);
}
VCL_BOOL
vmod_regex_match_dyn(const struct vrt_ctx *ctx, struct vmod_re_regex *re,
VCL_STRING pattern, VCL_STRING subject)
{
vre_t *vre;
int erroffset;
const char *error;
AN(pattern);
vre = VRE_compile(pattern, 0, &error, &erroffset);
if (vre == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: error compiling regex \"%s\": %s (position %d)",
pattern, error, erroffset);
return 0;
}
return match(ctx, re, vre, subject);
}
VCL_STRING
vmod_regex_backref(const struct vrt_ctx *ctx, struct vmod_re_regex *re,
VCL_INT refnum, VCL_STRING fallback)
{
void *p;
ov_t *ov; ov_t *ov;
char *substr; char *substr;
int len; int len;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
AN(fallback); AN(fallback);
assert(refnum >= 0); AN(task);
if (refnum >= MAX_MATCHES) { if (refnum < 0 || refnum >= MAX_MATCHES) {
VSLb(ctx->vsl, SLT_VCL_Error, VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: backref %ld out of range", refnum); "vmod re: backref %ld out of range", refnum);
return fallback; return fallback;
} }
p = pthread_getspecific(re->ovk); if (task->priv == NULL) {
if (p == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error, VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: backref called without prior match"); "vmod re: backref called without prior match");
return fallback; return fallback;
} }
if (p == match_failed) if (task->len <= 0)
return fallback; return fallback;
CAST_OBJ(ov, p, OV_MAGIC); CAST_OBJ(ov, task->priv, OV_MAGIC);
assert((char *)ov >= ctx->ws->s && (char *)ov < ctx->ws->e); assert((char *)ov >= ctx->ws->s && (char *)ov < ctx->ws->e);
assert(ov->subject >= ctx->ws->s && ov->subject < ctx->ws->e); assert(ov->subject >= ctx->ws->s && ov->subject < ctx->ws->e);
...@@ -296,8 +252,72 @@ vmod_regex_backref(const struct vrt_ctx *ctx, struct vmod_re_regex *re, ...@@ -296,8 +252,72 @@ vmod_regex_backref(const struct vrt_ctx *ctx, struct vmod_re_regex *re,
return substr; return substr;
} }
VCL_BOOL
vmod_regex_match(VRT_CTX, struct vmod_re_regex *re, VCL_STRING subject)
{
struct vmod_priv *task;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
/* compilation error at init time */
if (re->vre == NULL) {
AN(re->error);
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: error compiling regex: %s (position %d)",
re->error, re->erroffset);
return 0;
}
task = VRT_priv_task(ctx, re);
AN(task);
task->len = 0;
return match(ctx, re->vre, subject, task);
}
VCL_STRING
vmod_regex_backref(VRT_CTX, struct vmod_re_regex *re, VCL_INT refnum,
VCL_STRING fallback)
{
struct vmod_priv *task;
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
task = VRT_priv_task(ctx, re);
return backref(ctx, refnum, fallback, task);
}
VCL_BOOL
vmod_match_dyn(VRT_CTX, struct vmod_priv *task, VCL_STRING pattern,
VCL_STRING subject)
{
vre_t *vre;
int erroffset;
const char *error;
AN(pattern);
AN(task);
task->len = 0;
vre = VRE_compile(pattern, 0, &error, &erroffset);
if (vre == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: error compiling regex \"%s\": %s (position %d)",
pattern, error, erroffset);
return 0;
}
return match(ctx, vre, subject, task);
}
VCL_STRING
vmod_backref_dyn(VRT_CTX, struct vmod_priv *task, VCL_INT refnum,
VCL_STRING fallback)
{
return backref(ctx, refnum, fallback, task);
}
VCL_STRING VCL_STRING
vmod_version(const struct vrt_ctx *ctx __attribute__((unused))) vmod_version(VRT_CTX __attribute__((unused)))
{ {
return VERSION; return VERSION;
} }
src/vmod_re.vcc
View file @ 26f3394e
...@@ -44,7 +44,7 @@ The equivalent solution with the VMOD looks like this:: ...@@ -44,7 +44,7 @@ The equivalent solution with the VMOD looks like this::
sub vcl_recv { sub vcl_recv {
if (myregex.match(req.http.Foo)) { if (myregex.match(req.http.Foo)) {
set req.http.Baz = myregex.backref(1, ""); set req.http.Baz = myregex.backref(1);
} }
} }
...@@ -53,29 +53,34 @@ the capture expression, only describing the substring to be ...@@ -53,29 +53,34 @@ the capture expression, only describing the substring to be
matched. When a match with the ``match`` method succeeds, then a matched. When a match with the ``match`` method succeeds, then a
captured string can be obtained from the ``backref`` method. captured string can be obtained from the ``backref`` method.
Calls to the ``backref`` method refer back to the most recent
successful call to ``match`` for the same object in the same task
scope; that is, in the same client or backend context. For example if
``match`` is called for an object in one of the ``vcl_backend_*``
subroutines and returns ``true``, then subsequent calls to ``backref``
in the same backend scope extract substrings from the matched
substring.
The VMOD also supports dynamic regex matching with the ``match_dyn`` The VMOD also supports dynamic regex matching with the ``match_dyn``
method:: and ``backref_dyn`` functions::
import re; import re;
sub vcl_init {
new myregex = re.regex("");
}
sub vcl_backend_response { sub vcl_backend_response {
if (myregex.match_dyn(beresp.http.Bar + "(\d+)", if (re.match_dyn(beresp.http.Bar + "(\d+)",
req.http.Foo)) { req.http.Foo)) {
set beresp.http.Baz = myregex.backref(1, ""); set beresp.http.Baz = re.backref_dyn(1);
} }
} }
In ``match_dyn``, the regex in the first argument is compiled when it In ``match_dyn``, the regex in the first argument is compiled when it
is called, and matched against the string in the second argument; the is called, and matched against the string in the second
regex provided in ``vcl_init`` is ignored. Subsequent calls to argument. Subsequent calls to ``backref_dyn`` extract substrings from
``backref`` extract substrings from the matched string. the matched string for the most recent successful call to
``match_dyn`` in the same task scope.
As with the constructor, the regex argument to ``match_dyn`` should As with the constructor, the regex argument to ``match_dyn`` should
contain any capturing expressions needed for calls to ``backref``. contain any capturing expressions needed for calls to ``backref_dyn``.
``match_dyn`` makes it possible to construct regexen whose contents ``match_dyn`` makes it possible to construct regexen whose contents
are not fully known until runtime, but ``match`` is more efficient, are not fully known until runtime, but ``match`` is more efficient,
...@@ -85,9 +90,6 @@ never changes during the lifetime of VCL, use ``match``. ...@@ -85,9 +90,6 @@ never changes during the lifetime of VCL, use ``match``.
$Object regex(STRING) $Object regex(STRING)
Prototype
new OBJ = re.regex(STRING)
Description Description
Create a regex object with the given regular expression. The Create a regex object with the given regular expression. The
expression is compiled when the constructor is called. It expression is compiled when the constructor is called. It
...@@ -128,42 +130,59 @@ Description ...@@ -128,42 +130,59 @@ Description
Example Example
``if (myregex.match(beresp.http.Surrogate-Control)) { # ...`` ``if (myregex.match(beresp.http.Surrogate-Control)) { # ...``
$Method BOOL .match_dyn(STRING, STRING) $Method STRING .backref(INT, STRING fallback="**BACKREF METHOD FAILED**")
Description
Compiles the regular expression given in the first argument,
and determines whether it matches the string in the second
argument. The regex supplied in the constructor is ignored.
Example
``if (myregex.match_dyn(req.http.Foo + "(\d+)",``
``beresp.http.Bar)) { # ...``
$Method STRING .backref(INT, STRING)
Description Description
Extracts the `nth` subexpression of the most recent successful Extracts the `nth` subexpression of the most recent successful
call of the ``match`` or ``match_dyn`` method for this object call of the ``match`` method for this object in the same task
in the same VCL subroutine call, or a fallback string in case scope (client or backend context), or a fallback string in
the extraction fails. Backref 0 indicates the entire matched case the extraction fails. Backref 0 indicates the entire
string. Thus this function behaves like the ``\n`` symbols in matched string. Thus this function behaves like the ``\n``
``regsub`` and ``regsuball``, and the ``$1``, ``$2`` ... symbols in ``regsub`` and ``regsuball``, and the ``$1``,
variables in Perl. ``$2`` ... variables in Perl.
After unsuccessful matches, the ``fallback`` string is returned After unsuccessful matches, the ``fallback`` string is returned
for any call to ``backref``. for any call to ``backref``. The default value of ``fallback``
is ``"**BACKREF METHOD FAILED**"``.
The VCL infix operators ``~`` and ``!~`` do not affect this The VCL infix operators ``~`` and ``!~`` do not affect this
method, nor do the functions ``regsub`` or ``regsuball``. method, nor do the functions ``regsub`` or ``regsuball``.
If ``backref`` is called without any prior call to ``match`` If ``backref`` is called without any prior call to ``match``
or ``match_dyn`` for this object in the same VCL context, then for this object in the same task scope, then an error message
an error message is emitted to the Varnish log using the is emitted to the Varnish log using the ``VCL_Error`` tag, and
``VCL_Error`` tag, and the fallback string is returned. the fallback string is returned.
Example Example
``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);`` ``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);``
$Function BOOL match_dyn(PRIV_TASK, STRING, STRING)
Description
Compiles the regular expression given in the first argument,
and determines whether it matches the string in the second
argument.
Example
``if (re.match_dyn(req.http.Foo + "(\d+)", beresp.http.Bar)) { # ...``
$Function STRING backref_dyn(PRIV_TASK, INT,
STRING fallback="**BACKREF FUNCTION FAILED**")
Description
Similar to the ``backref`` method, this function extracts the
`nth` subexpression of the most recent successful call of the
``match_dyn`` function in the same task scope, or a fallback
string in case the extraction fails.
After unsuccessful matches, the ``fallback`` string is returned
for any call to ``backref_dyn``. The default value of ``fallback``
is ``"**BACKREF FUNCTION FAILED**"``.
If ``backref_dyn`` is called without any prior call to ``match_dyn``
in the same task scope, then a ``VCL_Error`` message is logged, and
the fallback string is returned.
$Function STRING version() $Function STRING version()
Description Description
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment