Skip to content

L
libvmod-re2
Commit c70e410a authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Breaking changes to invoke VCL failure for "can't happen" errors. 

Up to now we have emitted sentinel values for this class of errors,
a relic of the time before VCL failure was introduced, when the VMOD
was first developed. This may lead to fatal errors going unnoticed.
These are the sorts of errors that should fail fast in development
and testing, and never make it into production.

We were already using VCL failure for the integer() method, since
there is no sentinel integer.

VCL failure is now invoked for these errors:

- any regex compilation failure

- all out of workspace errors

- compile() or add() called in any VCL subroutine besides vcl_init

- match() called for a set that was not compiled

- failures reported by the RE2 lib for: match(), backref(),
  namedref(), the rewrite operations (sub, suball and extract),
  cost(), quotemeta()

- numeric index reference for a set object (n parameter) that is out
  of range (greater than the number of patterns in the set)

- functions and methods that require a previous successful match
  operation (with information stored in priv_task) when there was no
  prior match, or the previous macth failed.

- use of select=UNIQUE when more than one pattern in a set matched

- numeric (by index) or "associative" (after match) retrieval of an
  object for a set when no such object was saved in the constructor:
  strings, backends, regexen, etc

- any of the following are undefined (NULL): fallbacks; patterns for
  regex functions (which are compiled at runtime); the text and
  rewrite parameters for the rewrite operations; name parameter
  for namedrefs

- also if the named parameter for namedrefs is the empty string

- backref number out of range (greater than the number of backrefs)

- backref or namedref attempted without a prior match

- backref or namedref when never_capture=true in the contructor

- low-level failures reported by RE2 (eg cannot determine the number
  of backref groups). Most of these are possible, since the call
  returns an error status, but I have never seen them happen.

VTC tests have been revised for the new error handling, which changes
quite a bit in this commit.

While we're here, use backend None in the tests where appropriate.
Note that "bad_ip" backends are still necessary to test methods and
functions that retrieve backends (numeric or associative references
for sets).

Error reporting is now done with VCL_fail(), so we get rid of the
errmsg() function, and with it the re2.c source (since that was all
there was in re2.c).
parent 07cbe881
Expand all Show whitespace changes
Inline Side-by-side
Showing with 4845 additions and 1573 deletions
README.md
View file @ c70e410a
......@@ -1638,12 +1638,7 @@ Example:
black.hdr_filter(resp, false);
}
#### STRING quotemeta(STRING, STRING fallback)
STRING quotemeta(
STRING,
STRING fallback="**QUOTEMETA FUNCTION FAILED**"
)
#### STRING quotemeta(STRING)
Returns a copy of the argument string with all regex metacharacters
escaped via backslash. When the returned string is used as a regular
......
src/Makefile.am
View file @ c70e410a
......@@ -10,7 +10,6 @@ libvmod_re2_la_SOURCES = \
vmod_re2.h \
vmod_re2.c \
set.c \
re2.c \
vre2/vre2.h \
vre2/vre2.cpp \
vre2/vre2set.h \
......@@ -25,7 +24,7 @@ MAINTAINERCLEANFILES = $(dist_man_MANS)
libvmod_re2_la_LIBADD = @RE2_LIBS@
vmod_re2.c set.c re2.c: vmod_re2.h
vmod_re2.c set.c: vmod_re2.h
vmod_re2.h: vcc_if.h
......
src/re2.c deleted 100644 → 0
View file @ 07cbe881
/*-
* Copyright (c) 2017 UPLEX Nils Goroll Systemoptimierung
* All rights reserved
*
* Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include "vmod_re2.h"
void
errmsg(VRT_CTX, const char *fmt, ...)
{
va_list args;
AZ(ctx->method & VCL_MET_TASK_H);
va_start(args, fmt);
if (ctx->vsl)
VSLbv(ctx->vsl, SLT_VCL_Error, fmt, args);
else
VSLv(SLT_VCL_Error, 0, fmt, args);
va_end(args);
}
src/set.c
View file @ c70e410a
......@@ -213,7 +213,7 @@ vmod_set_add(VRT_CTX, struct vmod_re2_set *set, struct VARGS(set_add) *args)
if (pattern == NULL)
pattern = "";
if (!INIT(ctx)) {
VERR(ctx, ERR_PREFIX ".add() may only be called in vcl_init",
VFAIL(ctx, ERR_PREFIX ".add() may only be called in vcl_init",
set->vcl_name, pattern);
return;
}
......@@ -233,7 +233,7 @@ vmod_set_add(VRT_CTX, struct vmod_re2_set *set, struct VARGS(set_add) *args)
if ((set->string = realloc(set->string,
(n + 1) * (sizeof(char *))))
== NULL) {
VFAILNOMEM(ctx, ERR_PREFIX "adding string %s",
VERRNOMEM(ctx, ERR_PREFIX "adding string %s",
set->vcl_name, pattern, args->string);
return;
}
......@@ -245,7 +245,7 @@ vmod_set_add(VRT_CTX, struct vmod_re2_set *set, struct VARGS(set_add) *args)
if ((set->backend = realloc(set->backend,
(n + 1) * (sizeof(VCL_BACKEND))))
== NULL) {
VFAILNOMEM(ctx, ERR_PREFIX "adding backend %s",
VERRNOMEM(ctx, ERR_PREFIX "adding backend %s",
set->vcl_name, pattern,
VRT_BACKEND_string(args->backend));
return;
......@@ -257,7 +257,7 @@ vmod_set_add(VRT_CTX, struct vmod_re2_set *set, struct VARGS(set_add) *args)
if ((set->integer = realloc(set->integer,
(n + 1) * (sizeof(VCL_INT))))
== NULL) {
VFAILNOMEM(ctx, ERR_PREFIX "adding integer %jd",
VERRNOMEM(ctx, ERR_PREFIX "adding integer %jd",
set->vcl_name, pattern, args->integer);
return;
}
......@@ -313,7 +313,7 @@ vmod_set_compile(VRT_CTX, struct vmod_re2_set *set)
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if (!INIT(ctx)) {
VERR(ctx, ERR_PREFIX ".compile() may only be called in "
VFAIL(ctx, ERR_PREFIX ".compile() may only be called in "
"vcl_init", set->vcl_name);
return;
}
......@@ -356,14 +356,15 @@ vmod_set_match(VRT_CTX, struct vmod_re2_set *set, VCL_STRING subject)
subject = "";
if (!set->compiled) {
VERR(ctx, ERR_PREFIX "%s was not compiled", set->vcl_name,
VFAIL(ctx, ERR_PREFIX "%s was not compiled", set->vcl_name,
subject, set->vcl_name);
return 0;
}
priv = VRT_priv_task(ctx, set);
if (priv == NULL) {
ERR(ctx, "No priv_task - workspace overflow?");
VFAIL(ctx, ERR_PREFIX "No priv_task - workspace overflow?",
set->vcl_name, subject);
return 0;
}
if (priv->priv == NULL) {
......@@ -386,7 +387,7 @@ vmod_set_match(VRT_CTX, struct vmod_re2_set *set, VCL_STRING subject)
buf = WS_Reservation(ctx->ws);
if ((err = vre2set_match(set->set, subject, &match, buf, buflen,
&task->nmatches, &errkind)) != NULL) {
VERR(ctx, ERR_PREFIX "%s", set->vcl_name, subject, err);
VFAIL(ctx, ERR_PREFIX "%s", set->vcl_name, subject, err);
WS_Release(ctx->ws, 0);
return 0;
}
......@@ -402,7 +403,7 @@ vmod_set_match(VRT_CTX, struct vmod_re2_set *set, VCL_STRING subject)
case NOT_IMPLEMENTED:
break;
case OUT_OF_MEMORY:
VERR(ctx, ERR_PREFIX "RE2 lib indicates out-of-memory "
VFAIL(ctx, ERR_PREFIX "RE2 lib indicates out-of-memory "
"during match, consider increasing max_mem",
set->vcl_name, subject);
break;
......@@ -441,14 +442,14 @@ vmod_set_matched(VRT_CTX, struct vmod_re2_set *set, VCL_INT n)
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if (n < 1 || n > set->npatterns) {
VERR(ctx, "n=%d out of range in %s.matched() (%d patterns)", n,
set->vcl_name, set->npatterns);
VFAIL(ctx, "n=%jd out of range in %s.matched() (%d patterns)",
(intmax_t)n, set->vcl_name, set->npatterns);
return 0;
}
if ((task = get_task_data(ctx, set)) == NULL) {
VERR(ctx, "%s.matched(%d) called without prior match",
set->vcl_name, n);
VFAIL(ctx, "%s.matched(%jd) called without prior match",
set->vcl_name, (intmax_t)n);
return 0;
}
......@@ -479,7 +480,7 @@ vmod_set_nmatches(VRT_CTX, struct vmod_re2_set *set)
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if ((task = get_task_data(ctx, set)) == NULL) {
VERR(ctx, "%s.nmatches() called without prior match",
VFAIL(ctx, "%s.nmatches() called without prior match",
set->vcl_name);
return 0;
}
......@@ -494,28 +495,29 @@ get_match_idx(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects,
int idx = 0;
if (n > set->npatterns) {
VERR(ctx, "%s.%s(%lld): set has %d patterns", set->vcl_name,
method, n, set->npatterns);
VFAIL(ctx, "%s.%s(%jd): set has %d patterns", set->vcl_name,
method, (intmax_t)n, set->npatterns);
return -1;
}
if (n > 0)
return n - 1;
if ((task = get_task_data(ctx, set)) == NULL) {
VERR(ctx, "%s.%s() called without prior match", set->vcl_name,
VFAIL(ctx, "%s.%s() called without prior match", set->vcl_name,
method);
return -1;
}
if (task->nmatches == 0) {
VERR(ctx, "%s.%s(%lld): previous match was unsuccessful",
set->vcl_name, method, n);
VFAIL(ctx, "%s.%s(%jd): previous match was unsuccessful",
set->vcl_name, method, (intmax_t)n);
return -1;
}
if (task->nmatches > 1) {
if (selects == VENUM(UNIQUE)) {
VERR(ctx, "%s.%s(%lld): %d successful matches",
set->vcl_name, method, n, task->nmatches);
VFAIL(ctx, "%s.%s(%jd): %ld successful matches",
set->vcl_name, method, (intmax_t)n,
task->nmatches);
return -1;
}
if (selects == VENUM(LAST))
......@@ -547,8 +549,9 @@ rewritef(VRT_CTX, struct vmod_re2_set * const restrict set,
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if (set->regex == NULL) {
VERR(ctx, "%s.%s(%lld): No regexen were saved for %s",
set->vcl_name, rewrite_name[type], n, set->vcl_name);
VFAIL(ctx, "%s.%s(%jd): No regexen were saved for %s",
set->vcl_name, rewrite_name[type], (intmax_t)n,
set->vcl_name);
return NULL;
}
......@@ -557,9 +560,9 @@ rewritef(VRT_CTX, struct vmod_re2_set * const restrict set,
return NULL;
if (!vbit_test(set->added[REGEX], idx)) {
AN(selects);
VERR(ctx, "%s.%s(%s, %s, %lld, %s): Pattern %d was not saved",
set->vcl_name, rewrite_name[type], text, rewrite, n,
selects, idx + 1);
VFAIL(ctx, "%s.%s(%s, %s, %jd, %s): Pattern %d was not saved",
set->vcl_name, rewrite_name[type], text, rewrite,
(intmax_t)n, selects, idx + 1);
return NULL;
}
return (regex_rewrite[type])(ctx, set->regex[idx], text, rewrite,
......@@ -598,8 +601,8 @@ vmod_set_string(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects)
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if (set->string == NULL) {
VERR(ctx, "%s.string(%lld): No strings were set for %s",
set->vcl_name, n, set->vcl_name);
VFAIL(ctx, "%s.string(%jd): No strings were set for %s",
set->vcl_name, (intmax_t)n, set->vcl_name);
return NULL;
}
......@@ -608,8 +611,8 @@ vmod_set_string(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects)
return NULL;
if (!vbit_test(set->added[STRING], idx)) {
AN(selects);
VERR(ctx, "%s.string(%lld, %s): String %lld was not added",
set->vcl_name, n, selects, idx + 1);
VFAIL(ctx, "%s.string(%jd, %s): String %d was not added",
set->vcl_name, (intmax_t)n, selects, idx + 1);
return NULL;
}
return set->string[idx];
......@@ -623,8 +626,8 @@ vmod_set_backend(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects)
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(set, VMOD_RE2_SET_MAGIC);
if (set->backend == NULL) {
VERR(ctx, "%s.backend(%lld): No backends were set for %s",
set->vcl_name, n, set->vcl_name);
VFAIL(ctx, "%s.backend(%jd): No backends were set for %s",
set->vcl_name, (intmax_t)n, set->vcl_name);
return NULL;
}
......@@ -633,8 +636,8 @@ vmod_set_backend(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects)
return NULL;
if (!vbit_test(set->added[BACKEND], idx)) {
AN(selects);
VERR(ctx, "%s.backend(%lld, %s): Backend %lld was not added",
set->vcl_name, n, selects, idx + 1);
VFAIL(ctx, "%s.backend(%jd, %s): Backend %d was not added",
set->vcl_name, (intmax_t)n, selects, idx + 1);
return NULL;
}
return set->backend[idx];
......@@ -650,19 +653,16 @@ vmod_set_integer(VRT_CTX, struct vmod_re2_set *set, VCL_INT n, VCL_ENUM selects)
if (set->integer == NULL) {
VRT_fail(ctx,
"%s.integer(%jd): No integers were set for %s",
set->vcl_name, n, set->vcl_name);
set->vcl_name, (intmax_t)n, set->vcl_name);
return (0);
}
idx = get_match_idx(ctx, set, n, selects, "integer");
if (idx < 0) {
VRT_fail(ctx, "See previous VCL_Error");
if (idx < 0)
return (0);
}
if (!vbit_test(set->added[INTEGER], idx)) {
AN(selects);
VRT_fail(ctx,
"%s.integer(%jd, %s): Integer %d was not added",
VRT_fail(ctx, "%s.integer(%jd, %s): integer %d was not added",
set->vcl_name, n, selects, idx + 1);
return (0);
}
......@@ -732,8 +732,8 @@ vmod_set_hdr_filter(VRT_CTX, struct VPFX(re2_set) *set, VCL_HTTP hp,
if ((err = vre2set_matchonly(set->set, hdr, len, &match,
&errkind))
!= NULL) {
VERR(ctx, "%s.hdr_filter(%.*s): %s", set->vcl_name, len,
hdr, err);
VFAIL(ctx, "%s.hdr_filter(%.*s): %s", set->vcl_name,
len, hdr, err);
goto loop;
}
......@@ -742,7 +742,7 @@ vmod_set_hdr_filter(VRT_CTX, struct VPFX(re2_set) *set, VCL_HTTP hp,
case NOT_IMPLEMENTED:
break;
case OUT_OF_MEMORY:
VERR(ctx, "%s.hdr_filter(%.*s): RE2 lib indicates "
VFAIL(ctx, "%s.hdr_filter(%.*s): RE2 lib indicates "
"out-of-memory during match, consider increasing "
"max_mem", set->vcl_name, len, hdr);
goto loop;
......
src/tests/00basic.vtc
View file @ c70e410a
......@@ -23,7 +23,7 @@ varnish v1 -cli "vcl.list"
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_recv {
return(synth(200));
......
src/tests/backref_fail.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/backref_limit.vtc
View file @ c70e410a
......@@ -4,7 +4,8 @@ varnishtest "backref limits"
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
import std;
backend b None;
sub vcl_init {
new elevendots = re2.regex("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)");
......@@ -32,11 +33,13 @@ varnish v1 -vcl {
= elevendots.backref(10, "error10");
set resp.http.foo11
= elevendots.backref(11, "error11");
set resp.http.foo12
= elevendots.backref(12, "fallback");
} else {
set resp.status = 999;
}
if (req.http.Int11) {
set resp.http.Backref11
= elevendots.backref(std.integer(req.http.Int11));
}
if (twentydots.match("123456789012345678901")) {
set resp.http.bar0 = twentydots.backref(0, "error0");
......@@ -60,10 +63,13 @@ varnish v1 -vcl {
set resp.http.bar18 = twentydots.backref(18, "error18");
set resp.http.bar19 = twentydots.backref(19, "error19");
set resp.http.bar20 = twentydots.backref(20, "error20");
set resp.http.bar21 = twentydots.backref(21, "error21");
} else {
set resp.status = 999;
}
if (req.http.Int20) {
set resp.http.Backref20
= twentydots.backref(std.integer(req.http.Int20));
}
}
} -start
......@@ -84,7 +90,6 @@ client c1 {
expect resp.http.foo9 == "9"
expect resp.http.foo10 == "0"
expect resp.http.foo11 == "1"
expect resp.http.foo12 == "fallback"
expect resp.http.bar0 == "12345678901234567890"
expect resp.http.bar1 == "1"
expect resp.http.bar2 == "2"
......@@ -106,19 +111,40 @@ client c1 {
expect resp.http.bar18 == "8"
expect resp.http.bar19 == "9"
expect resp.http.bar20 == "0"
expect resp.http.bar21 == "error21"
} -run
logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
logexpect l1 -v v1 -d 0 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = VCL_Error {^vmod re2 failure: elevendots\.backref\(ref=12, fallback="[^"]+"\): backref out of range \(max 11\)$}
# "
expect 0 = RespHeader {^Backref11: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = VCL_Error "^vmod re2 error: elevendots.backref.ref=12, fallback=.fallback..: backref out of range .max 11.$"
expect * = VCL_Error "^vmod re2 error: twentydots.backref.ref=21, fallback=.error21..: backref out of range .max 20.$"
expect * = VCL_Error {^vmod re2 failure: twentydots\.backref\(ref=21, fallback="[^"]+"\): backref out of range \(max 20\)$}
# "
expect 0 = RespHeader {^Backref20: $}
expect 0 = VCL_return fail
expect * = End
} -start
client c2 {
txreq -hdr "Int11: 12"
expect_close
} -run
client c3 {
txreq -hdr "Int20: 21"
expect_close
} -run
logexpect l1 -wait
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
import std;
backend b None;
sub vcl_recv {
return(synth(200));
......@@ -139,10 +165,13 @@ varnish v1 -vcl {
set resp.http.foo9 = re2.backref(9, "error9");
set resp.http.foo10 = re2.backref(10, "error10");
set resp.http.foo11 = re2.backref(11, "error11");
set resp.http.foo12 = re2.backref(12, "fallback");
} else {
set resp.status = 999;
}
if (req.http.Int11) {
set resp.http.Backref11
= re2.backref(std.integer(req.http.Int11));
}
if (re2.match("(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)",
"123456789012345678901")) {
......@@ -167,21 +196,37 @@ varnish v1 -vcl {
set resp.http.bar18 = re2.backref(18, "error18");
set resp.http.bar19 = re2.backref(19, "error19");
set resp.http.bar20 = re2.backref(20, "error20");
set resp.http.bar21 = re2.backref(21, "error21");
} else {
set resp.status = 999;
}
if (req.http.Int20) {
set resp.http.Backref20
= re2.backref(std.integer(req.http.Int20));
}
}
}
logexpect l2 -v v1 -d 0 -g vxid -q "VCL_Error" {
expect * * Begin req
expect * = VCL_Error "^vmod re2 error: re2.backref.ref=12, fallback=.fallback..: backref out of range .max 11.$"
expect * = VCL_Error "^vmod re2 error: re2.backref.ref=21, fallback=.error21..: backref out of range .max 20.$"
client c1 -run
logexpect l1 -v v1 -d 0 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = VCL_Error {^vmod re2 failure: re2\.backref\(ref=12, fallback="[^"]+"\): backref out of range \(max 11\)$}
# "
expect 0 = RespHeader {^Backref11: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = VCL_Error {^vmod re2 failure: re2\.backref\(ref=21, fallback="[^"]+"\): backref out of range \(max 20\)$}
# "
expect 0 = RespHeader {^Backref20: $}
expect 0 = VCL_return fail
expect * = End
} -start
client c1 -run
client c2 -run
client c3 -run
logexpect l2 -wait
logexpect l1 -wait
src/tests/cached.vtc
View file @ c70e410a
......@@ -6,7 +6,7 @@ varnishtest "cached compiled regexen"
varnish v1 -vcl+backend {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_init {
new barbaz = re2.regex("(bar)(baz)");
......
src/tests/compile_fail.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/cost.vtc
View file @ c70e410a
......@@ -4,7 +4,7 @@ varnishtest "cost() function"
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_init {
# Tests from re2 testing/re2_test.cc
......@@ -26,8 +26,10 @@ varnish v1 -vcl {
set resp.http.C-Medium-F = re2.cost("medium.*regexp");
set resp.http.C-Complex-F = re2.cost("complex.{1,128}regexp");
if (req.http.Fail) {
set resp.http.Fail-F = re2.cost("(");
}
}
} -start
client c1 {
......@@ -43,11 +45,17 @@ client c1 {
expect resp.http.C-Simple-F == resp.http.C-Simple
expect resp.http.C-Medium-F == resp.http.C-Medium
expect resp.http.C-Complex-F == resp.http.C-Complex
expect resp.http.Fail-F == "-1"
} -run
client c1 {
txreq -hdr "Fail: oops"
expect_close
} -run
logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = VCL_Error {^vmod re2 error: re2\.cost\("\("\): Cannot compile: }
expect * = VCL_Error {^vmod re2 failure: re2\.cost\("\("\): Cannot compile: }
expect 0 = RespHeader {^Fail-F: -1$}
expect 0 = VCL_return fail
expect * = End
} -run
src/tests/encoding.vtc
View file @ c70e410a
......@@ -8,7 +8,7 @@ varnishtest "latin1 and utf8 encoding"
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_init {
new ninebytes = re2.regex("^.........$");
......@@ -125,7 +125,7 @@ client c1 {
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_init {
new ninebytes = re2.set();
......@@ -195,7 +195,7 @@ client c1 {
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_recv {
return(synth(200));
......@@ -242,7 +242,7 @@ client c1 {
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_recv {
return(synth(200));
......@@ -289,7 +289,7 @@ client c1 {
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_recv {
return(synth(200));
......
src/tests/extract.vtc
View file @ c70e410a
......@@ -2,20 +2,20 @@
varnishtest "extract() method and function"
varnish v1 -vcl {
server s1 {
rxreq
txresp
} -start
varnish v1 -vcl+backend {
import ${vmod_re2};
backend be { .host = "${bad_ip}"; }
sub vcl_init {
new email = re2.regex("(.*)@([^.]*)");
new dotstar = re2.regex(".*");
}
sub vcl_recv {
return(synth(200));
}
sub vcl_synth {
sub vcl_deliver {
# Tests from re2 testing/re2_test.cc
set resp.http.uucp = email.extract("boris@kremvax.ru", "\2!\1");
set resp.http.quote = dotstar.extract("foo", "'\0'");
......@@ -29,38 +29,60 @@ varnish v1 -vcl {
"fallbackf");
# Undefined fallback
if (req.http.Test == "fallbackundef") {
if (req.http.Call == "method") {
set resp.http.undeffallback
= email.extract("foo", "bar", req.http.undef);
set resp.http.undeffallbackf
= re2.extract(".", "foo", "bar", req.http.undef);
= email.extract("foo", "bar",
req.http.undef);
}
else {
set resp.http.undeffallback
= re2.extract(".", "foo", "bar",
req.http.undef);
}
}
# Undefined pattern in the function
if (req.http.Test == "patternundef") {
set resp.http.undefpattern
= re2.extract(req.http.undef, "", "", "pattern undef");
= re2.extract(req.http.undef, "", "");
}
# Undefined text
if (req.http.Test == "textundef") {
if (req.http.Call == "method") {
set resp.http.undeftext
= email.extract(req.http.undef, "x");
}
else {
set resp.http.undeftext
= email.extract(req.http.undef, "x", "text undef");
set resp.http.undeftextf
= re2.extract(".", req.http.undef, "x", "text undef");
= re2.extract(".", req.http.undef, "x");
}
}
# Undefined rewrite
if (req.http.Test == "rewriteundef") {
if (req.http.Call == "method") {
set resp.http.undefrewrite
= email.extract("b", req.http.undef);
}
else {
set resp.http.undefrewrite
= email.extract("b", req.http.undef, "rewrite undef");
set resp.http.undefrewritef
= re2.extract(".", "b", req.http.undef,"rewrite undef");
= re2.extract(".", "b", req.http.undef);
}
}
# Default fallbacks
set resp.http.fallback = email.extract("foo", "bar");
set resp.http.fallbackf = re2.extract("(.*)@([^.]*)", "foo",
"bar");
}
} -start
client c1 {
txreq
rxresp
expect resp.status == 200
expect resp.http.uucp == "kremvax!boris"
expect resp.http.quote == "'foo'"
expect resp.http.uucpf == "kremvax!boris"
......@@ -69,33 +91,131 @@ client c1 {
expect resp.http.fail == "fallback"
expect resp.http.failf == "fallbackf"
expect resp.http.undeffallback == "**EXTRACT METHOD FAILED**"
expect resp.http.undeffallbackf == "**EXTRACT FUNCTION FAILED**"
expect resp.http.fallback == "**EXTRACT METHOD FAILED**"
expect resp.http.fallbackf == "**EXTRACT FUNCTION FAILED**"
} -run
client c1 {
txreq -hdr "Test: fallbackundef" -hdr "Call: method"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undeffallback == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: fallbackundef" -hdr "Call: function"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undeffallback == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: patternundef"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undefpattern == <undef>
expect_close
} -run
expect resp.http.undefpattern == "pattern undef"
client c1 {
txreq -hdr "Test: textundef" -hdr "Call: method"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undeftext == <undef>
expect_close
} -run
expect resp.http.undeftext == "text undef"
expect resp.http.undeftextf == "text undef"
client c1 {
txreq -hdr "Test: textundef" -hdr "Call: function"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undeftext == <undef>
expect_close
} -run
expect resp.http.undefrewrite == "rewrite undef"
expect resp.http.undefrewritef == "rewrite undef"
client c1 {
txreq -hdr "Test: rewriteundef" -hdr "Call: method"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undefrewrite == <undef>
expect_close
} -run
expect resp.http.fallback == "**EXTRACT METHOD FAILED**"
expect resp.http.fallbackf == "**EXTRACT FUNCTION FAILED**"
client c1 {
txreq -hdr "Test: rewriteundef" -hdr "Call: function"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.undefrewrite == <undef>
expect_close
} -run
logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = ReqHeader {^Test: fallbackundef$}
expect * = ReqHeader {^Call: method$}
expect * = VCL_Error {^vmod re2 failure: email\.extract\(\): fallback is undefined$}
expect 0 = RespHeader {^undeffallback: $}
expect 0 = VCL_return fail
expect * = End
expect * = VCL_Error "^vmod re2 error: email.extract..: fallback is undefined$"
expect * = VCL_Error "^vmod re2 error: re2.extract..: fallback is undefined$"
expect * = VCL_Error "^vmod re2 error: re2.extract.pattern=<undefined>, fallback=.pattern undef..: pattern is undefined$"
expect 0 * Begin req
expect * = ReqHeader {^Test: fallbackundef$}
expect * = ReqHeader {^Call: function$}
expect * = VCL_Error {^vmod re2 failure: re2\.extract\(\): fallback is undefined$}
expect 0 = RespHeader {^undeffallback: $}
expect 0 = VCL_return fail
expect * = End
expect * = VCL_Error "^vmod re2 error: email.extract.text=<undefined>, fallback=.text undef..: text is undefined$"
expect * = VCL_Error "^vmod re2 error: re2.extract.pattern=..., text=<undefined>, fallback=.text undef..: text is undefined$"
expect 0 * Begin req
expect * = ReqHeader {^Test: patternundef$}
expect * = VCL_Error {^vmod re2 failure: re2\.extract\(pattern=<undefined>, fallback="[^"]+"\): pattern is undefined$}
# "
expect 0 = RespHeader {^undefpattern: $}
expect 0 = VCL_return fail
expect * = End
expect * = VCL_Error "^vmod re2 error: email.extract.text=.b., rewrite=<undefined>, fallback=.rewrite undef..: rewrite is undefined$"
expect * = VCL_Error "^vmod re2 error: re2.extract.pattern=..., text=.b., rewrite=<undefined>, fallback=.rewrite undef..: rewrite is undefined$"
expect 0 * Begin req
expect * = ReqHeader {^Test: textundef$}
expect * = ReqHeader {^Call: method$}
expect * = VCL_Error {^vmod re2 failure: email\.extract\(text=<undefined>, fallback="[^"]+"\): text is undefined$}
# "
expect 0 = RespHeader {^undeftext: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: textundef$}
expect * = ReqHeader {^Call: function$}
expect * = VCL_Error {^vmod re2 failure: re2\.extract\(pattern="\.", text=<undefined>, fallback="[^"]+"\): text is undefined$}
# "
expect 0 = RespHeader {^undeftext: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: rewriteundef$}
expect * = ReqHeader {^Call: method$}
expect * = VCL_Error {^vmod re2 failure: email\.extract\(text="b", rewrite=<undefined>, fallback="[^"]+"\): rewrite is undefined$}
# "
expect 0 = RespHeader {^undefrewrite: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: rewriteundef$}
expect * = ReqHeader {^Call: function$}
expect * = VCL_Error {^vmod re2 failure: re2\.extract\(pattern="\.", text="b", rewrite=<undefined>, fallback="[^"]+"\): rewrite is undefined$}
# "
expect 0 = RespHeader {^undefrewrite: $}
expect 0 = VCL_return fail
expect * = End
} -run
src/tests/hdr_filter.vtc
View file @ c70e410a
......@@ -4,7 +4,7 @@ varnishtest "set.hdr_filter()"
varnish v1 -vcl {
import ${vmod_re2};
backend be { .host = "${bad_ip}"; }
backend be None;
sub vcl_init {
new white = re2.set(anchor=start);
......
src/tests/hdrcleanup.vtc
View file @ c70e410a
......@@ -6,7 +6,7 @@ varnish v1 -vcl {
import ${vmod_re2};
import std;
backend be { .host = "${bad_ip}"; }
backend be None;
sub vcl_init {
# \b is insufficient for the ,<remove> case:
......
src/tests/match.vtc
View file @ c70e410a
......@@ -72,7 +72,7 @@ client c1 {
varnish v1 -vcl {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
backend b None;
sub vcl_recv {
return(synth(200));
......@@ -87,13 +87,11 @@ varnish v1 -vcl {
client c2 {
txreq
rxresp
expect resp.status == "200"
expect resp.http.undef == <undef>
expect_close
} -run
logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = VCL_Error "^vmod re2 error: re2.match.pattern=.<undefined>., text=...: pattern is undefined$"
expect * = VCL_Error "^vmod re2 failure: re2.match.pattern=.<undefined>., text=...: pattern is undefined$"
expect * = End
} -run
src/tests/max_mem.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/namedref.vtc
View file @ c70e410a
......@@ -2,9 +2,13 @@
varnishtest "namedref()"
varnish v1 -vcl {
server s1 {
rxreq
txresp
} -start
varnish v1 -vcl+backend {
import ${vmod_re2};
backend b { .host = "${bad_ip}"; }
sub vcl_init {
new barbaz = re2.regex("(?P<bar>bar)(?P<baz>baz)");
......@@ -12,28 +16,26 @@ varnish v1 -vcl {
never_capture=true);
}
sub vcl_recv {
return(synth(200));
}
sub vcl_synth {
sub vcl_deliver {
if (barbaz.match("barbaz")) {
set resp.http.num0 = barbaz.backref(0, "error0");
set resp.http.num1 = barbaz.backref(1, "error1");
set resp.http.num2 = barbaz.backref(2, "error2");
set resp.http.bar = barbaz.namedref("bar", "error_bar");
set resp.http.baz = barbaz.namedref("baz", "error_baz");
if (req.http.Test == "nosuchgrp"
&& req.http.Call == "method") {
set resp.http.quux
= barbaz.namedref("quux", "error_quux");
}
}
if (never.match("barbaz")) {
set resp.http.never = "match";
set resp.http.neverbar
= never.namedref("bar", "error_bar");
set resp.http.neverbaz
= never.namedref("baz", "error_baz");
set resp.http.neverquux
= never.namedref("quux", "error_quux");
if (req.http.Test == "never"
&& req.http.Call == "method") {
set resp.http.neverref
= never.namedref(req.http.Ref);
}
}
if (re2.match("(?P<bar>bar)(?P<baz>baz)", "barbaz")) {
......@@ -42,18 +44,20 @@ varnish v1 -vcl {
set resp.http.num2f = re2.backref(2, "error2");
set resp.http.barf = re2.namedref("bar", "error_bar");
set resp.http.bazf = re2.namedref("baz", "error_baz");
if (req.http.Test == "nosuchgrp"
&& req.http.Call == "function") {
set resp.http.quuxf
= re2.namedref("quux", "error_quux");
}
}
if (re2.match("(?P<bar>bar)(?P<baz>baz)", "barbaz",
never_capture=true)) {
set resp.http.neverf = "match";
set resp.http.neverbarf
= re2.namedref("bar", "error_bar");
set resp.http.neverbazf
= re2.namedref("baz", "error_baz");
set resp.http.neverquuxf
= re2.namedref("quux", "error_quux");
if (req.http.Test == "never"
&& req.http.Call == "function") {
set resp.http.neverref
= re2.namedref(req.http.Ref);
}
}
}
......@@ -62,38 +66,168 @@ varnish v1 -vcl {
client c1 {
txreq
rxresp
expect resp.status == 200
expect resp.http.num0 == "barbaz"
expect resp.http.num1 == "bar"
expect resp.http.num2 == "baz"
expect resp.http.bar == "bar"
expect resp.http.baz == "baz"
expect resp.http.quux == "error_quux"
expect resp.http.never == "match"
expect resp.http.neverbar == "error_bar"
expect resp.http.neverbaz == "error_baz"
expect resp.http.neverquux == "error_quux"
expect resp.http.num0f == "barbaz"
expect resp.http.num1f == "bar"
expect resp.http.num2f == "baz"
expect resp.http.barf == "bar"
expect resp.http.bazf == "baz"
expect resp.http.quuxf == "error_quux"
expect resp.http.neverf == "match"
expect resp.http.neverbarf == "error_bar"
expect resp.http.neverbazf == "error_baz"
expect resp.http.neverquuxf == "error_quux"
} -run
client c1 {
txreq -hdr "Test: nosuchgrp" -hdr "Call: method"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.quux == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: method" -hdr "Ref: bar"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: method" -hdr "Ref: baz"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: method" -hdr "Ref: quux"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: nosuchgrp" -hdr "Call: function"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.quuxf == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: function" -hdr "Ref: bar"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: function" -hdr "Ref: baz"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
client c1 {
txreq -hdr "Test: never" -hdr "Call: function" -hdr "Ref: quux"
rxresp
expect resp.status == 503
expect resp.reason == "VCL failed"
expect resp.http.neverref == <undef>
expect_close
} -run
logexpect l1 -v v1 -d 1 -g vxid -q "VCL_Error" {
expect 0 * Begin req
expect * = VCL_Error "^vmod re2 error: namedref name=.quux., fallback=.error_quux.: no such named group$"
expect * = VCL_Error "^vmod re2 error: never.namedref.name=.bar., fallback=.error_bar..: never_capture is true for object never$"
expect * = VCL_Error "^vmod re2 error: never.namedref.name=.baz., fallback=.error_baz..: never_capture is true for object never$"
expect * = VCL_Error "^vmod re2 error: never.namedref.name=.quux., fallback=.error_quux..: never_capture is true for object never$"
expect * = VCL_Error "^vmod re2 error: namedref name=.quux., fallback=.error_quux.: no such named group$"
expect * = VCL_Error "^vmod re2 error: re2.namedref.name=.bar., fallback=.error_bar..: never_capture was true in previous match$"
expect * = VCL_Error "^vmod re2 error: re2.namedref.name=.baz., fallback=.error_baz..: never_capture was true in previous match$"
expect * = VCL_Error "^vmod re2 error: re2.namedref.name=.quux., fallback=.error_quux..: never_capture was true in previous match$"
expect * = ReqHeader {^Test: nosuchgrp$}
expect * = ReqHeader {^Call: method$}
expect * = VCL_Error {^vmod re2 failure: namedref name="quux", fallback="error_quux": no such named group$}
expect 0 = RespHeader {^quux: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: method$}
expect * = ReqHeader {^Ref: bar$}
expect * = VCL_Error {^vmod re2 failure: never\.namedref\(name="bar", fallback="[^"]+"\): never_capture is true for object never$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: method$}
expect * = ReqHeader {^Ref: baz$}
expect * = VCL_Error {^vmod re2 failure: never\.namedref\(name="baz", fallback="[^"]+"\): never_capture is true for object never$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: method$}
expect * = ReqHeader {^Ref: quux$}
expect * = VCL_Error {^vmod re2 failure: never\.namedref\(name="quux", fallback="[^"]+"\): never_capture is true for object never$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: nosuchgrp$}
expect * = ReqHeader {^Call: function$}
expect * = VCL_Error {^vmod re2 failure: namedref name="quux", fallback="error_quux": no such named group$}
expect 0 = RespHeader {^quuxf: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: function$}
expect * = ReqHeader {^Ref: bar$}
expect * = VCL_Error {^vmod re2 failure: re2\.namedref\(name="bar", fallback="[^"]+"\): never_capture was true in previous match$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: function$}
expect * = ReqHeader {^Ref: baz$}
expect * = VCL_Error {^vmod re2 failure: re2\.namedref\(name="baz", fallback="[^"]+"\): never_capture was true in previous match$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
expect 0 * Begin req
expect * = ReqHeader {^Test: never$}
expect * = ReqHeader {^Call: function$}
expect * = ReqHeader {^Ref: quux$}
expect * = VCL_Error {^vmod re2 failure: re2\.namedref\(name="quux", fallback="[^"]+"\): never_capture was true in previous match$}
# "
expect 0 = RespHeader {^neverref: $}
expect 0 = VCL_return fail
expect * = End
} -run
src/tests/non_capture_paren.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/options.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/quotemeta.vtc
View file @ c70e410a
......@@ -4,7 +4,7 @@ varnishtest "quotemeta() function"
varnish v1 -vcl {
import ${vmod_re2};
backend be { .host = "${bad_ip}"; }
backend be None;
sub vcl_recv {
return(synth(200));
......
src/tests/set.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/set_extract.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/set_retrieve.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/set_sub.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/set_suball.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/set_usage.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/sub.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/suball.vtc
View file @ c70e410a
This diff is collapsed.
src/tests/vcl_regex.vtc
View file @ c70e410a
This diff is collapsed.
src/vmod_re2.c
View file @ c70e410a
This diff is collapsed.
src/vmod_re2.h
View file @ c70e410a
This diff is collapsed.
src/vmod_re2.vcc
View file @ c70e410a
......@@ -1511,8 +1511,7 @@ Example::
black.hdr_filter(resp, false);
}
$Function STRING quotemeta(STRING,
STRING fallback="**QUOTEMETA FUNCTION FAILED**")
$Function STRING quotemeta(STRING)
Returns a copy of the argument string with all regex metacharacters
escaped via backslash. When the returned string is used as a regular
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment