...
 
Commits (31)
INSTALLATION
============
RPMs
~~~~
Binary, debuginfo and source RPMs for VMOD re are available at
packagecloud:
https://packagecloud.io/uplex/varnish
The packages are built for Enterprise Linux 7 (el7), and hence will
run on compatible distros (such as RHEL7, Fedora, CentOS 7 and Amazon
Linux).
To set up your YUM repository for the RPMs, follow these instructions:
https://packagecloud.io/uplex/varnish/install#manual-rpm
You will also need these additional repositories:
* EPEL7
* ``yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm``
* Official Varnish packages from packagecloud (since version 6.0.0)
* Follow the instructions at: https://packagecloud.io/varnishcache/varnish60/install#manual-rpm
If you have problems or questions concerning the RPMs, post an issue
to one of the source repository web sites, or contact
<varnish-support@uplex.de>.
Building from source
~~~~~~~~~~~~~~~~~~~~
The VMOD is built on a system where an instance of Varnish is
installed, and the auto-tools will attempt to locate the Varnish
instance, and then pull in libraries and other support files from
there.
Quick start
-----------
This sequence should be enough in typical setups:
1. ``./autogen.sh`` (for git-installation)
2. ``./configure``
3. ``make``
4. ``make check`` (regression tests)
5. ``make install`` (may require root: sudo make install)
Alternative configs
-------------------
If you have installed Varnish to a non-standard directory, call
``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
the appropriate path. For example, when varnishd configure was called
with ``--prefix=$PREFIX``, use::
PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
export PKG_CONFIG_PATH
For developers
--------------
As with Varnish, you can use these ``configure`` options for stricter
compiling:
* ``--enable-developer-warnings``
* ``--enable-extra-developer-warnings`` (for GCC 4)
* ``--enable-werror``
The VMOD must always build successfully with these options enabled.
Also as with Varnish, you can add ``--enable-debugging-symbols``, so
that the VMOD's symbols are available to debuggers, in core dumps and
so forth.
Copyright (c) 2013-2014 UPLEX Nils Goroll Systemoptimierung
Copyright 2013-2018 UPLEX Nils Goroll Systemoptimierung
All rights reserved.
Redistribution and use in source and binary forms, with or without
......
......@@ -6,10 +6,8 @@
.. role:: ref(emphasis)
.. _vmod_re(3):
=======
vmod_re
VMOD re
=======
-------------------------------------------------------------------------
......@@ -21,8 +19,20 @@ Varnish Module for Regular Expression Matching with Subexpression Capture
SYNOPSIS
========
import re [from "path"] ;
::
import re;
# object interface
new <obj> = re.regex(STRING, [, INT limit] [, INT limit_recursion])
BOOL <obj>.match(STRING, [, INT limit] [, INT limit_recursion])
STRING <obj>.backref(INT [, STRING fallback])
# function interface
BOOL re.match_dyn(STRING, [, INT limit] [, INT limit_recursion])
STRING re.backref_dyn(INT [, STRING fallback])
STRING re.version()
DESCRIPTION
===========
......@@ -103,22 +113,18 @@ since it re-uses the compiled expression obtained at VCL
initialization. So if you are matching against a fixed pattern that
never changes during the lifetime of VCL, use ``match``.
CONTENTS
========
* regex(STRING)
* BOOL match_dyn(PRIV_TASK, STRING, STRING)
* STRING backref_dyn(PRIV_TASK, INT, STRING)
* STRING version()
.. _obj_regex:
.. _vmod_re.regex:
regex
-----
new xregex = re.regex(STRING, INT limit, INT limit_recursion)
-------------------------------------------------------------
::
new OBJ = regex(STRING)
new xregex = re.regex(
STRING,
INT limit=1000,
INT limit_recursion=1000
)
Description
Create a regex object with the given regular expression. The
......@@ -129,34 +135,54 @@ Description
If the regular expression fails to compile, then the VCL
load fails with an error message describing the problem.
The optional parameters ``limit`` and ``limit_recursion`` are
per-object defaults for the respective parameters of the
`vmod_re.regex.match`_ method.
Example
``new myregex = re.regex("\bmax-age\s*=\s*(\d+)");``
.. _func_regex.match:
.. _vmod_re.regex.match:
regex.match
-----------
BOOL xregex.match(STRING, INT limit, INT limit_recursion)
---------------------------------------------------------
::
BOOL regex.match(STRING)
BOOL xregex.match(STRING, INT limit=0, INT limit_recursion=0)
Description
Determines whether the given string matches the regex compiled
by the constructor; functionally equivalent to VCL's infix
operator ``~``.
The optional parameter ``limit`` restricts the number of
internal matching function calls in a ``pcre_exec()``
execution, analogous to the varnishd ``pcre_match_limit``
parameter. For the default value 0, the ``limit`` given to the
constructor `vmod_re.regex`_ is used.
The optional parameter ``limit_recursion`` restricts the
number of internal matching function recursions in a
``pcre_exec()`` execution, analogous to the varnishd
``pcre_match_limit_recursion`` parameter. For the default
value 0, the ``limit_recursion`` given to the constructor
`vmod_re.regex`_ is used.
Example
``if (myregex.match(beresp.http.Surrogate-Control)) { # ...``
.. _func_regex.backref:
.. _vmod_re.regex.backref:
regex.backref
-------------
STRING xregex.backref(INT, STRING fallback)
-------------------------------------------
::
STRING regex.backref(INT, STRING fallback="**BACKREF METHOD FAILED**")
STRING xregex.backref(
INT,
STRING fallback="**BACKREF METHOD FAILED**"
)
Description
Extracts the `nth` subexpression of the most recent successful
......@@ -182,14 +208,19 @@ Description
Example
``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);``
.. _func_match_dyn:
.. _vmod_re.match_dyn:
match_dyn
---------
BOOL match_dyn(STRING, STRING, INT limit, INT limit_recursion)
--------------------------------------------------------------
::
BOOL match_dyn(PRIV_TASK, STRING, STRING)
BOOL match_dyn(
STRING,
STRING,
INT limit=1000,
INT limit_recursion=1000
)
Description
Compiles the regular expression given in the first argument,
......@@ -201,17 +232,24 @@ Description
with the tag ``VCL_Error``, and ``match_dyn`` returns
``false``.
For parameters ``limit`` and ``limit_recursion`` see
`vmod_re.regex.match`_, except that there is no object to inherit
defaults from.
Example
``if (re.match_dyn(req.http.Foo + "(\d+)", beresp.http.Bar)) { # ...``
.. _func_backref_dyn:
.. _vmod_re.backref_dyn:
backref_dyn
-----------
STRING backref_dyn(INT, STRING fallback)
----------------------------------------
::
STRING backref_dyn(PRIV_TASK, INT, STRING fallback="**BACKREF FUNCTION FAILED**")
STRING backref_dyn(
INT,
STRING fallback="**BACKREF FUNCTION FAILED**"
)
Description
Similar to the ``backref`` method, this function extracts the
......@@ -227,14 +265,10 @@ Description
in the same task scope, then a ``VCL_Error`` message is logged, and
the fallback string is returned.
.. _func_version:
.. _vmod_re.version:
version
-------
::
STRING version()
STRING version()
----------------
Description
Returns the version string for this vmod.
......@@ -245,86 +279,14 @@ Example
REQUIREMENTS
============
The VMOD requires the Varnish master branch since commit a339f63. See
the project repository for versions that are compatible with other
versions of Varnish.
The VMOD requires the Varnish since version 6.0.0 or the master
branch. See the project repository for versions that are compatible
with other versions of Varnish.
INSTALLATION
============
The VMOD is built on a system where an instance of Varnish is
installed, and the auto-tools will attempt to locate the Varnish
instance, and then pull in libraries and other support files from
there.
Quick start
-----------
This sequence should be enough in typical setups:
1. ``./autogen.sh`` (for git-installation)
2. ``./configure``
3. ``make``
4. ``make check`` (regression tests)
5. ``make install`` (may require root: sudo make install)
Alternative configs
-------------------
If you have installed Varnish to a non-standard directory, call
``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
the appropriate path. For example, when varnishd configure was called
with ``--prefix=$PREFIX``, use
PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
export PKG_CONFIG_PATH
For developers
--------------
As with Varnish, you can use these ``configure`` options for stricter
compiling:
* ``--enable-developer-warnings``
* ``--enable-extra-developer-warnings`` (for GCC 4)
* ``--enable-werror``
The VMOD must always build successfully with these options enabled.
Also as with Varnish, you can add ``--enable-debugging-symbols``, so
that the VMOD's symbols are available to debuggers, in core dumps and
so forth.
AUTHORS
=======
* Geoffrey Simmons <geoff@uplex.de>
* Nils Goroll <nils.goroll@uplex.de>
UPLEX Nils Goroll Systemoptimierung
HISTORY
=======
Version 0.1: Initial version, compatible with Varnish 3
Version 0.2: various fixes, last version compatible with Varnish 3
Version 0.3: compatible with Varnish 4
Version 0.4: support dynamic matches
Version 0.5: add the failed() and error() methods
Version 0.6: bugfix backrefs for which no string is captured
Version 1.0: stable version compatible with Varnish 4.0, maintained on
branch 4.0, before beginning upgrades for 4.1
Version 1.1: compatible with Varnish 5.0
Version 2.0: compatible with Varnish 5.1
See `INSTALL.rst <INSTALL.rst>`_ in the project source repository.
LIMITATIONS
===========
......@@ -352,28 +314,25 @@ full expression indicated by backref 0. If a ``match`` or
(10 substrings and the full string), then a ``VCL_Error`` message is
emitted to the Varnish log, and the captures are limited to 11.
Regular expression matching is subject to the same limitations that
hold for standard regexen in VCL, for example as set by the runtime
parameters ``pcre_match_limit`` and ``pcre_match_limit_recursion``.
SEE ALSO
========
* varnishd(1)
* vcl(7)
* pcre(3)
* http://lassekarstensen.wordpress.com/2013/12/19/converting-a-varnish-3-0-vmod-to-4-0/
* source repository: https://code.uplex.de/uplex-varnish/libvmod-re
COPYRIGHT
=========
::
Copyright (c) 2014-2015 UPLEX Nils Goroll Systemoptimierung
Copyright 2014-2018 UPLEX Nils Goroll Systemoptimierung
All rights reserved
This document is licensed under the same conditions as the libvmod-re
project. See LICENSE for details.
Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
Nils Goroll <nils.goroll@uplex.de>
AC_PREREQ(2.59)
AC_COPYRIGHT([Copyright (c) 2013-2015 UPLEX Nils Goroll Systemoptimierung])
AC_INIT([libvmod-re], [trunk])
AC_INIT([libvmod-re], [2.6.0])
AC_CONFIG_MACRO_DIR([m4])
m4_ifndef([VARNISH_VMOD_INCLUDES], AC_MSG_ERROR([Need varnish.m4 -- see README.rst]))
AC_CONFIG_SRCDIR(src/vmod_re.vcc)
......
#! /bin/bash -ex
# Build script for a VMOD re el7 RPM
# to be run in the rpmbuild/centos7 docker container.
# Env variables VERSION and RELEASE MUST be set in the docker invocation.
# DIST is set by the rpmbuild/centos7 container.
# The directory mounted to /srv MUST contain these files:
# - this script (named pkg)
# - VMOD re tarball (libvmod-re-v$VERSION.tar.gz)
# - RPM spec file (vmod-re.spec)
# - yum repo config for varnish61@packagecloud (varnishcache_varnish61.repo)
# (see https://packagecloud.io/varnishcache/varnish61/install#manual-rpm)
# So this is a sample docker invocation:
#
# $ docker run -t -v $PWD:/srv -e VERSION=1.2.3 -e RELEASE=4711 \
# rpmbuild/centos7 /srv/pkg
# At the end of the run, binary, source and debuginfo RPMs are in the
# directory mounted to /srv.
if [ -z $VERSION ]; then
echo "Env variable VERSION MUST be set"
exit 1
fi
if [ -z $RELEASE ]; then
echo "Env variable RELEASE MUST be set"
exit 1
fi
# delete the peculiar macros from the rpmbuild/centos7 image
rm /home/builder/.rpmmacros
# set up the build environment
cd /home/builder
mkdir -p rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
cp /srv/vmod-re.spec rpmbuild/SPECS
cp /srv/libvmod-re-${VERSION}.tar.gz rpmbuild/SOURCES
# install epel7 repo
sudo yum install -y -q \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# set up varnish repo
sudo cp /srv/varnishcache_varnish61.repo /etc/yum.repos.d/
sudo yum -q makecache -y --disablerepo='*' --enablerepo='varnishcache_varnish61'
# build requirements
sudo yum install -y -q varnish-devel pkgconfig make gcc python-docutils
# build RPMs
rpmbuild -ba -D "dist .${DIST}" \
-D "_version ${VERSION}" \
-D "_release ${RELEASE}" \
rpmbuild/SPECS/vmod-re.spec
sudo cp rpmbuild/RPMS/*/* /srv
sudo cp rpmbuild/SRPMS/* /srv
[varnishcache_varnish61]
name=varnishcache_varnish61
baseurl=https://packagecloud.io/varnishcache/varnish61/el/7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/varnishcache/varnish61/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
[varnishcache_varnish61-source]
name=varnishcache_varnish61-source
baseurl=https://packagecloud.io/varnishcache/varnish61/el/7/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/varnishcache/varnish61/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
# -D MUST pass in _version and _release, and SHOULD pass in dist.
Summary: Regular Expression module for Varnish Cache
Name: vmod-re
Version: %{_version}
Release: %{_release}%{?dist}
License: BSD
Group: System Environment/Daemons
URL: https://code.uplex.de/uplex-varnish/libvmod-re
Source0: lib%{name}-%{version}.tar.gz
# varnish from varnish60 at packagecloud
# The Requires specifies ABI >= 8.0 (Varnish 6.1, libvarnishapi.so.2)
Requires: varnishd(vrt)%{?_isa} >= 8
BuildRequires: varnish-devel >= 6.1.0
BuildRequires: pkgconfig
BuildRequires: make
BuildRequires: gcc
BuildRequires: python-docutils >= 0.6
# git builds
#BuildRequires: automake
#BuildRequires: autoconf
#BuildRequires: autoconf-archive
#BuildRequires: libtool
#BuildRequires: python-docutils >= 0.6
Provides: vmod-re, vmod-re-debuginfo
%description
Varnish Module (VMOD) for matching strings against regular
expressions, and for extracting captured substrings after matches.
%prep
%setup -q -n lib%{name}-%{version}
%build
# if this were a git build
# ./autogen.sh
%configure
make -j
%check
make check -j
%install
make install DESTDIR=%{buildroot}
# Only use the version-specific docdir created by %doc below
rm -rf %{buildroot}%{_docdir}
# None of these for fedora/epel
find %{buildroot}/%{_libdir}/ -name '*.la' -exec rm -f {} ';'
find %{buildroot}/%{_libdir}/ -name '*.a' -exec rm -f {} ';'
%clean
rm -rf %{buildroot}
%files
%defattr(-,root,root,-)
%{_libdir}/varnish*/vmods/
%{_mandir}/man3/*.3*
%doc README.rst COPYING LICENSE
%post
/sbin/ldconfig
%changelog
......@@ -12,12 +12,14 @@ libvmod_re_la_SOURCES = \
vcc_if.c: vcc_if.h
vcc_if.h vmod_re.man.rst: @VMODTOOL@ $(top_srcdir)/src/vmod_re.vcc
@VMODTOOL@ $(top_srcdir)/src/vmod_re.vcc
@PYTHON@ @VMODTOOL@ $(top_srcdir)/src/vmod_re.vcc
vmod_re.c: vcc_if.h
VMOD_TESTS = $(top_srcdir)/src/tests/*.vtc
.PHONY: $(VMOD_TESTS)
$(top_srcdir)/src/tests/*.vtc:
$(top_srcdir)/src/tests/*.vtc: libvmod_re.la
PATH=@LIBVARNISHAPI_SBINDIR@:$$PATH \
@VARNISHTEST@ -Dvmod_topbuild=$(abs_top_builddir) $@
......
varnishtest "regex compilation failure"
varnishtest "regex constructor errors"
varnish v1 -vcl { backend b { .host = "${bad_ip}"; } } -start
......@@ -10,3 +10,21 @@ varnish v1 -errvcl {vmod re: error compiling regex} {
new paren = re.regex("(");
}
}
varnish v1 -errvcl {vmod re: invalid limit 0 in bad constructor} {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
backend b { .host = "${bad_ip}"; }
sub vcl_init {
new bad = re.regex(".", limit=0);
}
}
varnish v1 -errvcl {vmod re: invalid limit_recursion 0 in bad constructor} {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
backend b { .host = "${bad_ip}"; }
sub vcl_init {
new bad = re.regex(".", limit_recursion=0);
}
}
/*-
* Copyright (c) 2013 - 2014 UPLEX Nils Goroll Systemoptimierung
* Copyright 2013 - 2018 UPLEX Nils Goroll Systemoptimierung
* All rights reserved
*
* Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
......@@ -30,12 +30,13 @@
#include "config.h"
#include <stdlib.h>
#include <string.h>
#include "cache/cache.h"
#include "vcl.h"
#include "vre.h"
#include "cache/cache.h"
#include "vrt.h"
#include "vas.h"
#include "vsb.h"
#include "miniobj.h"
#include "vcc_if.h"
......@@ -57,16 +58,17 @@
#define MAX_OV_USED ((MAX_MATCHES) * 2)
struct vmod_re_regex {
unsigned magic;
unsigned magic;
#define VMOD_RE_REGEX_MAGIC 0x955706ee
vre_t *vre;
vre_t *vre;
struct vre_limits vre_limits;
};
typedef struct ov_s {
unsigned magic;
unsigned magic;
#define OV_MAGIC 0x844bfa39
const char *subject;
int ovector[MAX_OV_USED];
const char *subject;
int ovector[MAX_OV_USED];
} ov_t;
static void
......@@ -74,25 +76,18 @@ errmsg(VRT_CTX, const char *fmt, ...)
{
va_list args;
AZ(ctx->method & VCL_MET_TASK_H);
va_start(args, fmt);
if (ctx->vsl)
VSLbv(ctx->vsl, SLT_VCL_Error, fmt, args);
else
VSLv(SLT_VCL_Error, 0, fmt, args);
va_end(args);
if (ctx->method == VCL_MET_INIT) {
AN(ctx->msg);
va_start(args, fmt);
VSB_vprintf(ctx->msg, fmt, args);
va_end(args);
VRT_handling(ctx, VCL_RET_FAIL);
}
}
VCL_VOID
vmod_regex__init(VRT_CTX, struct vmod_re_regex **rep, const char *vcl_name,
VCL_STRING pattern)
VCL_STRING pattern, VCL_INT limit, VCL_INT limit_recursion)
{
struct vmod_re_regex *re;
vre_t *vre;
......@@ -105,15 +100,31 @@ vmod_regex__init(VRT_CTX, struct vmod_re_regex **rep, const char *vcl_name,
AN(vcl_name);
AN(pattern);
if ((vre = VRE_compile(pattern, 0, &error, &erroffset)) == NULL)
errmsg(ctx, "vmod re: error compiling regex \"%s\" in %s "
"constructor: %s (at offset %d)", pattern, vcl_name,
error, erroffset);
if (limit < 1) {
VRT_fail(ctx, "vmod re: invalid limit %d in %s constructor",
limit, vcl_name);
return;
}
if (limit_recursion < 1) {
VRT_fail(ctx, "vmod re: invalid limit_recursion %d "
"in %s constructor", limit_recursion, vcl_name);
return;
}
if ((vre = VRE_compile(pattern, 0, &error, &erroffset)) == NULL) {
VRT_fail(ctx, "vmod re: error compiling regex \"%s\" in %s "
"constructor: %s (at offset %d)", pattern, vcl_name,
error, erroffset);
return;
}
ALLOC_OBJ(re, VMOD_RE_REGEX_MAGIC);
AN(re);
*rep = re;
re->vre = vre;
re->vre_limits.match = limit;
re->vre_limits.match_recursion = limit_recursion;
*rep = re;
}
VCL_VOID
......@@ -132,7 +143,8 @@ vmod_regex__fini(struct vmod_re_regex **rep)
}
static VCL_BOOL
match(VRT_CTX, vre_t *vre, VCL_STRING subject, struct vmod_priv *task)
match(VRT_CTX, vre_t *vre, VCL_STRING subject, struct vmod_priv *task,
const struct vre_limits *vre_limits)
{
ov_t *ov;
int s, nov[MAX_OV];
......@@ -159,8 +171,7 @@ match(VRT_CTX, vre_t *vre, VCL_STRING subject, struct vmod_priv *task)
}
len = strlen(subject);
s = VRE_exec(vre, subject, len, 0, 0, nov, MAX_OV,
&cache_param->vre_limits);
s = VRE_exec(vre, subject, len, 0, 0, nov, MAX_OV, vre_limits);
if (s <= VRE_ERROR_NOMATCH) {
if (s < VRE_ERROR_NOMATCH)
VSLb(ctx->vsl, SLT_VCL_Error,
......@@ -194,7 +205,7 @@ static VCL_STRING
backref(VRT_CTX, VCL_INT refnum, VCL_STRING fallback, struct vmod_priv *task)
{
ov_t *ov;
char *substr;
const char *substr, *start;
int len;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
......@@ -223,11 +234,14 @@ backref(VRT_CTX, VCL_INT refnum, VCL_STRING fallback, struct vmod_priv *task)
if (ov->ovector[refnum] == -1)
return fallback;
start = ov->subject + ov->ovector[refnum];
len = ov->ovector[refnum+1] - ov->ovector[refnum];
assert(len <= ov->ovector[1] - ov->ovector[0]);
WS_Assert_Allocated(ctx->ws, ov->subject + ov->ovector[refnum], len);
substr = WS_Printf(ctx->ws, "%.*s", len,
ov->subject + ov->ovector[refnum]);
WS_Assert_Allocated(ctx->ws, start, len + 1);
if (start[len] == '\0')
substr = start;
else
substr = WS_Printf(ctx->ws, "%.*s", len, start);
if (substr == NULL) {
VSLb(ctx->vsl, SLT_VCL_Error,
"vmod re: insufficient workspace");
......@@ -236,19 +250,49 @@ backref(VRT_CTX, VCL_INT refnum, VCL_STRING fallback, struct vmod_priv *task)
return substr;
}
static inline const struct vre_limits *
get_limits(const struct vmod_re_regex *re, struct vre_limits *limits,
VCL_INT limit, VCL_INT limit_recursion)
{
if (limit <= 0 && limit_recursion <= 0)
return (&re->vre_limits);
if (limit > 0)
limits->match = limit;
else
limits->match = re->vre_limits.match;
if (limit_recursion > 0)
limits->match_recursion = limit_recursion;
else
limits->match_recursion = re->vre_limits.match_recursion;
return (limits);
}
VCL_BOOL
vmod_regex_match(VRT_CTX, struct vmod_re_regex *re, VCL_STRING subject)
vmod_regex_match(VRT_CTX, struct vmod_re_regex *re, VCL_STRING subject,
VCL_INT limit, VCL_INT limit_recursion)
{
struct vmod_priv *task;
struct vre_limits buf;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
AN(re->vre);
task = VRT_priv_task(ctx, re);
if (task == NULL) {
errmsg(ctx, "vmod re: no priv - out of workspace?");
return (0);
}
AN(task);
task->len = 0;
return match(ctx, re->vre, subject, task);
return match(ctx, re->vre, subject, task,
get_limits(re, &buf, limit, limit_recursion));
}
VCL_STRING
......@@ -260,21 +304,41 @@ vmod_regex_backref(VRT_CTX, struct vmod_re_regex *re, VCL_INT refnum,
CHECK_OBJ_NOTNULL(re, VMOD_RE_REGEX_MAGIC);
task = VRT_priv_task(ctx, re);
if (task == NULL) {
errmsg(ctx, "vmod re: no priv - out of workspace?");
return (0);
}
return backref(ctx, refnum, fallback, task);
}
VCL_BOOL
vmod_match_dyn(VRT_CTX, struct vmod_priv *task, VCL_STRING pattern,
VCL_STRING subject)
VCL_STRING subject, VCL_INT limit, VCL_INT limit_recursion)
{
vre_t *vre;
int erroffset;
const char *error;
VCL_BOOL dyn_return;
struct vre_limits limits;
AN(pattern);
AN(task);
if (limit < 1) {
errmsg(ctx, "vmod re: invalid limit %d for regex \"%s\"",
limit, pattern);
return 0;
}
if (limit_recursion < 1) {
errmsg(ctx, "vmod re: invalid limit_recursion %d "
"for regex \"%s\"", limit_recursion, pattern);
return 0;
}
limits.match = limit;
limits.match_recursion = limit_recursion;
task->len = 0;
vre = VRE_compile(pattern, 0, &error, &erroffset);
if (vre == NULL) {
......@@ -283,7 +347,9 @@ vmod_match_dyn(VRT_CTX, struct vmod_priv *task, VCL_STRING pattern,
pattern, error, erroffset);
return 0;
}
dyn_return = match(ctx, vre, subject, task);
dyn_return = match(ctx, vre, subject, task, &limits);
VRE_free(&vre);
return dyn_return;
}
......
#-
# Copyright (c) 2014-2015 UPLEX Nils Goroll Systemoptimierung
# Copyright 2014-2018 UPLEX Nils Goroll Systemoptimierung
# All rights reserved
#
# This document is licensed under the same conditions as the libvmod-re
# project. See LICENSE for details.
#
# Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
# Authors: Geoffrey Simmons <geoffrey.simmons@uplex.de>
# Nils Goroll <nils.goroll@uplex.de>
$Module re 3 Varnish Module for Regular Expression Matching with Subexpression Capture
$Module re 3 "Varnish Module for Regular Expression Matching with Subexpression Capture"
$Synopsis manual
SYNOPSIS
========
::
import re;
# object interface
new <obj> = re.regex(STRING, [, INT limit] [, INT limit_recursion])
BOOL <obj>.match(STRING, [, INT limit] [, INT limit_recursion])
STRING <obj>.backref(INT [, STRING fallback])
# function interface
BOOL re.match_dyn(STRING, [, INT limit] [, INT limit_recursion])
STRING re.backref_dyn(INT [, STRING fallback])
STRING re.version()
DESCRIPTION
===========
......@@ -88,7 +109,7 @@ since it re-uses the compiled expression obtained at VCL
initialization. So if you are matching against a fixed pattern that
never changes during the lifetime of VCL, use ``match``.
$Object regex(STRING)
$Object regex(STRING, INT limit=1000, INT limit_recursion=1000)
Description
Create a regex object with the given regular expression. The
......@@ -99,16 +120,33 @@ Description
If the regular expression fails to compile, then the VCL
load fails with an error message describing the problem.
The optional parameters ``limit`` and ``limit_recursion`` are
per-object defaults for the respective parameters of the
`vmod_re.regex.match`_ method.
Example
``new myregex = re.regex("\bmax-age\s*=\s*(\d+)");``
$Method BOOL .match(STRING)
$Method BOOL .match(STRING, INT limit=0, INT limit_recursion=0)
Description
Determines whether the given string matches the regex compiled
by the constructor; functionally equivalent to VCL's infix
operator ``~``.
The optional parameter ``limit`` restricts the number of
internal matching function calls in a ``pcre_exec()``
execution, analogous to the varnishd ``pcre_match_limit``
parameter. For the default value 0, the ``limit`` given to the
constructor `vmod_re.regex`_ is used.
The optional parameter ``limit_recursion`` restricts the
number of internal matching function recursions in a
``pcre_exec()`` execution, analogous to the varnishd
``pcre_match_limit_recursion`` parameter. For the default
value 0, the ``limit_recursion`` given to the constructor
`vmod_re.regex`_ is used.
Example
``if (myregex.match(beresp.http.Surrogate-Control)) { # ...``
......@@ -138,7 +176,8 @@ Description
Example
``set beresp.ttl = std.duration(myregex.backref(1, "120"), 120s);``
$Function BOOL match_dyn(PRIV_TASK, STRING, STRING)
$Function BOOL match_dyn(PRIV_TASK, STRING, STRING,
INT limit=1000, INT limit_recursion=1000)
Description
Compiles the regular expression given in the first argument,
......@@ -150,6 +189,10 @@ Description
with the tag ``VCL_Error``, and ``match_dyn`` returns
``false``.
For parameters ``limit`` and ``limit_recursion`` see
`vmod_re.regex.match`_, except that there is no object to inherit
defaults from.
Example
``if (re.match_dyn(req.http.Foo + "(\d+)", beresp.http.Bar)) { # ...``
......@@ -181,86 +224,14 @@ Example
REQUIREMENTS
============
The VMOD requires the Varnish master branch since commit a339f63. See
the project repository for versions that are compatible with other
versions of Varnish.
The VMOD requires the Varnish since version 6.0.0 or the master
branch. See the project repository for versions that are compatible
with other versions of Varnish.
INSTALLATION
============
The VMOD is built on a system where an instance of Varnish is
installed, and the auto-tools will attempt to locate the Varnish
instance, and then pull in libraries and other support files from
there.
Quick start
-----------
This sequence should be enough in typical setups:
1. ``./autogen.sh`` (for git-installation)
2. ``./configure``
3. ``make``
4. ``make check`` (regression tests)
5. ``make install`` (may require root: sudo make install)
Alternative configs
-------------------
If you have installed Varnish to a non-standard directory, call
``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
the appropriate path. For example, when varnishd configure was called
with ``--prefix=$PREFIX``, use
PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
export PKG_CONFIG_PATH
For developers
--------------
As with Varnish, you can use these ``configure`` options for stricter
compiling:
* ``--enable-developer-warnings``
* ``--enable-extra-developer-warnings`` (for GCC 4)
* ``--enable-werror``
The VMOD must always build successfully with these options enabled.
Also as with Varnish, you can add ``--enable-debugging-symbols``, so
that the VMOD's symbols are available to debuggers, in core dumps and
so forth.
AUTHORS
=======
* Geoffrey Simmons <geoff@uplex.de>
* Nils Goroll <nils.goroll@uplex.de>
UPLEX Nils Goroll Systemoptimierung
HISTORY
=======
Version 0.1: Initial version, compatible with Varnish 3
Version 0.2: various fixes, last version compatible with Varnish 3
Version 0.3: compatible with Varnish 4
Version 0.4: support dynamic matches
Version 0.5: add the failed() and error() methods
Version 0.6: bugfix backrefs for which no string is captured
Version 1.0: stable version compatible with Varnish 4.0, maintained on
branch 4.0, before beginning upgrades for 4.1
Version 1.1: compatible with Varnish 5.0
Version 2.0: compatible with Varnish 5.1
See `INSTALL.rst <INSTALL.rst>`_ in the project source repository.
LIMITATIONS
===========
......@@ -288,14 +259,12 @@ full expression indicated by backref 0. If a ``match`` or
(10 substrings and the full string), then a ``VCL_Error`` message is
emitted to the Varnish log, and the captures are limited to 11.
Regular expression matching is subject to the same limitations that
hold for standard regexen in VCL, for example as set by the runtime
parameters ``pcre_match_limit`` and ``pcre_match_limit_recursion``.
SEE ALSO
========
* varnishd(1)
* vcl(7)
* pcre(3)
* http://lassekarstensen.wordpress.com/2013/12/19/converting-a-varnish-3-0-vmod-to-4-0/
* source repository: https://code.uplex.de/uplex-varnish/libvmod-re
$ABI vrt