Commit ff4d7838 authored by Geoff Simmons's avatar Geoff Simmons

Initial

parents
Makefile
Makefile.in
aclocal.m4
autom4te.cache/
config.guess
config.h
config.h.in
config.log
config.status
config.sub
configure
depcomp
install-sh
libtool
ltmain.sh
m4/libtool.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4
missing
src/.deps/
src/.libs/
src/Makefile
src/Makefile.in
src/libvmod_re.la
src/vcc_if.c
src/vcc_if.h
src/vcc_if.lo
src/vcc_if.o
src/vmod_re.lo
src/vmod_re.o
stamp-h1
vmod_re.3
*~
\ No newline at end of file
Copyright (c) 2013 UPLEX Nils Goroll Systemoptimierung
...
See LICENSE for details.
You're free to use and distribute this under terms in the
LICENSE. Please add your relevant copyright statements.
Copyright (c) 2013 UPLEX Nils Goroll Systemoptimierung
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = src
dist_man_MANS = vmod_re.3
MAINTAINERCLEANFILES = $(dist_man_MANS)
EXTRA_DIST = README.rst
vmod_re.3: README.rst
if HAVE_RST2MAN
${RST2MAN} README.rst $@
else
@echo "========================================"
@echo "You need rst2man installed to make dist"
@echo "========================================"
@false
endif
=======
vmod_re
=======
-------------------------------------------------------------------
Varnish Module for Regular Expression Matching with Backref Capture
-------------------------------------------------------------------
:Manual section: 3
:Author: Geoffrey Simmons
:Date: 2013-09-09
:Version: 0.1
SYNOPSIS
========
::
import re;
re.match(<string>, <regular expresssion>)
re.backref(<integer>, <fallback>)
re.version()
DESCRIPTION
===========
Varnish Module (vmod) for matching strings against regular expressions,
and for extracting captured substrings after matches.
FUNCTIONS
=========
Example VCL::
import re;
sub vcl_recv {
if (re.match(req.http.Cookie, "\bfoo=(.+)\b")) {
set req.http.Foo = re.backref(1, "");
}
}
match
-----
Prototype
re.match(<string>, <regular expression>)
Returns
boolean
Description
Determines whether a string matches the given regular expression;
functionally equivalent to VCL's infix operator `~`. It is subject
to the same limitations, for example as set by the runtime
paramters `pcre_match_limit` and `pcre_match_limit_recursion`.
Example
``re.match(beresp.http.Surrogate-Control, "max-age=(\d+);mysite")``
backref
-------
Prototype
re.backref(<integer>, <fallback>)
Returns
String
Description
Extracts the `nth` subexpression of the most recent successful
call to `re.match()` in the current session, or a fallback string
in case the extraction fails. Backref 0 indicates the full match.
Thus this function behaves like the `\\n` symbols in `regsub`
and `regsuball`, and the `$1`, `$2` ... variables in Perl.
After unsuccessful matches, the `fallback` string is returned
for any call to `re.backref()`.
The VCL infix operators `~` and `!~` do not affect this function,
nor do `regsub` or `regsuball`.
`re.backref` can extract up to 10 subexpressions, in addition to
to the full expression indicated by backref 0.
Example
``set beresp.ttl = std.duration(re.backref(1, "120"), 120s);``
version
-------
Prototype
re.version()
Returns
string
Description
Returns the string constant version-number of the header vmod.
Example
``set resp.http.X-re-version = re.version();``
INSTALLATION
============
Installation requires the Varnish source tree (only the source matching the
binary installation).
1. `./autogen.sh` (for git-installation)
2. `./configure VARNISHSRC=/path/to/your/varnish/source/varnish-cache`
3. `make`
4. `make install` (may require root: sudo make install)
5. `make check` (Optional for regression tests)
VARNISHSRCDIR is the directory of the Varnish source tree for which to
compile your vmod. Both the VARNISHSRCDIR and VARNISHSRCDIR/include
will be added to the include search paths for your module.
Optionally you can also set the vmod install dir by adding VMODDIR=DIR
(defaults to the pkg-config discovered directory from your Varnish
installation).
ACKNOWLEDGEMENTS
================
Author: Geoffrey Simmons <geoff@uplex.de>, UPLEX Nils Goroll Systemoptimierung.
The implementation was inspired by ideas from Nils Goroll's esicookies VMOD
and pmatch patch for Varnish 2, and by Kristian Lyngstøl's header VMOD.
HISTORY
=======
Version 0.1: Initial version
BUGS
====
You can't use dynamic regular expressions, which also holds true for normal
regular expressions in regsub(), but VCL isn't able to warn you about this
when it comes to vmods yet.
SEE ALSO
========
* varnishd(1)
* vcl(7)
COPYRIGHT
=========
This document is licensed under the same license as the
libvmod-header project. See LICENSE for details.
* Copyright (c) 2013 UPLEX Nils Goroll Systemoptimierung
#!/bin/sh
warn() {
echo "WARNING: $@" 1>&2
}
case `uname -s` in
Darwin)
LIBTOOLIZE=glibtoolize
;;
FreeBSD)
LIBTOOLIZE=libtoolize
;;
Linux)
LIBTOOLIZE=libtoolize
;;
SunOS)
LIBTOOLIZE=libtoolize
;;
*)
warn "unrecognized platform:" `uname -s`
LIBTOOLIZE=libtoolize
esac
automake_version=`automake --version | tr ' ' '\n' | egrep '^[0-9]\.[0-9a-z.-]+'`
if [ -z "$automake_version" ] ; then
warn "unable to determine automake version"
else
case $automake_version in
0.*|1.[0-8]|1.[0-8][.-]*)
warn "automake ($automake_version) detected; 1.9 or newer recommended"
;;
*)
;;
esac
fi
set -ex
$LIBTOOLIZE --copy --force
aclocal -I m4
autoheader
automake --add-missing --copy --foreign
autoconf
AC_PREREQ(2.59)
AC_COPYRIGHT([Copyright (c) 2013 UPLEX Nils Goroll Systemoptimierung])
AC_INIT([libvmod-header], [0.1])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_SRCDIR(src/vmod_re.vcc)
AM_CONFIG_HEADER(config.h)
AC_CANONICAL_SYSTEM
AC_LANG(C)
AM_INIT_AUTOMAKE([foreign])
AC_GNU_SOURCE
AC_PROG_CC
AC_PROG_CC_STDC
if test "x$ac_cv_prog_cc_c99" = xno; then
AC_MSG_ERROR([Could not find a C99 compatible compiler])
fi
AC_PROG_CPP
AC_PROG_INSTALL
AC_PROG_LIBTOOL
AC_PROG_MAKE_SET
# Check for rst utilities
AC_CHECK_PROGS(RST2MAN, [rst2man rst2man.py], "no")
if test "x$RST2MAN" = "xno"; then
AC_MSG_WARN([rst2man not found - not building man pages])
fi
AM_CONDITIONAL(HAVE_RST2MAN, [test "x$RST2MAN" != "xno"])
# Check for pkg-config
PKG_PROG_PKG_CONFIG
# Checks for header files.
AC_HEADER_STDC
AC_CHECK_HEADERS([sys/stdlib.h])
# Check for python
AC_CHECK_PROGS(PYTHON, [python3 python3.1 python3.2 python2.7 python2.6 python2.5 python2 python], [AC_MSG_ERROR([Python is needed to build this vmod, please install python.])])
# Varnish source tree
AC_ARG_VAR([VARNISHSRC], [path to Varnish source tree (mandatory)])
if test "x$VARNISHSRC" = x; then
AC_MSG_ERROR([No Varnish source tree specified])
fi
VARNISHSRC=`cd $VARNISHSRC && pwd`
AC_CHECK_FILE([$VARNISHSRC/include/varnishapi.h],
[],
[AC_MSG_FAILURE(["$VARNISHSRC" is not a Varnish source directory])]
)
# Check that varnishtest is built in the varnish source directory
AC_CHECK_FILE([$VARNISHSRC/bin/varnishtest/varnishtest],
[],
[AC_MSG_FAILURE([Can't find "$VARNISHSRC/bin/varnishtest/varnishtest". Please build your varnish source directory])]
)
# vmod installation dir
AC_ARG_VAR([VMODDIR], [vmod installation directory @<:@LIBDIR/varnish/vmods@:>@])
if test "x$VMODDIR" = x; then
VMODDIR=`pkg-config --variable=vmoddir varnishapi`
if test "x$VMODDIR" = x; then
AC_MSG_FAILURE([Can't determine vmod installation directory])
fi
fi
# This corresponds to FreeBSD's WARNS level 6
DEVELOPER_CFLAGS="-Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wformat"
# Additional flags for GCC 4
EXTRA_DEVELOPER_CFLAGS="-Wextra -Wno-missing-field-initializers -Wno-sign-compare"
# --enable-developer-warnings
AC_ARG_ENABLE(developer-warnings,
AS_HELP_STRING([--enable-developer-warnings],[enable strict warnings (default is NO)]),
CFLAGS="${CFLAGS} ${DEVELOPER_CFLAGS}")
# --enable-debugging-symbols
AC_ARG_ENABLE(debugging-symbols,
AS_HELP_STRING([--enable-debugging-symbols],[enable debugging symbols (default is NO)]),
CFLAGS="${CFLAGS} -O0 -g -fno-inline")
# --enable-diagnostics
AC_ARG_ENABLE(diagnostics,
AS_HELP_STRING([--enable-diagnostics],[enable run-time diagnostics (default is NO)]),
CFLAGS="${CFLAGS} -DDIAGNOSTICS")
# --enable-extra-developer-warnings
AC_ARG_ENABLE(extra-developer-warnings,
AS_HELP_STRING([--enable-extra-developer-warnings],[enable even stricter warnings (default is NO)]),
[],
[enable_extra_developer_warnings=no])
if test "x$enable_stack_protector" != "xno"; then
save_CFLAGS="$CFLAGS"
CFLAGS="${CFLAGS} ${EXTRA_DEVELOPER_CFLAGS}"
AC_COMPILE_IFELSE(
[AC_LANG_PROGRAM([],[],[])],
[],
[AC_MSG_WARN([All of ${EXTRA_DEVELOPER_CFLAGS} not supported, disabling])
CFLAGS="$save_CFLAGS"])
fi
# --enable-stack-protector
AC_ARG_ENABLE(stack-protector,
AS_HELP_STRING([--enable-stack-protector],[enable stack protector (default is NO)]),
[],
[enable_stack_protector=no])
if test "x$enable_stack_protector" != "xno"; then
save_CFLAGS="$CFLAGS"
CFLAGS="${CFLAGS} -fstack-protector-all"
AC_COMPILE_IFELSE(
[AC_LANG_PROGRAM([],[],[])],
[],
[AC_MSG_WARN([-fstack-protector not supported, disabling])
CFLAGS="$save_CFLAGS"])
fi
# --enable-tests
AC_ARG_ENABLE(tests,
AS_HELP_STRING([--enable-tests],[build test programs (default is NO)]))
AM_CONDITIONAL([ENABLE_TESTS], [test x$enable_tests = xyes])
# --enable-werror
AC_ARG_ENABLE(werror,
AS_HELP_STRING([--enable-werror],[use -Werror (default is NO)]),
CFLAGS="${CFLAGS} -Werror")
AC_CONFIG_FILES([
Makefile
src/Makefile
])
AC_OUTPUT
INCLUDES = -I$(VARNISHSRC)/include -I$(VARNISHSRC)
vmoddir = $(VMODDIR)
vmod_LTLIBRARIES = libvmod_re.la
libvmod_re_la_LDFLAGS = -module -export-dynamic -avoid-version
libvmod_re_la_SOURCES = \
vcc_if.c \
vcc_if.h \
vmod_re.c
vcc_if.c vcc_if.h: $(VARNISHSRC)/lib/libvmod_std/vmod.py $(top_srcdir)/src/vmod_re.vcc
@PYTHON@ $(VARNISHSRC)/lib/libvmod_std/vmod.py $(top_srcdir)/src/vmod_re.vcc
VMOD_TESTS = tests/*.vtc
.PHONY: $(VMOD_TESTS)
tests/*.vtc:
$(VARNISHSRC)/bin/varnishtest/varnishtest -Dvarnishd=$(VARNISHSRC)/bin/varnishd/varnishd -Dvmod_topbuild=$(abs_top_builddir) $@
check: $(VMOD_TESTS)
EXTRA_DIST = \
vmod_re.vcc \
$(VMOD_TESTS)
CLEANFILES = $(builddir)/vcc_if.c $(builddir)/vcc_if.h
varnishtest "re-vmod: Test basic import"
server s1 {
timeout 10
rxreq
expect req.url == "/"
txresp -status 200 -hdr "foo: 1"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
} -start
client c1 {
txreq -url "/"
rxresp
expect resp.status == 200
expect resp.http.foo == 1
} -run
# Somewhat oversimplified, but at least it ensures that SOMETHING works
# (The alternative would be to automakeify this test or manually update it
# every time there's a new version, which defeats the purpose somewhat).
varnishtest "re-vmod: Test version string"
server s1 {
rxreq
expect req.url == "/"
txresp -status 200
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
set beresp.http.x-version = re.version();
if (!beresp.http.x-version) {
set beresp.status = 500;
}
return(deliver);
}
} -start
client c1 {
txreq -url "/"
rxresp
expect resp.status == 200
} -run
varnishtest "regexp match and no-match (cf. varnish b00028.vtc & v00016.vtc)"
server s1 {
rxreq
txresp -hdr "Foo: bar" -hdr "Bar: foo" -body "1111\n"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_recv {
if ((re.match(req.url, "foobar"))) {
return(pass);
} else if (re.match(req.url, "snafu")) {
return(pipe);
} else {
return(pass);
}
}
sub vcl_fetch {
if (re.match(beresp.http.foo, "bar")) {
set beresp.http.foo1 = "1";
} else {
error 999;
}
if (!re.match(beresp.http.bar, "bar")) {
set beresp.http.bar1 = "2";
} else {
error 999;
}
}
} -start
client c1 {
txreq
rxresp
expect resp.http.foo1 == "1"
expect resp.http.bar1 == "2"
} -run
varnishtest "cached compiled regexen"
# run two clients, to test caching of compiled regexen
server s1 {
rxreq
txresp -hdr "Foo: barbaz" -hdr "Bar: bazquux" -body "1111\n"
rxreq
txresp -hdr "Foo: barbaz" -hdr "Bar: bazquux" -body "1111\n"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
if (re.match(beresp.http.foo, "(bar)(baz)")) {
set beresp.http.foo1 = re.backref(1, "error1");
set beresp.http.foo2 = re.backref(2, "error2");
} else {
error 999;
}
}
} -start
client c1 {
txreq
rxresp
expect resp.http.foo1 == "bar"
expect resp.http.foo2 == "baz"
} -run
client c2 {
txreq -url /foo
rxresp
expect resp.http.foo1 == "bar"
expect resp.http.foo2 == "baz"
} -run
varnishtest "strings over 256 bytes (== varnish r00896.vtc)"
server s1 {
rxreq
txresp
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_recv {
if (re.match(req.http.host, "^(abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij|abcdefghij)")) {
error 500 "not ok";
}
}
} -start
client c1 {
txreq
rxresp
expect resp.status == 200
} -run
varnishtest "test null match (varnish r00913.vtc)"
server s1 {
rxreq
expect req.url == "/bar"
txresp -body "foobar"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
if (re.match(beresp.http.bar, "$")) {
set beresp.http.foo = "XXX";
}
if (re.match(beresp.http.foo, "")) {
set beresp.http.bar = "YYY";
}
}
} -start
client c1 {
txreq -url /bar -hdr "Foo: bar"
rxresp
expect resp.http.content-length == 6
expect resp.http.foo == "XXX"
expect resp.http.bar == "YYY"
} -run
varnishtest "basic backrefs"
server s1 {
rxreq
txresp -hdr "Foo: barbaz" -hdr "Bar: bazquux" -hdr "Barf: barf" \
-body "1111\n"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
if (re.match(beresp.http.foo, "(bar)(baz)")) {
set beresp.http.foo0 = re.backref(0, "error0");
set beresp.http.foo1 = re.backref(1, "error1");
set beresp.http.foo2 = re.backref(2, "error2");
set beresp.http.foo3 = re.backref(3, "foofallback");
} else {
error 999;
}
if (re.match(beresp.http.bar, "(baz)(.+)")) {
set beresp.http.bar0 = re.backref(0, "error0");
set beresp.http.bar1 = re.backref(1, "error1");
set beresp.http.bar2 = re.backref(2, "error2");
set beresp.http.bar3 = re.backref(3, "barfallback");
} else {
error 999;
}
if (re.match(beresp.http.barf, "(.)(.)(.)(.)")) {
set beresp.http.frap
= "_" + re.backref(0, "error0") + "_"
+ re.backref(5, "")
+ re.backref(4, "error4")
+ re.backref(3, "error3")
+ re.backref(2, "error2") + "p_";
} else {
error 999;
}
if (re.match(beresp.http.foo, "(frob)(nitz)")) {
set beresp.http.frob = "nitz";
}
set beresp.http.frob0 = re.backref(0, "fallback0");
set beresp.http.frob1 = re.backref(1, "fallback1");
set beresp.http.frob2 = re.backref(2, "fallback2");
}
} -start
client c1 {
txreq
rxresp
expect resp.http.foo0 == "barbaz"
expect resp.http.foo1 == "bar"
expect resp.http.foo2 == "baz"
expect resp.http.foo3 == "foofallback"
expect resp.http.bar0 == "bazquux"
expect resp.http.bar1 == "baz"
expect resp.http.bar2 == "quux"
expect resp.http.bar3 == "barfallback"
expect resp.http.frap == "_barf_frap_"
expect resp.http.frob != "nitz"
expect resp.http.frob0 == "fallback0"
expect resp.http.frob1 == "fallback1"
expect resp.http.frob2 == "fallback2"
} -run
varnishtest "non-capturing parentheses"
server s1 {
rxreq
txresp -hdr "Foo: barbaz" -body "1111\n"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
if (re.match(beresp.http.foo, "(?:bar)(baz)")) {
set beresp.http.foo0 = re.backref(0, "error0");
set beresp.http.foo1 = re.backref(1, "error1");
set beresp.http.foo2 = re.backref(2, "fallback");
} else {
error 999;
}
}
} -start
client c1 {
txreq
rxresp
expect resp.http.foo0 == "barbaz"
expect resp.http.foo1 == "baz"
expect resp.http.foo2 == "fallback"
} -run
varnishtest "limit to backrefs 0 to 10"
server s1 {
rxreq
txresp -hdr "Foo: 12345678901" -hdr "Bar: 123456789012" -body "1111\n"
} -start
varnish v1 -vcl+backend {
import re from "${vmod_topbuild}/src/.libs/libvmod_re.so";
sub vcl_fetch {
if (re.match(beresp.http.foo, "(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)")
) {
set beresp.http.foo0 = re.backref(0, "error0");
set beresp.http.foo1 = re.backref(1, "error1");
set beresp.http.foo2 = re.backref(2, "error2");
set beresp.http.foo3 = re.backref(3, "error3");
set beresp.http.foo4 = re.backref(4, "error4");
set beresp.http.foo5 = re.backref(5, "error5");
set beresp.http.foo6 = re.backref(6, "error6");
set beresp.http.foo7 = re.backref(7, "error7");
set beresp.http.foo8 = re.backref(8, "error8");
set beresp.http.foo9 = re.backref(9, "error9");
set beresp.http.foo10 = re.backref(10, "error10");
set beresp.http.foo11 = re.backref(11, "fallback");
} else {
error 999;
}
if (re.match(beresp.http.bar,
"(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)(.)")
) {
set beresp.http.bar0 = re.backref(0, "error0");
set beresp.http.bar1 = re.backref(1, "error1");
set beresp.http.bar2 = re.backref(2, "error2");
set beresp.http.bar3 = re.backref(3, "error3");
set beresp.http.bar4 = re.backref(4, "error4");
set beresp.http.bar5 = re.backref(5, "error5");
set beresp.http.bar6 = re.backref(6, "error6");
set beresp.http.bar7 = re.backref(7, "error7");
set beresp.http.bar8 = re.backref(8, "error8");
set beresp.http.bar9 = re.backref(9, "error9");
set beresp.http.bar10 = re.backref(10, "error10");
set beresp.http.bar11 = re.backref(11, "error11");
} else {
error 999;