Initial commit, passes simple encode tests.

.dir-locals.el

+((nil . ((indent-tabs-mode . t)))
+ (c-mode . ((c-file-style . "BSD"))))

.gitignore

+Makefile
+Makefile.in
+.deps/
+.libs/
+*.o
+*.lo
+*.la
+*~
+*.[1-9]
+.dirstamp
+
+/aclocal.m4
+/ar-lib
+/autom4te.cache/
+/compile
+/config.guess
+/config.h
+/config.h.in
+/config.log
+/config.status
+/config.sub
+/configure
+/depcomp
+/install-sh
+/libtool
+/ltmain.sh
+/missing
+/stamp-h1
+/test-driver
+/m4/
+
+/src/vcc_if.c
+/src/vcc_if.h
+/src/vmod_*rst
+
+/src/tests/*.log
+/src/tests/*.trs
+/src/test-suite.log

CONTRIBUTING.rst

+CONTRIBUTING
+============
+
+To contribute code or documentation, submit a pull request at the
+`source repository website
+<https://code.uplex.de/uplex-varnish/libvfp-brotli>`_.
+
+If you have a problem or discover a bug, you can post an `issue
+<https://code.uplex.de/uplex-varnish/libvfp-brotli/issues>`_ at
+the website. You can also write to <varnish-support@uplex.de>.
+
+For developers
+--------------
+
+The VFP source code is in C, and compilation has been tested with gcc
+and clang. The code MUST always compile successfully with both of
+them.
+
+The build specifies C99 conformance for C sources (``-std=c99``). All
+compiler warnings are turned on, and all warnings are considered
+errors (``-Werror -Wall -Wextra``).  The code MUST always build
+without warnings or errors under these constraints.
+
+By default, ``CFLAGS`` is set to ``-g -O2``, so that symbols are
+included in the shared library, and optimization is at level
+``O2``. To change or disable these options, set ``CFLAGS`` explicitly
+before calling ``configure`` (it may be set to the empty string).
+
+For development/debugging cycles, the ``configure`` option
+``--enable-debugging`` is recommended (off by default). This will turn
+off optimizations and function inlining, so that a debugger will step
+through the code as expected.
+
+By default, the VMOD is built with the stack protector enabled
+(compile option ``-fstack-protector``), but it can be disabled with
+the ``configure`` option ``--disable-stack-protector``.

COPYING

+See LICENSE for details.
+

INSTALL.rst

+INSTALLATION
+============
+
+Building from source
+~~~~~~~~~~~~~~~~~~~~
+
+The VFP is built against a Varnish installation, and the autotools
+use ``pkg-config(1)`` to locate the necessary header files and other
+resources. This sequence will install the VMOD::
+
+  > ./autogen.sh	# for builds from the git repo
+  > ./configure
+  > make
+  > make check		# to run unit tests in src/tests/*.vtc
+  > make distcheck	# run check and prepare a distribution tarball
+  > sudo make install
+
+See `CONTRIBUTING.rst <CONTRIBUTING.rst>`_ for notes about building
+from source.
+
+If you have installed Varnish in non-standard directories, call
+``autogen.sh`` and ``configure`` with the ``PKG_CONFIG_PATH``
+environment variable set to include the paths where the ``.pc`` file
+can be located for ``varnishapi``. For example, when varnishd
+configure was called with ``--prefix=$PREFIX``, use::
+
+  > PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
+  > export PKG_CONFIG_PATH
+
+By default, the ``configure`` script installs the VFP in the same
+directory as Varnish, determined via ``pkg-config(1)``. The vmod
+installation directory can be overridden by passing the ``VMOD_DIR``
+variable to ``configure``.
+
+Other files such as the man-page are installed in the locations
+determined by ``configure``, which inherits its default ``--prefix``
+setting from Varnish.

LICENSE

+Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGE.

Makefile.am

+ACLOCAL_AMFLAGS = -I m4 -I ${VARNISHAPI_DATAROOTDIR}/aclocal
+
+SUBDIRS = src
+
+DISTCHECK_CONFIGURE_FLAGS = \
+	VMOD_DIR='$${libdir}/varnish/vmods'
+
+EXTRA_DIST = README.rst LICENSE COPYING CONTRIBUTING.rst INSTALL.rst
+
+doc_DATA = README.rst LICENSE COPYING CONTRIBUTING.rst INSTALL.rst
+
+README.rst: src/vfp_brotli.vcc
+	$(MAKE) $(AM_MAKEFLAGS) -C src vmod_brotli.man.rst
+	cp src/vmod_brotli.man.rst README.rst
+
+%.1 %.2 %.3 %.4 %.5 %.6 %.7 %.8 %.9:
+if HAVE_RST2MAN
+	${RST2MAN} $< $@
+else
+	@echo "========================================"
+	@echo "You need rst2man installed to make dist"
+	@echo "========================================"
+	@false
+endif

README.rst

+..
+.. NB:  This file is machine generated, DO NOT EDIT!
+..
+.. Edit vmod.vcc and run make instead
+..
+
+.. role:: ref(emphasis)
+
+===========
+vmod_brotli
+===========
+
+--------------------------------------------------
+Varnish Fetch Processor for brotli de-/compression
+--------------------------------------------------
+
+:Manual section: 3
+
+
+SYNOPSIS
+========
+
+::
+
+  import brotli;
+
+  # Encoder and decoder objects
+  new <obj> = brotli.encoder(...)
+  # VOID <obj>.create_stats()
+  new <obj> = brotli.decoder(...)
+  # VOID <obj>.create_stats()
+
+  # XXX ...
+
+  # VFP version
+  STRING brotli.version()
+
+DESCRIPTION
+===========
+
+Varnish Fetch Processor (VFP) to support the brotli compresion
+algorithm for responses fetched from backends.
+
+XXX ...
+
+# $Object encoder(BOOL case_sensitive=1)
+
+XXX ...
+
+.. _vmod_brotli.encoder_version:
+
+STRING encoder_version()
+------------------------
+
+Return the version string for the brotli encoder.
+
+Example::
+
+  std.log("Using brotli encoder version: " + brotli.encoder_version());
+
+#$Function STRING decoder_version()
+
+Return the version string for the brotli decoder.
+
+Example::
+
+  std.log("Using brotli decoder version: " + brotli.decoder_version());
+
+.. _vmod_brotli.version:
+
+STRING version()
+----------------
+
+Return the version string for this VFP.
+
+Example::
+
+  std.log("Using VFP brotli version: " + brotli.version());
+
+STATISTICS
+==========
+
+XXX ...
+
+ERRORS
+======
+
+XXX ...
+
+REQUIREMENTS
+============
+
+The VFP requires Varnish version XXX ...
+
+INSTALLATION
+============
+
+See `INSTALL.rst <INSTALL.rst>`_ in the source repository.
+
+LIMITATIONS
+===========
+
+XXX ...
+
+AUTHOR
+======
+
+* Geoffrey Simmons <geoff@uplex.de>
+
+UPLEX Nils Goroll Systemoptimierung
+
+SEE ALSO
+========
+
+* varnishd(1)
+* vcl(7)
+* varnishstat(1)
+* VMOD source repository: https://code.uplex.de/uplex-varnish/libvfp-brotli
+* brotli repository: https://github.com/google/brotli
+* RFC 7932: https://tools.ietf.org/html/rfc7932
+
+COPYRIGHT
+=========
+
+::
+
+  Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
+  All rights reserved
+ 
+  Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
+ 
+  See LICENSE
+ 

autogen.sh

+#!/bin/sh
+
+warn() {
+	echo "WARNING: $@" 1>&2
+}
+
+case `uname -s` in
+Darwin)
+	LIBTOOLIZE=glibtoolize
+	;;
+FreeBSD)
+	LIBTOOLIZE=libtoolize
+	;;
+Linux)
+	LIBTOOLIZE=libtoolize
+	;;
+SunOS)
+	LIBTOOLIZE=libtoolize
+	;;
+*)
+	warn "unrecognized platform:" `uname -s`
+	LIBTOOLIZE=libtoolize
+esac
+
+automake_version=`automake --version | tr ' ' '\n' | egrep '^[0-9]\.[0-9a-z.-]+'`
+if [ -z "$automake_version" ] ; then
+	warn "unable to determine automake version"
+else
+	case $automake_version in
+		0.*|1.[0-8]|1.[0-8][.-]*)
+			warn "automake ($automake_version) detected; 1.9 or newer recommended"
+			;;
+		*)
+			;;
+	esac
+fi
+
+# check for varnishapi.m4 in custom paths
+dataroot=$(pkg-config --variable=datarootdir varnishapi 2>/dev/null)
+if [ -z "$dataroot" ] ; then
+	cat >&2 <<'EOF'
+Package varnishapi was not found in the pkg-config search path.
+Perhaps you should add the directory containing `varnishapi.pc'
+to the PKG_CONFIG_PATH environment variable
+EOF
+	exit 1
+fi
+set -ex
+aclocal -I m4 -I ${dataroot}/aclocal
+$LIBTOOLIZE --copy --force
+autoheader
+automake --add-missing --copy --foreign
+autoconf

configure.ac

+AC_PREREQ(2.68)
+AC_COPYRIGHT([Copyright (c) 2019 UPLEX - Nils Goroll Systemoptimierung])
+AC_INIT([libvfp-brotli], [trunk], [varnish-support@uplex.de], [vfp-brotli])
+AC_CONFIG_MACRO_DIR([m4])
+AC_CONFIG_SRCDIR(src/vfp_brotli.vcc)
+AM_CONFIG_HEADER(config.h)
+AC_CANONICAL_SYSTEM
+AC_LANG(C)
+
+AM_INIT_AUTOMAKE([1.12 -Wall -Werror foreign parallel-tests])
+AM_SILENT_RULES([yes])
+AM_PROG_AR
+
+LT_PREREQ([2.2.6])
+LT_INIT([dlopen disable-static])
+
+AC_PROG_CC
+AC_PROG_CC_STDC
+if test "x$ac_cv_prog_cc_c99" = xno; then
+	AC_MSG_ERROR([Could not find a C99 compatible compiler])
+fi
+AC_PROG_CPP
+
+AX_PTHREAD(,[AC_MSG_ERROR([Could not configure pthreads support])])
+
+LIBS="$PTHREAD_LIBS $LIBS"
+CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+CC="$PTHREAD_CC"
+
+AC_ARG_WITH([rst2man],
+	AS_HELP_STRING(
+		[--with-rst2man=PATH],
+		[Location of rst2man (auto)]),
+	[RST2MAN="$withval"],
+	AC_CHECK_PROGS(RST2MAN, [rst2man rst2man.py], []))
+AM_CONDITIONAL(HAVE_RST2MAN, [test "x$RST2MAN" != "xno"])
+
+m4_ifndef([VARNISH_PREREQ], AC_MSG_ERROR([Need varnish.m4 -- see README.rst]))
+
+PKG_CHECK_MODULES([BROTLIENC], [libbrotlienc])
+
+VARNISH_PREREQ([trunk])
+VARNISH_VMODS([brotli])
+#VARNISH_COUNTERS([brotli])
+
+VMOD_TESTS="$(cd $srcdir/src && echo tests/*.vtc)"
+AC_SUBST(VMOD_TESTS)
+
+PKG_CHECK_VAR([LIBVARNISHAPI_LIBDIR], [varnishapi], [libdir])
+AC_SUBST([VARNISH_LIBRARY_PATH],
+	[$LIBVARNISHAPI_LIBDIR:$LIBVARNISHAPI_LIBDIR/varnish])
+
+# Checks for C sources
+AC_CHECK_FUNCS([strdup])
+# XXX ...
+
+# --enable-stack-protector
+AC_ARG_ENABLE(stack-protector,
+       AS_HELP_STRING([--enable-stack-protector],[enable stack protector (default is YES)]),
+       [],
+       [enable_stack_protector=yes])
+
+if test "x$enable_stack_protector" != "xno"; then
+	AX_CHECK_COMPILE_FLAG([-fstack-protector],
+		AX_CHECK_LINK_FLAG([-fstack-protector],
+			[CFLAGS="${CFLAGS} -fstack-protector"], [], []),
+		[], [])
+fi
+
+# --enable-debugging
+AC_ARG_ENABLE(debugging,
+	AS_HELP_STRING([--enable-debugging],[enable debugging (default is NO)]),
+	[],
+	[enable_debugging=no])
+
+# AC_PROG_CC sets CFLAGS to '-g -O2' unless already set, so there's no
+# need to add -g. Disable or change by explicitly setting CFLAGS. If
+# this option is enabled, then -Og or -O0 becomes the last
+# optimization option, and hence takes precedence.
+if test "x$enable_debugging" != "xno"; then
+	CFLAGS="${CFLAGS} -fno-inline"
+	AX_CHECK_COMPILE_FLAG([-Og],
+		[CFLAGS="${CFLAGS} -O0 -Og"],
+		[CFLAGS="${CFLAGS} -O0"],
+		[])
+fi
+
+AC_CONFIG_FILES([
+	Makefile
+	src/Makefile
+])
+AC_OUTPUT

src/Makefile.am

+AUTOMAKE_OPTIONS = subdir-objects
+
+AM_CFLAGS = $(VARNISHAPI_CFLAGS) -Wall -Werror -Wextra -std=c99 \
+	@BROTLIENC_CFLAGS@
+AM_LDFLAGS = $(VARNISHAPI_LIBS) $(VMOD_LDFLAGS) -ldl
+
+vmod_LTLIBRARIES = libvmod_brotli.la
+
+libvmod_brotli_la_SOURCES = \
+	vfp_brotli.c
+
+nodist_libvmod_brotli_la_SOURCES = \
+	vcc_if.c \
+	vcc_if.h
+
+dist_man_MANS = vfp_brotli.3
+
+vfp_brotli.lo: $(nodist_libvmod_brotli_la_SOURCES)
+
+libvmod_brotli_la_LIBADD = @BROTLIENC_LIBS@
+
+vcc_if.h vmod_brotli.rst vmod_brotli.man.rst: vcc_if.c
+
+vcc_if.c: vfp_brotli.vcc
+	$(AM_V_VMODTOOL) $(PYTHON) $(VMODTOOL) -o vcc_if $(srcdir)/vfp_brotli.vcc
+
+vfp_brotli.3: vmod_brotli.man.rst
+	$(AM_V_GEN) $(RST2MAN) vmod_brotli.man.rst vfp_brotli.3
+
+AM_TESTS_ENVIRONMENT = \
+	PATH="$(VMOD_TEST_PATH)" \
+	LD_LIBRARY_PATH="$(VARNISH_LIBRARY_PATH)"
+TEST_EXTENSIONS = .vtc
+VTC_LOG_COMPILER = varnishtest -v
+AM_VTC_LOG_FLAGS = -Dvmod_brotli="$(VMOD_BROTLI)"
+
+# To test an individual VTC test named test.vtc:
+# $ cd src/
+# $ make check TESTS=tests/test.vtc
+
+TESTS = @VMOD_TESTS@
+
+EXTRA_DIST = \
+	vfp_brotli.vcc \
+	$(VMOD_TESTS)
+
+CLEANFILES = \
+	$(builddir)/vcc_if.c \
+	$(builddir)/vcc_if.h \
+	$(builddir)/vmod_brotli.rst \
+	$(builddir)/vmod_brotli.man.rst \
+	$(builddir)/vfp_brotli.3

src/tests/00basic.vtc

+# looks like -*- vcl -*-
+
+varnishtest "vcl.use and .discard, and version strings"
+
+varnish v1 -vcl {
+	import ${vmod_brotli};
+	backend b { .host = "${bad_ip}"; }
+} -start
+
+varnish v1 -vcl {backend b { .host = "${bad_ip}"; }}
+
+varnish v1 -cli "vcl.list"
+varnish v1 -cli "vcl.use vcl1"
+
+varnish v1 -cli "vcl.use vcl2"
+varnish v1 -cli "vcl.use vcl1"
+
+varnish v1 -cli "vcl.show vcl1"
+varnish v1 -cli "vcl.use vcl2"
+varnish v1 -cli "vcl.discard vcl1"
+varnish v1 -cli "vcl.list"
+
+varnish v1 -vcl {
+	import ${vmod_brotli};
+	backend b { .host = "${bad_ip}"; }
+
+	sub vcl_recv {
+		return(synth(200));
+	}
+
+	sub vcl_synth {
+		set resp.http.x-version = brotli.version();
+		if (!resp.http.x-version) {
+			set resp.status = 500;
+		}
+		set resp.http.x-encoder-version = brotli.encoder_version();
+		if (!resp.http.x-encoder-version) {
+			set resp.status = 500;
+		}
+		return(deliver);
+	}
+}
+
+client c1 {
+	txreq -url "/"
+	rxresp
+	expect resp.status == 200
+	expect resp.http.x-version ~ "^.+$"
+	expect resp.http.x-encoder-version ~ {^\d+\.\d+\.\d+$}
+} -run

src/tests/encode.vtc

+# looks like -*- vcl -*-
+
+varnishtest "brotli compression"
+
+server s1 {
+	rxreq
+	txresp -body {The quick brown fox jumps over the lazy dog}
+	rxreq
+	txresp -body {Xyzzy}
+} -start
+
+varnish v1 -vcl+backend {
+	import ${vmod_brotli};
+
+	sub vcl_backend_response {
+		set beresp.filters = "br";
+		set beresp.uncacheable = true;
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.http.Content-Encoding == "br"
+	expect resp.http.Vary == "Accept-Encoding"
+	expect resp.bodylen == 47
+	expect resp.body == "The quick brown fox jumps over the lazy dog"
+
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.http.Content-Encoding == "br"
+	expect resp.http.Vary == "Accept-Encoding"
+	expect resp.bodylen == 9
+	expect resp.body == {Xyzzy}
+} -run

src/vfp_brotli.c

+/*-
+ * Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
+ * All rights reserved
+ *
+ * Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+
+/* for strdup() */
+//#define _POSIX_C_SOURCE 200809L
+
+#include "config.h"
+
+#include <stdlib.h>
+#include <string.h>
+#include <brotli/encode.h>
+
+#include "cache/cache.h"
+#include "cache/cache_filter.h"
+
+#include "vcc_if.h"
+
+/* XXX make this configurable; cf. varnishd default gzip_buffer */
+#define BUFFER_SZ (32 * 1024 * 1024)
+
+struct vbr_enc {
+	unsigned		magic;
+#define VBR_ENC_MAGIC		0x467c387b
+	BrotliEncoderState	*state;
+	uint8_t			*buf;
+	const uint8_t		*next_in;
+	uint8_t			*next_out;
+	size_t			bufsz;
+	size_t			buflen;
+	size_t			avail_in;
+	size_t			avail_out;
+	size_t			total_out;
+};
+
+static struct vbr_enc *
+newEnc(void)
+{
+	struct vbr_enc *vbr_enc;
+
+	/* XXX inc encs counter */
+	ALLOC_OBJ(vbr_enc, VBR_ENC_MAGIC);
+	if (vbr_enc == NULL)
+		return (NULL);
+	vbr_enc->state = BrotliEncoderCreateInstance(NULL, NULL, NULL);
+	if (vbr_enc->state == NULL)
+		return (NULL);
+	/* XXX configure the handle */
+	return (vbr_enc);
+}
+
+static void
+destroy(struct vbr_enc **vp)
+{
+	struct vbr_enc *vbr_enc;
+
+	TAKE_OBJ_NOTNULL(vbr_enc, vp, VBR_ENC_MAGIC);
+	/* XXX log results: bytes in, bytes out */
+	/* XXX IMO no finish op necessary here */
+	BrotliEncoderDestroyInstance(vbr_enc->state);
+	if (vbr_enc->buf != NULL)
+		free(vbr_enc->buf);
+	FREE_OBJ(vbr_enc);
+}
+
+static int
+getBuf(struct vbr_enc *vbr_enc)
+{
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+	AZ(vbr_enc->bufsz);
+	AZ(vbr_enc->buflen);
+	AZ(vbr_enc->buf);
+
+	/* XXX make this configurable */
+	vbr_enc->bufsz = BUFFER_SZ;
+	vbr_enc->buf = malloc(vbr_enc->bufsz);
+	if (vbr_enc->buf == NULL) {
+		vbr_enc->bufsz = 0;
+		return (-1);
+	}
+	return (0);
+}
+
+static void
+setInputBuf(struct vbr_enc *vbr_enc, const void *ptr, ssize_t len)
+{
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+
+	AZ(vbr_enc->avail_in);
+	vbr_enc->next_in = TRUST_ME(ptr);
+	vbr_enc->avail_in = len;
+}
+
+static int
+isInputBufEmpty(const struct vbr_enc *vbr_enc)
+{
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+	return (vbr_enc->avail_in == 0);
+}
+
+static void
+setOutputBuf(struct vbr_enc *vbr_enc, const void *ptr, ssize_t len)
+{
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+
+	vbr_enc->next_out = TRUST_ME(ptr);
+	vbr_enc->avail_out = len;
+}
+
+#if 0
+static int
+isOutputBufFull(const struct vbr_enc *vbr_enc)
+{
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+	return (vbr_enc->avail_out == 0);
+}
+#endif
+
+static BROTLI_BOOL
+encode(struct vbr_enc *vbr_enc, ssize_t *dl, int finished)
+{
+	BROTLI_BOOL ret;
+	const uint8_t *before;
+	enum BrotliEncoderOperation op = BROTLI_OPERATION_PROCESS;
+
+	CHECK_OBJ_NOTNULL(vbr_enc, VBR_ENC_MAGIC);
+	before = vbr_enc->next_out;
+
+	if (finished)
+		op = BROTLI_OPERATION_FINISH;
+	ret = BrotliEncoderCompressStream(vbr_enc->state, op,
+					  &vbr_enc->avail_in,
+					  &vbr_enc->next_in,
+					  &vbr_enc->avail_out,
+					  &vbr_enc->next_out,
+					  &vbr_enc->total_out);
+	*dl = (const uint8_t *)vbr_enc->next_out - before;
+	return (ret);
+}
+
+/* VFP interface */
+
+static enum vfp_status v_matchproto_(vfp_init_f)
+vfp_brenc_init(struct vfp_ctx *ctx, struct vfp_entry *ent)
+{
+	struct vbr_enc *vbr_enc;
+
+	CHECK_OBJ_NOTNULL(ctx, VFP_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ent, VFP_ENTRY_MAGIC);
+
+	/* XXX: cache_gzip.c has:
+	 if (http_GetStatus(vc->resp) == 206)
+	 	return (VFP_NULL);
+	 What do we want to do about partial responses?
+	*/
+
+	if (http_GetHdr(ctx->resp, H_Content_Encoding, NULL))
+		return (VFP_NULL);
+	if ((vbr_enc = newEnc()) == NULL)
+		return (VFP_ERROR);
+	ent->priv1 = vbr_enc;
+	if (getBuf(vbr_enc))
+		return (VFP_ERROR);
+	setInputBuf(vbr_enc, vbr_enc->buf, 0);
+	AZ(vbr_enc->buflen);
+
+	http_Unset(ctx->resp, H_Content_Length);
+	RFC2616_Weaken_Etag(ctx->resp);
+	http_SetHeader(ctx->resp, "Content-Encoding: br");
+	RFC2616_Vary_AE(ctx->resp);
+	return (VFP_OK);
+}
+
+static enum vfp_status v_matchproto_(vfp_pull_f)
+vfp_brenc_pull(struct vfp_ctx *ctx, struct vfp_entry *ent, void *ptr,
+	       ssize_t *lenp)
+{
+	struct vbr_enc *vbr_enc;
+	ssize_t len, dl;
+	enum vfp_status vp = VFP_ERROR;
+	int finished = 0;
+	BROTLI_BOOL done;
+
+	CHECK_OBJ_NOTNULL(ctx, VFP_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ent, VFP_ENTRY_MAGIC);
+	CAST_OBJ_NOTNULL(vbr_enc, ent->priv1, VBR_ENC_MAGIC);
+	AN(ptr);
+	AN(lenp);
+
+	len = *lenp;
+	*lenp = 0;
+	setOutputBuf(vbr_enc, ptr, len);
+
+	do {
+		if (isInputBufEmpty(vbr_enc)) {
+			len = vbr_enc->bufsz;
+			vp = VFP_Suck(ctx, vbr_enc->buf, &len);
+			if (vp == VFP_ERROR)
+				break;
+			if (vp == VFP_END)
+				finished = 1;
+			setInputBuf(vbr_enc, vbr_enc->buf, len);
+		}
+		if (!isInputBufEmpty(vbr_enc) || finished) {
+			done = encode(vbr_enc, &dl, finished);
+			if (done != BROTLI_TRUE)
+				return (VFP_Error(ctx, "brotli encode failed"));
+			if (dl > 0) {
+				*lenp = dl;
+				return (VFP_OK);
+			}
+		}
+		AN(isInputBufEmpty(vbr_enc));
+	} while (!finished);
+
+	if (done != BROTLI_TRUE)
+		return (VFP_Error(ctx, "brotli encode failed"));
+	return (VFP_END);
+}
+
+static void v_matchproto_(vfp_fini_f)
+vfp_brenc_fini(struct vfp_ctx *ctx, struct vfp_entry *ent)
+{
+	struct vbr_enc *vbr_enc;
+
+	CHECK_OBJ_NOTNULL(ctx, VFP_CTX_MAGIC);
+	CHECK_OBJ_NOTNULL(ent, VFP_ENTRY_MAGIC);
+
+	if (ent->priv1 != NULL) {
+		CAST_OBJ(vbr_enc, ent->priv1, VBR_ENC_MAGIC);
+		ent->priv1 = NULL;
+		destroy(&vbr_enc);
+	}
+}
+
+static const struct vfp vfp_brenc = {
+	.name = "br",
+	.init = vfp_brenc_init,
+	.pull = vfp_brenc_pull,
+	.fini = vfp_brenc_fini,
+};
+
+/* Event function */
+
+int
+vmod_event(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e)
+{
+	ASSERT_CLI();
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(priv);
+
+	switch(e) {
+	case VCL_EVENT_LOAD:
+		VRT_AddVFP(ctx, &vfp_brenc);
+		return (0);
+	case VCL_EVENT_DISCARD:
+		VRT_RemoveVFP(ctx, &vfp_brenc);
+		return (0);
+	case VCL_EVENT_WARM:
+	case VCL_EVENT_COLD:
+		return (0);
+	default:
+		WRONG("illegal event enum");
+	}
+	NEEDLESS(return (0));
+}
+
+static VCL_STRING
+brotli_version(VRT_CTX, uint32_t version)
+{
+	VCL_STRING vers;
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	unsigned major = version >> 24;
+	unsigned minor = (version >> 12) & 0xFFF;
+	unsigned patch = version & 0xFFF;
+	if ((vers = WS_Printf(ctx->ws, "%u.%u.%u", major, minor, patch))
+	    == NULL) {
+		VSLb(ctx->vsl, SLT_VCL_Error, "vfp brotli error: "
+		     "insufficient workspace for version string");
+		return ("(insufficient workspace for version string)");
+	}
+	return vers;
+}
+
+VCL_STRING
+vmod_encoder_version(VRT_CTX)
+{
+	return (brotli_version(ctx, BrotliEncoderVersion()));
+}
+
+#if 0
+VCL_STRING
+vmod_decoder_version(VRT_CTX)
+{
+	return (brotli_version(ctx, BrotliDecoderVersion()));
+}
+#endif
+
+VCL_STRING
+vmod_version(VRT_CTX)
+{
+	(void) ctx;
+	return VERSION;
+}

src/vfp_brotli.vcc

+#-
+# Copyright (c) 2019 UPLEX Nils Goroll Systemoptimierung
+# All rights reserved
+#
+# Author: Geoffrey Simmons <geoffrey.simmons@uplex.de>
+#
+# See LICENSE
+#
+
+$Module brotli 3 "Varnish Fetch Processor for brotli de-/compression"
+
+$ABI vrt
+
+$Synopsis manual
+
+SYNOPSIS
+========
+
+::
+
+  import brotli;
+
+  # Encoder and decoder objects
+  new <obj> = brotli.encoder(...)
+  # VOID <obj>.create_stats()
+  new <obj> = brotli.decoder(...)
+  # VOID <obj>.create_stats()
+
+  # XXX ...
+
+  # VFP version
+  STRING brotli.version()
+
+DESCRIPTION
+===========
+
+Varnish Fetch Processor (VFP) to support the brotli compresion
+algorithm for responses fetched from backends.
+
+XXX ...
+
+# $Object encoder(BOOL case_sensitive=1)
+
+XXX ...
+
+$Function STRING encoder_version()
+
+Return the version string for the brotli encoder.
+
+Example::
+
+  std.log("Using brotli encoder version: " + brotli.encoder_version());
+
+#$Function STRING decoder_version()
+
+Return the version string for the brotli decoder.
+
+Example::
+
+  std.log("Using brotli decoder version: " + brotli.decoder_version());
+
+$Function STRING version()
+
+Return the version string for this VFP.
+
+Example::
+
+  std.log("Using VFP brotli version: " + brotli.version());
+
+STATISTICS
+==========
+
+XXX ...
+
+ERRORS
+======
+
+XXX ...
+
+REQUIREMENTS
+============
+
+The VFP requires Varnish version XXX ...
+
+INSTALLATION
+============
+
+See `INSTALL.rst <INSTALL.rst>`_ in the source repository.
+
+LIMITATIONS
+===========
+
+XXX ...
+
+AUTHOR
+======
+
+* Geoffrey Simmons <geoff@uplex.de>
+
+UPLEX Nils Goroll Systemoptimierung
+
+SEE ALSO
+========
+
+* varnishd(1)
+* vcl(7)
+* varnishstat(1)
+* VMOD source repository: https://code.uplex.de/uplex-varnish/libvfp-brotli
+* brotli repository: https://github.com/google/brotli
+* RFC 7932: https://tools.ietf.org/html/rfc7932
+
+$Event event