Initial public release

parents
# build system
.deps/
.libs/
autom4te.cache/
build-aux/
m4/
*.la
*.lo
*.o
*.tar.gz
Makefile
Makefile.in
aclocal.m4
config.h
config.h.in
config.log
config.status
configure
libtool
stamp-h1
# test suite
*.log
*.trs
# vmodtool
vcc_*_if.[ch]
vmod_*.rst
# man
*.1
*_options.rst
*_synopsis.rst
vmod_*.3
INSTALLATION
============
Building from source
~~~~~~~~~~~~~~~~~~~~
The VMOD is built on a system where an instance of Varnish is
installed, and the auto-tools will attempt to locate the Varnish
instance, and then pull in libraries and other support files from
there.
Quick start
-----------
This sequence should be enough in typical setups:
1. ``./bootstrap`` (for git-installation)
3. ``make``
4. ``make check`` (regression tests)
5. ``make install`` (may require root: sudo make install)
Alternative configs
-------------------
If you have installed Varnish to a non-standard directory, call
``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
the appropriate path. For example, when varnishd configure was called
with ``--prefix=$PREFIX``, use::
PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
ACLOCAL_PATH=${PREFIX}/share/aclocal
export PKG_CONFIG_PATH ACLOCAL_PATH
Copyright 2023 UPLEX Nils Goroll Systemoptimierung
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
ACLOCAL_AMFLAGS = -I m4 -I @VARNISHAPI_DATAROOTDIR@/aclocal
DISTCHECK_CONFIGURE_FLAGS = RST2MAN=:
SUBDIRS = src
=========================================
A JSON formatter for VCL which sucks less
=========================================
.. role:: ref(emphasis)
.. _Varnish-Cache: https://varnish-cache.org/
This project provides JSON formatting facilities for `Varnish-Cache`_
VCL as a module (VMOD).
PROJECT RESOURCES
=================
* The primary repository is at https://code.uplex.de/uplex-varnish/libvmod-j
This server does not accept user registrations, so please use ...
* the mirror at https://gitlab.com/uplex/varnish/libvmod-j for issues,
merge requests and all other interactions.
INTRODUCTION
============
**BEFORE USING THIS VMOD, DO READ THIS DOCUMENTATION AND IN PARTICULAR
THE WARNING IN** `vmod_j(3) <src/vmod_j.man.rst>`_.
.. _JSON: https://www.json.org/json-en.html
Formatting `JSON`_ in pure VCL is a PITA, because string processing in
VCL was never made for it. VCL being a Domain Specific Language, it
was made for processing HTTP headers.
Consider this simple example of a JSON object with a single key
``key``, for which the value ``value`` is to be taken from a VCL
header variable::
{"key":"value"}
in pure VCL, you have to write something like this::
{"{"key":""} + req.http.value + {"""} + "}"
Applause if you do not lose your mental sanity trying to understand
what this does.
This vmod has been written because it drove the author crazy to
maintain VCL code with constructs like the above (and that's an
exceptionally trivial case).
DESCRIPTION (Excerpt)
=====================
(The full version is in `vmod_j(3) <src/vmod_j.man.rst>`_)
With VMOD *j*, the example above looks like this::
j.object("key" + req.http.value)
Things get more interesting with more complex data
structures. Consider this toy example::
j.object(
j.str("A") + j.null() +
"B" + j.object(
"BB" + j.number(string="42.42e42") +
"CC" + false +
"DD" + true) +
"C" + j.array("A" + 2 + j.object(j.nil()))
)
The JSON produced by this code looks like this when reformatted with
:ref:`jq(1)`::
{
"A": null,
"B": {
"BB": 4.242e+43,
"CC": false,
"DD": true
},
"C": [
"A",
2,
{}
]
}
Continue reading the documentation in `vmod_j(3) <src/vmod_j.man.rst>`_.
INSTALLATION
============
See `INSTALL.rst <INSTALL.rst>`_ in the source repository.
SUPPORT
=======
.. _gitlab.com issues: https://gitlab.com/uplex/varnish/libvmod-j/-/issues
To report bugs, use `gitlab.com issues`_.
For enquiries about professional service and support, please contact
info@uplex.de\ .
CONTRIBUTING
============
.. _merge requests on gitlab.com: https://gitlab.com/uplex/varnish/libvmod-j/-/merge_requests
To contribute to the project, please use `merge requests on gitlab.com`_.
To support the project's development and maintenance, there are
several options:
.. _paypal: https://www.paypal.com/donate/?hosted_button_id=BTA6YE2H5VSXA
.. _github sponsor: https://github.com/sponsors/nigoroll
* Donate money through `paypal`_. If you wish to receive a commercial
invoice, please add your details (address, email, any requirements
on the invoice text) to the message sent with your donation.
* Become a `github sponsor`_.
* Contact info@uplex.de to receive a commercial invoice for SWIFT payment.
COPYRIGHT
=========
::
Copyright 2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
#!/bin/sh
set -e
set -u
WORK_DIR=$(pwd)
ROOT_DIR=$(dirname "$0")
cd "$ROOT_DIR"
if ! command -v libtoolize >/dev/null
then
echo "libtoolize: command not found, falling back to glibtoolize" >&2
alias libtoolize=glibtoolize
fi
mkdir -p m4
aclocal
libtoolize --copy --force
autoheader
automake --add-missing --copy --foreign
autoconf
cd "$WORK_DIR"
"$ROOT_DIR"/configure "$@"
AC_PREREQ([2.68])
AC_INIT([libvmod-vj], [0.1])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
AM_INIT_AUTOMAKE([1.12 -Wall -Werror foreign parallel-tests])
AM_SILENT_RULES([yes])
AC_ARG_WITH([rst2man],
AS_HELP_STRING(
[--with-rst2man=PATH],
[Location of rst2man (auto)]),
[RST2MAN="$withval"],
[AC_CHECK_PROGS(RST2MAN, [rst2man rst2man.py], [])])
VARNISH_PREREQ([6.0.0])
VARNISH_VMODS([j])
AM_PROG_AR
LT_PREREQ([2.2.6])
LT_INIT([dlopen disable-static])
AC_CONFIG_FILES([
Makefile
src/Makefile
])
AC_OUTPUT
AS_ECHO("
==== $PACKAGE_STRING ====
varnish: $VARNISH_VERSION
prefix: $prefix
vmoddir: $vmoddir
vcldir: $vcldir
pkgvcldir: $pkgvcldir
compiler: $CC
cflags: $CFLAGS
ldflags: $LDFLAGS
")
AM_CFLAGS = $(VARNISHAPI_CFLAGS)
# Modules
vmod_LTLIBRARIES = \
libvmod_j.la
libvmod_j_la_LDFLAGS = $(VMOD_LDFLAGS)
libvmod_j_la_SOURCES = vmod_j.c
nodist_libvmod_j_la_SOURCES = \
vcc_j_if.c \
vcc_j_if.h
@BUILD_VMOD_J@
# Test suite
AM_TESTS_ENVIRONMENT = \
PATH="$(abs_builddir):$(VARNISH_TEST_PATH):$(PATH)" \
LD_LIBRARY_PATH="$(VARNISH_LIBRARY_PATH)"
TEST_EXTENSIONS = .vtc
VTC_LOG_COMPILER = varnishtest -vl
AM_VTC_LOG_FLAGS = \
-p vcl_path="$(abs_top_srcdir)/vcl:$(VARNISHAPI_VCLDIR)" \
-p vmod_path="$(abs_builddir)/.libs:$(vmoddir):$(VARNISHAPI_VMODDIR)"
TESTS = \
vtc/vmod_j.vtc
# Documentation
dist_doc_DATA = \
vmod_j.vcc \
$(TESTS)
dist_man_MANS = \
vmod_j.3
.rst.1:
$(AM_V_GEN) $(RST2MAN) $< $@
.PHONY: flint
flint:
flexelint $(AM_CFLAGS) -I .. flint.lnt *.c
// must always be included to ensure sanity
-efile(766, config.h)
-e717 // do ... while(0)
// we can not change external interfaces / code
-efile(766, *_if.c)
// constructors not referenced
-esym(528, init_*)
-esym(528, assert_*)
/*-
* Copyright 2023 UPLEX Nils Goroll Systemoptimierung
* All rights reserved
*
* Author: Nils Goroll <nils.goroll@uplex.de>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include "config.h"
#include <stdio.h>
#include <string.h>
#include <cache/cache.h>
#include <vsb.h>
#include "vcc_j_if.h"
static int
is_jnumber(const char *s)
{
if (*s == '-')
s++;
// 0 or [1-9][0-9]*
if (*s == '0')
s++;
else if (! (*s >= '1' && *s <= '9'))
return (0);
else {
// *s == 1..9
s++;
while (*s >= '0' && *s <= '9')
s++;
}
// fraction
if (*s == '.') {
s++;
while (*s >= '0' && *s <= '9')
s++;
}
// exponent
if (*s == 'e' || *s == 'E') {
s++;
if (*s == '-' || *s == '+')
s++;
while (*s >= '0' && *s <= '9')
s++;
}
if (*s != '\0')
return (0);
return (1);
}
static int
is_jlit(const char *s)
{
return (strcmp(s, "true") == 0 ||
strcmp(s, "false") == 0 ||
strcmp(s, "null") == 0);
}
// magic values
#define JM_object 0x81
#define JM_min JM_object
#define JM_array 0x82
#define JM_string 0x83
#define JM_number 0x84
#define JM_lit 0x85
#define JM_max JM_lit
#define AL __attribute__((aligned(2)))
static const unsigned char jm_object[2] AL = { JM_object, '{' };
static const unsigned char jm_array[2] AL = { JM_array, '[' };
static const unsigned char jm_string[2] AL = { JM_string, '"' };
static const unsigned char jnil_object[4] AL = { JM_object, '{','}', 0 };
static const unsigned char jnil_array[4] AL = { JM_array, '[',']', 0 };
static const unsigned char jnull[6] AL = { JM_lit, 'n','u','l','l', 0};
static const unsigned char jtrue[6] AL = { JM_lit, 't','r','u','e', 0};
static const unsigned char jfalse[7] AL = { JM_lit, 'f','a','l','s','e', 0};
#define VSB_mcat(vsb, x) AZ(VSB_bcat(vsb, x, sizeof(x)))
#define isJ(p, X) (((uintptr_t)(p) & 1) == 1 && \
memcmp((unsigned char *)(p) - 1, jm_ ## X, sizeof(jm_ ## X)) == 0)
#define is_Jobject(p) isJ(p, object)
#define is_Jarray(p) isJ(p, array)
#define is_Jstring(p) isJ(p, string)
// is this specific enough?
#define is_J(p) ((uintptr_t)(p) & 1 && \
(unsigned char)(p)[-1] >= JM_min && \
(unsigned char)(p)[-1] <= JM_max)
static void __attribute__((constructor))
assert_statics(void)
{
//lint --e{506}
assert(is_Jobject(jm_object + 1));
assert(is_Jobject(jnil_object + 1));
assert(is_Jarray(jm_array + 1));
assert(is_Jarray(jnil_array + 1));
assert(is_Jstring(jm_string + 1));
assert(is_J(jnull + 1));
assert(is_J(jtrue + 1));
assert(is_J(jfalse + 1));
}
static inline int
is_Jnumber(const char *p)
{
if (((uintptr_t)p & 1) == 0)
return (0);
if ((unsigned char)p[-1] != JM_number)
return (0);
if (*p == '-')
p++;
return (*p >= '0' && *p <= '9');
}
VCL_STRANDS
vmod_nil(VRT_CTX)
{
(void) ctx;
return (vrt_null_strands);
}
VCL_STRING
vmod_null(VRT_CTX)
{
(void) ctx;
return ((const char *)jnull + 1);
}
VCL_STRING
vmod_true(VRT_CTX)
{
(void) ctx;
return ((const char *)jtrue + 1);
}
VCL_STRING
vmod_false(VRT_CTX)
{
(void) ctx;
return ((const char *)jfalse + 1);
}
#define ints_sz (size_t)4
#define ints_lim (int)100
static char ints[ints_lim][ints_sz] AL;
static void __attribute__((constructor))
init_ints(void)
{
size_t u;
int r;
for (u = 0; u < ints_lim; u++) {
r = snprintf(ints[u], ints_sz, "%c%zu", JM_number, u);
assert((unsigned)r < ints_sz);
assert(is_Jnumber(ints[u] + 1));
}
}
VCL_STRING
vmod_number(VRT_CTX, struct VARGS(number)*a)
{
struct vsb vsb[1];
const char *p;
int valid_i, valid_d;
intmax_t i = 0;
double d = nan("");
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
AN(a);
(void)a;
valid_i = a->valid_integer + a->valid_bytes;
valid_d = a->valid_real + a->valid_duration + a->valid_time;
if (valid_i + valid_d + a->valid_string != 1) {
VRT_fail(ctx, "j.number(): exactly one argument must be given");
return (NULL);
}
if (a->valid_string) {
if (! is_jnumber(a->string)) {
VRT_fail(ctx, "j.number(): string %s invalid" ,
a->string);
return (NULL);
}
}
else if (a->valid_integer)
i = a->integer;
else if (a->valid_bytes)
i = a->bytes;
else if (a->valid_real)
d = a->real;
else if (a->valid_duration)
d = a->duration;
else if (a->valid_time)
d = a->time;
else
WRONG("args invalid");
if (valid_i && ! VRT_INT_is_valid(i)) {
VRT_fail(ctx, "INT overflow converting to string (0x%jx)", i);
return (NULL);
}
else if (valid_d && ! VRT_REAL_is_valid(d)) {
VRT_fail(ctx, "REAL overflow converting to string (%e)", d);
return (NULL);
}
if (valid_i && i >= 0 && i < ints_lim)
return (ints[i] + 1);
// on %.15g: https://stackoverflow.com/questions/30658919/the-precision-of-printf-with-specifier-g/54162486#54162486
WS_VSB_new(vsb, ctx->ws);
AZ(VSB_putc(vsb, JM_number));
if (a->valid_string)
AZ(VSB_bcat(vsb, a->string, (ssize_t)strlen(a->string)));
else if (valid_i)
AZ(VSB_printf(vsb, "%jd", i));
else if (valid_d)
AZ(VSB_printf(vsb, "%.15g", d));
else
WRONG("valid_X");
p = WS_VSB_finish(vsb, ctx->ws, NULL);
if (p == NULL) {
VRT_fail(ctx, "j.number(): our of workspace");
return (NULL);
}
AZ((uintptr_t)p & 1);
return (p + 1);
}
// add json string
static void
vsbjstring(struct vsb *vsb, const char *p)
{
AZ(VSB_putc(vsb, '"'));
VSB_quote(vsb, p, -1, VSB_QUOTE_JSON);
AZ(VSB_putc(vsb, '"'));
}
VCL_STRING
vmod_string(VRT_CTX, VCL_STRANDS s)
{
struct vsb vsb[1];
const char *p;
int i;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
AN(s);
WS_VSB_new(vsb, ctx->ws);
VSB_mcat(vsb, jm_string); // "
for (i = 0; i < s->n; i++)
VSB_quote(vsb, s->p[i], -1, VSB_QUOTE_JSON);
AZ(VSB_putc(vsb, '"'));
p = WS_VSB_finish(vsb, ctx->ws, NULL);
if (p == NULL) {
VRT_fail(ctx, "j.string(): our of workspace");
return (NULL);
}
AZ((uintptr_t)p & 1);
return (p + 1);
}
// add a json value to the vsb:
// - json, number or literal as is
// - anything else as string
static void
vsbjvalue(struct vsb *vsb, const char *p)
{
if (! (is_J(p) || is_jnumber(p) || is_jlit(p)))
vsbjstring(vsb, p);
else
AZ(VSB_bcat(vsb, p, (ssize_t)strlen(p)));
}
VCL_STRING
vmod_array(VRT_CTX, VCL_STRANDS s)
{
struct vsb vsb[1];
const char *p;
int i;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
AN(s);
if (s->n == 0)
return ((const char *)jnil_array + 1);
for (i = 0; i < s->n; i++) {
if (s->p[i] != NULL)
continue;
VRT_fail(ctx, "Strand %d is NULL in j.array()", i);
return (NULL);
}
WS_VSB_new(vsb, ctx->ws);
VSB_mcat(vsb, jm_array); // [
vsbjvalue(vsb, s->p[0]);
for (i = 1; i < s->n; i++) {
AZ(VSB_putc(vsb, ','));
vsbjvalue(vsb, s->p[i]);
}
AZ(VSB_putc(vsb, ']'));
p = WS_VSB_finish(vsb, ctx->ws, NULL);
if (p == NULL) {
VRT_fail(ctx, "j.array(): our of workspace");
return (NULL);
}
AZ((uintptr_t)p & 1);
return (p + 1);
}
// add a json key:
// - json string as is
// - other json, number or literal fails
// - anything else as string
static int
vsbjkey(VRT_CTX, struct vsb *vsb, const char *p)
{
if (is_Jstring(p))
AZ(VSB_bcat(vsb, p, (ssize_t)strlen(p)));
else if (is_J(p) || is_jnumber(p) || is_jlit(p)) {
VRT_fail(ctx, "keys must be strings, got %s", p);
return (0);
}
else
vsbjstring(vsb, p);
return (1);
}
// add key: value
static int
vsbjkv(VRT_CTX, struct vsb *vsb, const char *k, const char *v)
{
if (! vsbjkey(ctx, vsb, k))
return (0);