Initial public release

parent ce11522e
......@@ -30,7 +30,7 @@ stamp-h1
# vmodtool
vcc_*_if.[ch]
vmod_*.rst
vmod_iconv.rst
# man
......
INSTALLATION
============
Building from source
~~~~~~~~~~~~~~~~~~~~
The VMOD is built on a system where an instance of Varnish is
installed, and the auto-tools will attempt to locate the Varnish
instance, and then pull in libraries and other support files from
there.
Quick start
-----------
This sequence should be enough in typical setups:
1. ``./bootstrap`` (for git-installation)
3. ``make``
4. ``make check`` (regression tests)
5. ``make install`` (may require root: sudo make install)
Alternative configs
-------------------
If you have installed Varnish to a non-standard directory, call
``autogen.sh`` and ``configure`` with ``PKG_CONFIG_PATH`` pointing to
the appropriate path. For example, when varnishd configure was called
with ``--prefix=$PREFIX``, use::
PKG_CONFIG_PATH=${PREFIX}/lib/pkgconfig
ACLOCAL_PATH=${PREFIX}/share/aclocal
export PKG_CONFIG_PATH ACLOCAL_PATH
Copyright 2023 UPLEX Nils Goroll Systemoptimierung
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
====================================
iconv character encoding conversions
====================================
.. role:: ref(emphasis)
.. _Varnish-Cache: https://varnish-cache.org/
This project provides access to :ref:`iconv(3)` character encoding
conversions to VCL.
PROJECT RESOURCES
=================
* The primary repository is at https://code.uplex.de/uplex-varnish/libvmod-iconv
This server does not accept user registrations, so please use ...
* the mirror at https://gitlab.com/uplex/varnish/libvmod-iconv for issues,
merge requests and all other interactions.
DESCRIPTION
===========
.. _UTF-8: https://en.wikipedia.org/wiki/UTF-8
.. _RFC9110: https://datatracker.ietf.org/doc/html/rfc9110#name-field-values
While at the time of release of this VMOD basically all of planet
earth has converged onto using `UTF-8`_ as the universal encoding for
digital text, other character sets still exist and, historically,
ISO-8859-1 encoding was even allowed in HTTP headers (see `RFC9110`_).
This VMOD provides access to the standard :ref:`iconv(3)` function for
character encoding conversion.
From/to conversion pairs are defined as objects, on which the
`xconverter.iconv()`_ method can be called.
Example
::
import iconv;
sub vcl_init {
# "UTF-8" is default and can be left out
new eurutf = iconv.converter(from="ISO-8859-15", to="UTF-8");
}
sub vcl_recv {
set req.http.conv = eurutf.iconv(req.http.input);
}
For cases where conversions might fail, a ``fallback`` argument can be
provided to the `xconverter.iconv()`_, which is returned instead of
triggering a VCL error for failures.
Continue reading the documentation in `vmod_iconv(3) <src/vmod_iconv.man.rst>`_.
INSTALLATION
============
See `INSTALL.rst <INSTALL.rst>`_ in the source repository.
SUPPORT
=======
.. _gitlab.com issues: https://gitlab.com/uplex/varnish/libvmod-iconv/-/issues
To report bugs, use `gitlab.com issues`_.
For enquiries about professional service and support, please contact
info@uplex.de\ .
CONTRIBUTING
============
.. _merge requests on gitlab.com: https://gitlab.com/uplex/varnish/libvmod-iconv/-/merge_requests
To contribute to the project, please use `merge requests on gitlab.com`_.
To support the project's development and maintenance, there are
several options:
.. _paypal: https://www.paypal.com/donate/?hosted_button_id=BTA6YE2H5VSXA
.. _github sponsor: https://github.com/sponsors/nigoroll
* Donate money through `paypal`_. If you wish to receive a commercial
invoice, please add your details (address, email, any requirements
on the invoice text) to the message sent with your donation.
* Become a `github sponsor`_.
* Contact info@uplex.de to receive a commercial invoice for SWIFT payment.
COPYRIGHT
=========
::
Copyright 2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
......@@ -9,3 +9,8 @@
// constructors not referenced
-esym(528, init_*)
-esym(528, assert_*)
// redundant declaration
-esym(763, vmod_*)
-e747 // prot coersion
\ No newline at end of file
/*-
* Copyright 2023 UPLEX Nils Goroll Systemoptimierung
* All rights reserved
*
* Author: Nils Goroll <nils.goroll@uplex.de>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
#include "config.h"
#include <iconv.h>
#include <string.h>
#include <stdlib.h>
#include <cache/cache.h>
#include "vcc_iconv_if.h"
struct VPFX(iconv_converter) {
unsigned magic;
#define ICONV_CONVERTER_MAGIC 0x1ded5b1d
char *vcl_name;
char *from;
char *to;
};
static void
icfree(struct VPFX(iconv_converter) **icp)
{
struct VPFX(iconv_converter) *ic;
TAKE_OBJ_NOTNULL(ic, icp, ICONV_CONVERTER_MAGIC);
//lint --e{774}
REPLACE(ic->vcl_name, NULL);
REPLACE(ic->from, NULL);
REPLACE(ic->to, NULL);
FREE_OBJ(ic);
}
VCL_VOID
vmod_converter__init(VRT_CTX, struct VPFX(iconv_converter) **icp,
const char *vcl_name, VCL_STRING from, VCL_STRING to)
{
struct VPFX(iconv_converter) *ic;
iconv_t cd;
AN(icp);
AZ(*icp);
if (from == NULL)
from = "UTF-8";
if (to == NULL)
to = "UTF-8";
cd = iconv_open(to, from);
if (cd == (iconv_t)-1) {
VRT_fail(ctx, "iconv.converter() failed: %s (%d)",
VAS_errtxt(errno), errno);
return;
}
AZ(iconv_close(cd));
ALLOC_OBJ(ic, ICONV_CONVERTER_MAGIC);
if (ic == NULL) {
VRT_fail(ctx, "iconv.converter() alloc failed");
return;
}
REPLACE(ic->vcl_name, vcl_name);
REPLACE(ic->from, from);
REPLACE(ic->to, to);
if (ic->vcl_name == NULL || ic->from == NULL || ic->to == NULL)
icfree(&ic);
else
*icp = ic;
return;
}
VCL_VOID
vmod_converter__fini(struct VPFX(iconv_converter) **icp)
{
AN(icp);
if (*icp == NULL)
return;
icfree(icp);
}
VCL_STRING
vmod_hello(VRT_CTX)
vmod_converter_iconv(VRT_CTX, struct VPFX(iconv_converter) *ic,
VCL_STRANDS in, VCL_STRING fallback)
{
const char *ret;
char *outp, *inp;
size_t convl, inl, outl;
iconv_t cd;
int i;
CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
return ("vmod-iconv");
CHECK_OBJ_NOTNULL(ic, ICONV_CONVERTER_MAGIC);
(void)ic;
AN(in);
cd = iconv_open(ic->to, ic->from);
if (cd == (iconv_t)-1) {
VRT_fail(ctx, ".iconv() failed: %s (%d)",
VAS_errtxt(errno), errno);
return (NULL);
}
inp = "<start>";
convl = (size_t)-1;
outl = WS_ReserveAll(ctx->ws);
ret = outp = WS_Reservation(ctx->ws);
for (i = 0; i < in->n; i++) {
inp = TRUST_ME(in->p[i]);
if (inp == NULL)
continue;
inl = strlen(inp);
convl = iconv(cd, &inp, &inl, &outp, &outl);
if (convl == (size_t)-1)
break;
}
AZ(iconv_close(cd));
// check space for NUL
if (convl != (size_t)-1 && outl == 0)
errno = E2BIG;
if (convl == (size_t)-1 || outl == 0) {
WS_Release(ctx->ws, 0);
if (fallback) {
VSLb(ctx->vsl, SLT_Error,
".iconv() failed with %s (%d) at %s",
VAS_errtxt(errno), errno, inp);
}
else {
VRT_fail(ctx, ".iconv() failed with %s (%d) at %s",
VAS_errtxt(errno), errno, inp);
}
return (fallback);
}
AN(outl);
*outp = '\0';
outp++;
WS_ReleaseP(ctx->ws, outp);
return (ret);
}
..
.. NB: This file is machine generated, DO NOT EDIT!
..
.. Edit ./vmod_iconv.vcc and run make instead
..
.. role:: ref(emphasis)
==========
vmod_iconv
==========
------------------------------------
iconv character encoding conversions
------------------------------------
:Manual section: 3
SYNOPSIS
========
.. parsed-literal::
import iconv [as name] [from "path"]
new xconverter = iconv.converter(STRING from, STRING to)
STRING xconverter.iconv(STRING, STRING fallback)
DESCRIPTION
===========
.. _UTF-8: https://en.wikipedia.org/wiki/UTF-8
.. _RFC9110: https://datatracker.ietf.org/doc/html/rfc9110#name-field-values
While at the time of release of this VMOD basically all of planet
earth has converged onto using `UTF-8`_ as the universal encoding for
digital text, other character sets still exist and, historically,
ISO-8859-1 encoding was even allowed in HTTP headers (see `RFC9110`_).
This VMOD provides access to the standard :ref:`iconv(3)` function for
character encoding conversion.
From/to conversion pairs are defined as objects, on which the
`xconverter.iconv()`_ method can be called.
Example
::
import iconv;
sub vcl_init {
# "UTF-8" is default and can be left out
new eurutf = iconv.converter(from="ISO-8859-15", to="UTF-8");
}
sub vcl_recv {
set req.http.conv = eurutf.iconv(req.http.input);
}
For cases where conversions might fail, a ``fallback`` argument can be
provided to the `xconverter.iconv()`_, which is returned instead of
triggering a VCL error for failures.
VMOD INTERFACE REFERENCE
========================
.. _iconv.converter():
new xconverter = iconv.converter(STRING from, STRING to)
--------------------------------------------------------
::
new xconverter = iconv.converter(STRING from=0, STRING to=0)
Define an iconv converter from character set *from* to character set
*to*.
Left out arguments default to ``UTF-8``.
The permitted values are usually listed by the ``iconv --list``
command, see :ref:`iconv(1)`.
Typical examples for valid character sets are ``UTF-8`` and
``ISO-8859-15``.
Systems with the GNU C library and the GNU libiconv library support
the following two suffixes to character set names:
* ``//TRANSLIT``: Replace characters which can not be represented in
the *to* character set with an approximation through one or several
similarly looking characters.
* ``//IGNORE``: Silently discard characters which can not be
represented in the *to* character set.
Use of invalid character sets fails loading the VCL.
.. _xconverter.iconv():
STRING xconverter.iconv(STRING, STRING fallback=0)
--------------------------------------------------
Run the converter on the given input. If conversion fails and
*fallback* is given, it is returned. Otherwise a VCL failure is
triggered.
SEE ALSO
========
* :ref:`vcl(7)`
* :ref:`varnishd(1)`
* :ref:`iconv(1)`
* :ref:`iconv_open(3)`
* :ref:`iconv(3)`
COPYRIGHT
=========
::
Copyright 2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
$Module iconv 3 "Varnish iconv Module"
#-
# Copyright 2023 UPLEX Nils Goroll Systemoptimierung
# All rights reserved
#
# Author: Nils Goroll <nils.goroll@uplex.de>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
$Module iconv 3 "iconv character encoding conversions"
$ABI vrt
DESCRIPTION
===========
This VCC file was generated by VCDK, it is used to for both the VMOD
interface and its manual using reStructuredText.
.. _UTF-8: https://en.wikipedia.org/wiki/UTF-8
.. _RFC9110: https://datatracker.ietf.org/doc/html/rfc9110#name-field-values
While at the time of release of this VMOD basically all of planet
earth has converged onto using `UTF-8`_ as the universal encoding for
digital text, other character sets still exist and, historically,
ISO-8859-1 encoding was even allowed in HTTP headers (see `RFC9110`_).
This VMOD provides access to the standard :ref:`iconv(3)` function for
character encoding conversion.
XXX: document vmod-iconv
From/to conversion pairs are defined as objects, on which the
`xconverter.iconv()`_ method can be called.
Example
::
import iconv;
import iconv;
sub vcl_init {
# "UTF-8" is default and can be left out
new eurutf = iconv.converter(from="ISO-8859-15", to="UTF-8");
}
sub vcl_recv {
set req.http.conv = eurutf.iconv(req.http.input);
}
For cases where conversions might fail, a ``fallback`` argument can be
provided to the `xconverter.iconv()`_, which is returned instead of
triggering a VCL error for failures.
VMOD INTERFACE REFERENCE
========================
$Object converter(STRING from = 0, STRING to = 0)
Define an iconv converter from character set *from* to character set
*to*.
Left out arguments default to ``UTF-8``.
The permitted values are usually listed by the ``iconv --list``
command, see :ref:`iconv(1)`.
Typical examples for valid character sets are ``UTF-8`` and
``ISO-8859-15``.
Systems with the GNU C library and the GNU libiconv library support
the following two suffixes to character set names:
* ``//TRANSLIT``: Replace characters which can not be represented in
the *to* character set with an approximation through one or several
similarly looking characters.
sub vcl_deliver {
set resp.http.Hello = iconv.hello();
}
* ``//IGNORE``: Silently discard characters which can not be
represented in the *to* character set.
XXX: define vmod-iconv interface
Use of invalid character sets fails loading the VCL.
$Function STRING hello()
$Method STRING .iconv(STRANDS, STRING fallback = 0)
Description
Hello world for vmod-iconv
Run the converter on the given input. If conversion fails and
*fallback* is given, it is returned. Otherwise a VCL failure is
triggered.
SEE ALSO
========
vcl\(7),
varnishd\(1)
* :ref:`vcl(7)`
* :ref:`varnishd(1)`
* :ref:`iconv(1)`
* :ref:`iconv_open(3)`
* :ref:`iconv(3)`
varnishtest "test vmod-iconv"
server s1 {
rxreq
txresp
} -start
varnish v1 -vcl {
import iconv;
import blob;
backend proforma none;
varnish v1 -vcl+backend {
import iconv;
sub vcl_init {
new eurutf = iconv.converter("ISO-8859-15");
new utfeur = iconv.converter(to = "ISO-8859-15");
}
sub vcl_deliver {
set resp.http.Hello = iconv.hello();
}
sub vcl_recv {
return (synth(200));
}
sub do_eurutf {
if (req.http.fallback != "") {
set resp.http.out = eurutf.iconv(
blob.transcode(BASE64, encoded=req.http.in),
fallback=req.http.fallback);
} else {
set resp.http.out = eurutf.iconv(
blob.transcode(BASE64, encoded=req.http.in));
}
return (deliver);
}
sub do_utfeur {
if (req.http.fallback != "") {
set resp.http.out = blob.transcode(encoding=URL,
encoded=utfeur.iconv(req.http.in,
fallback=req.http.fallback));
} else {
set resp.http.out = blob.transcode(encoding=URL,
encoded=utfeur.iconv(req.http.in));
}
return (deliver);
}
sub vcl_synth {
if (req.url == "/eurutf") {
call do_eurutf;
} else if (req.url == "/utfeur") {
call do_utfeur;
}
set resp.status = 404;
}
} -start
client c1 {
txreq
txreq -url "/eurutf" -hdr "in: pA=="
rxresp
expect resp.status == 200
expect resp.http.out == "€"
txreq -url "/utfeur" -hdr "in: €"
rxresp
expect resp.http.out == "%a4"
expect resp.status == 200
txreq -url "/utfeur" -hdr "fallback: bad" -hdr "in: 🐰"
rxresp
expect resp.status == 200
expect resp.http.Hello == "vmod-iconv"
expect resp.http.out == "bad"
txreq -url "/utfeur" -hdr "in: 🐰"
rxresp
expect resp.status == 500
} -run
varnish v1 -errvcl {iconv.converter() failed: Invalid argument (22)} {
import iconv;
backend proforma none;
sub vcl_init {
new eurutf = iconv.converter("FOO");
}
}
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment