Add markdown for sr.ht and improve documentation

parent 4c09e08e
---
subtitle: Installing from Source
title: vmod zipflow Installation Instructions
---
# generic
- Compile and install varnish-cache 7.3.0 or later by the [Compiling
Varnish from
source](https://varnish-cache.org/docs/trunk/installation/install_source.html)
documentation.
Note in particular the `autoconf-archive` requirement.
## compiling
- set and export the `VARNISHSRC` environment variable to point to the
varnish-cache source tree
- In the source tree, run:
./bootstrap
make check
make install
The last step will probably need to be run as root (e.g. using `sudo`).
======================================
vmod zipflow Installation Instructions
======================================
.. role:: ref(emphasis)
Installing from Source
======================
.. _Compiling Varnish from source: https://varnish-cache.org/docs/trunk/installation/install_source.html
generic
-------
* Compile and install varnish-cache 7.3.0 or later by the
`Compiling Varnish from source`_ documentation.
Note in particular the ``autoconf-archive`` requirement.
compiling
~~~~~~~~~
* set and export the ``VARNISHSRC`` environment variable to point to
the varnish-cache source tree
* In the source tree, run::
./bootstrap
make check
make install
The last step will probably need to be run as root (e.g. using ``sudo``).
Copyright 2022 UPLEX Nils Goroll Systemoptimierung
Copyright 2022, 2023 UPLEX Nils Goroll Systemoptimierung
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
......
......@@ -3,3 +3,12 @@ ACLOCAL_AMFLAGS = -I m4 -I @VARNISHAPI_DATAROOTDIR@/aclocal
DISTCHECK_CONFIGURE_FLAGS = RST2MAN=:
SUBDIRS = src
.rst.md:
pandoc -s -o $@ $<
noinst_DATA = README.md INSTALL.md
BUILT_SOURCES = $(noinst_DATA)
DISTCLEANFILES = $(noinst_DATA)
---
title: "vmod zipflow: On-the-fly ZIP bundling for Varnish Cache"
---
This module provides a [Varnish-Cache](https://varnish-cache.org/)
Delivery Processor (VDP) interface to [Mark
Adler](https://mstdn.social/@madler)\'s
[zipflow](https://github.com/madler/zipflow) library to package and
compress responses into the ZIP format.
# PROJECT RESOURCES
- The primary repository is at
<https://code.uplex.de/uplex-varnish/slash>
This server does not accept user registrations, so please \...
- use the [sourcehut bug tracker](https://todo.sr.ht/~slink/klarlack)
to report issues.
You may also use the [sourcehut git
repository](https://git.sr.ht/~slink/libvmod-zipflow).
# INTRODUCTION
Using this VMOD, ZIP responses can be generated on the fly from an
arbitrary number of objects. The mechanism is similar to ESI handling in
Varnish-Cache: For each object to be added as a file in the ZIP archive,
a sub-request is generated.
The resulting ZIP is generated while it is being delived. It is not
stored (this can, however, be achieved with requests from varnish to
itself).
## Examples
Example: Send the response body as a zip file containing \"filename\"
: :
import zipflow;
sub vcl_init {
: zipflow.set_level(9);
}
sub vcl_deliver {
: zipflow.meta(\"filename\"); set resp.filters += \" zipflow\";
}
Example: Create a two subrequests for other URLs, which are bundled into
the ZIP response :
import zipflow;
sub vcl_recv {
if (req.url == "/zip") {
return (synth(1200));
}
}
sub synth_zipflow {
if (zipflow.is_subreq()) {
return;
}
# activate zipflow
set resp.filters = "zipflow";
set resp.body = " "; // REQUIRED!
# do not put this body into the zip
zipflow.bundle(false);
# create two subrequests to put into the zip
zipflow.subreq("/file1");
zipflow.subreq("/file2");
return (deliver);
}
sub vcl_synth {
if (resp.status == 1200) {
call synth_zipflow;
}
}
This README is only intended as a first introduction, for more details,
please refer to the *vmod_zipflow(3)* man page. If you are reading this
file on line, it should also be available as
[vmod_zipflow.man.md](https://git.sr.ht/~slink/libvmod-zipflow/tree/master/item/src/vmod_zipflow.man.md).
# INSTALLATION
See
[INSTALL.md](https://git.sr.ht/~slink/libvmod-zipflow/tree/master/item/INSTALL.md).
# SUPPORT
To report bugs, please use the [sourcehut bug
tracker](https://todo.sr.ht/~slink/klarlack).
For enquiries about professional service and support, please contact
<info@uplex.de>.
# CONTRIBUTING
To contribute to the project, please use the [sourcehut bug
tracker](https://todo.sr.ht/~slink/klarlack).
To support the project\'s development and maintenance, there are several
options:
- Donate money through
[paypal](https://www.paypal.com/donate/?hosted_button_id=BTA6YE2H5VSXA).
If you wish to receive a commercial invoice, please add your details
(address, email, any requirements on the invoice text) to the
message sent with your donation.
- Become a [github sponsor](https://github.com/sponsors/nigoroll).
- Contact <info@uplex.de> to receive a commercial invoice for SWIFT
payment.
# ACKNOWLEDGEMENTS
This project was greatly facilitated by [Mark
Adler](https://mstdn.social/@madler)\'s
[zipflow](https://github.com/madler/zipflow) library.
# COPYRIGHT
Copyright 2022, 2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
See LICENSE
..
.. NB: This file is machine generated, DO NOT EDIT!
..
.. Edit ./vmod_zipflow.vcc and run make instead
..
=======================================================
vmod zipflow: On-the-fly ZIP bundling for Varnish Cache
=======================================================
.. role:: ref(emphasis)
============
vmod_zipflow
============
.. _Varnish-Cache: https://varnish-cache.org/
----------------------
Varnish zipflow Module
----------------------
.. _zipflow: https://github.com/madler/zipflow
:Manual section: 3
.. _Mark Adler: https://mstdn.social/@madler
SYNOPSIS
========
This module provides a `Varnish-Cache`_ Delivery Processor (VDP)
interface to `Mark Adler`_\ 's `zipflow`_ library to package and compress
responses into the ZIP format.
.. parsed-literal::
PROJECT RESOURCES
=================
import zipflow [as name] [from "path"]
VOID subreq(STRING url, STRING host)
VOID subreqs_from_body(ENUM which)
BOOL is_subreq()
VOID bundle(BOOL)
VOID set_level(INT level)
VOID meta([STRING name], [STRING mode], [TIME atime], [TIME mtime])
NOTE
====
* The primary repository is at https://code.uplex.de/uplex-varnish/slash
This vmod needs https://github.com/varnishcache/varnish-cache/pull/3843
This server does not accept user registrations, so please ...
DESCRIPTION
===========
.. _sourcehut bug tracker: https://todo.sr.ht/~slink/klarlack
.. _zipflow: https://github.com/madler/zipflow
* use the `sourcehut bug tracker`_ to report issues.
This module provides a Varnish Delivery Processor (VDP) interface to
Mark Adler's `zipflow`_ library to package and compress
responses into the ZIP format.
.. _sourcehut git repository: https://git.sr.ht/~slink/libvmod-zipflow
You may also use the `sourcehut git repository`_.
INTRODUCTION
============
Using this VMOD, ZIP responses can be generated on the fly from an
arbitrary number of objects. The mechanism is similar to ESI handling
in Varnish-Cache: For each object to be added as a file in the ZIP
archive, a sub-request is generated.
The resulting ZIP is generated while it is being delived. It is not
stored (this can, however, be achieved with requests from varnish to
itself).
Examples
--------
Example: Send the response body as a zip file containing "filename"
::
......@@ -98,233 +93,60 @@ into the ZIP response
}
}
Example: Read URLs to bundle into the ZIP from the request body
::
import zipflow;
import std;
sub vcl_recv {
if (req.url == "/zip" && req.method == "POST") {
if (! std.cache_req_body(1M)) {
return (synth(400, "Need request body"));
}
return (synth(1200));
}
}
sub synth_zipflow {
if (zipflow.is_subreq()) {
return;
}
# activate zipflow
set resp.filters = "zipflow";
set resp.body = " "; // REQUIRED!
# do not put this body into the zip
zipflow.bundle(false);
# read urls from request body
zipflow.subreqs_from_body(req_body);
return (deliver);
}
sub vcl_synth {
if (resp.status == 1200) {
call synth_zipflow;
}
}
VCL INTERFACE REFERENCE
=======================
**NOTE** Varnish-Cache does not support content generated by Varnish
Delivery Processors if there is not response body to be sent. This
means that, on the top level, a response body must always be present,
as illustrated by ``set resp.body = " ";`` in the examples above.
.. _zipflow.subreq():
VOID subreq(STRING url, STRING host=0)
--------------------------------------
Restricted to: ``client``.
Issue a sub requets to *host*/*uri* when the VDP runs, similar to ESI
processing.
If *host* is omitted (default), it is taken from the parent request.
.. _vmod_zipflow.man.md: https://git.sr.ht/~slink/libvmod-zipflow/tree/master/item/src/vmod_zipflow.man.md
This function can be called any number of times to add multiple
files, it can even be called from a sub request, which is to say that
more files can be added while requests for files are processed.
This README is only intended as a first introduction, for more
details, please refer to the :ref:`vmod_zipflow(3)` man page. If you
are reading this file on line, it should also be available as
`vmod_zipflow.man.md`_.
The sub request can be identified using `zipflow.is_subreq()`_. In the
sub request, `zipflow.set_level()`_ and `zipflow.meta()`_ should be
used to control how zipflow handles the body.
Only sub requests with reponse status 200 will be included in the
resulting zip file.
.. _zipflow.subreqs_from_body():
VOID subreqs_from_body(ENUM {req_body, resp_body} which)
--------------------------------------------------------
Restricted to: ``client``.
*Note* this function should eventually be superseded with something
more versatile.
Parse the given body for tokens in one of the following formats,
separated by any whitespace (``\\r\\n\\t\\s``)
* ``http://``\ *host*\ *url*
* ``https://``\ *host*\ *url*
* ``//``\ *host*\ *url*
* *url*
with *host* containing any non-whitespace character except for ``/``
and *url* starting with ``/`` and run a sub request for each token as
if ``subreq(``\ *url*\ ``, ``\ *host*\ ``)`` was invoced, but not
using any workspace memory.
.. _RFC 3965: https://www.ietf.org/rfc/rfc3986.txt
*host* may not be empty. The syntactic ambiguity prevents *urls*
starting with ``///``.
(Note: *host* and *url* are used in varnish-cache terminology, in `RFC
3965`_ parlance, *host* is called authority and *url* is a path. The
``//`` scheme is called a network-path reference)
This function can only be called from the top level, that is, not from
a sub request.
For ``which = resp_body``, bundling of the current body is turned
off.
For ``which = req_body``, ``std.cache_req_body()`` must have been
called earlier.
The order of subrequests is:
* `zipflow.subreq()`_ initiated by the top level request
* ``resp_body``
* each subreq followed by any `zipflow.subreq()`_ initiated from it
* ``req_body``
* each subreq followed by any `zipflow.subreq()`_ initiated from it
Only sub requests with reponse status 200 will be included in the
resulting zip file.
.. _zipflow.is_subreq():
BOOL is_subreq()
----------------
Restricted to: ``client``.
True if the current request is a zipflow sub request.
.. _zipflow.bundle():
VOID bundle(BOOL)
-----------------
Restricted to: ``client``.
By default, any content handled by zipflow, be it in the parent or sub
reuqest, is bundled in the resulting ZIP file. This function can be
used to diable or re-enable bundling of the request's body.
.. _zipflow.set_level():
VOID set_level(INT level)
-------------------------
Restricted to: ``vcl_init``, ``client``.
Set the (default) compression level for the zipflow filter.
Valid values for `level` are -1 to select the default compression
level ``Z_DEFAULT_COMPRESSION``, which is also the overall default, or
between 0 and 9: 1 gives best speed, 9 gives best compression, 0 gives
no compression at all (the input data is simply copied a block at a
time). ``Z_DEFAULT_COMPRESSION`` requests a default compromise
between speed and compression (currently equivalent to level 6). [Text
copied from zlib.h]
When used from ``vcl_init {}``, sets the default for all tasks from
all VCLs. Any such call affects all VCLs using this vmod, setting the
default per VCL is not currently supported.
When used elsewhere, sets the level for the current task. Calls from
``esi_level`` > 0 have no effect.
.. _zipflow.meta():
INSTALLATION
============
VOID meta([STRING name], [STRING mode], [TIME atime], [TIME mtime])
-------------------------------------------------------------------
.. _INSTALL.md: https://git.sr.ht/~slink/libvmod-zipflow/tree/master/item/INSTALL.md
::
See `INSTALL.md`_.
VOID meta(
[STRING name],
[STRING mode],
[TIME atime],
[TIME mtime]
)
SUPPORT
=======
Set zip metadata for the currently streamed object. Defaults apply if
`zipflow.meta()`_ is not called, or arguments are not given.
To report bugs, please use the `sourcehut bug tracker`_.
* `name`: Filename
For enquiries about professional service and support, please contact
info@uplex.de\ .
Default: Last component of ``req.url`` or, if it is empty,
``unnamed_file``.
CONTRIBUTING
============
* `mode`: UNIX mode bits
To contribute to the project, please use the `sourcehut bug tracker`_.
This argument is of string type and interpreted as *octal*.
To support the project's development and maintenance, there are
several options:
Default: ``0644``
.. _paypal: https://www.paypal.com/donate/?hosted_button_id=BTA6YE2H5VSXA
* `atime`: Access time
.. _github sponsor: https://github.com/sponsors/nigoroll
Default: ``now``.
* Donate money through `paypal`_. If you wish to receive a commercial
invoice, please add your details (address, email, any requirements
on the invoice text) to the message sent with your donation.
* `mtime`: Modification time
* Become a `github sponsor`_.
Default: ``Last-Modified`` header of the response or, if it is not
available, ``now``.
* Contact info@uplex.de to receive a commercial invoice for SWIFT payment.
SEE ALSO
========
ACKNOWLEDGEMENTS
================
vcl\(7),
varnishd\(1)
This project was greatly facilitated by `Mark Adler`_\ 's `zipflow`_
library.
COPYRIGHT
=========
::
Copyright 2022 UPLEX Nils Goroll Systemoptimierung
Copyright 2022, 2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
......
AC_PREREQ([2.68])
AC_INIT([libvmod-zipflow], [0.1])
AC_INIT([libvmod-zipflow], [0.8])
AC_CONFIG_MACRO_DIR([m4])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
......
......@@ -60,10 +60,18 @@ dist_doc_DATA = \
dist_man_MANS = \
vmod_zipflow.3
noinst_DATA = vmod_zipflow.man.md
BUILT_SOURCES = $(noinst_DATA)
DISTCLEANFILES = $(noinst_DATA)
.rst.1:
$(AM_V_GEN) $(RST2MAN) $< $@
.rst.md:
$(AM_V_GEN) pandoc -s -o $@ $<
.PHONY: flint
flint:
......
---
manual section: |
3
subtitle: Varnish zipflow Module
title: vmod_zipflow
---
# SYNOPSIS
::: parsed-literal
import zipflow \[as name\] \[from \"path\"\]
VOID subreq(STRING url, STRING host)
VOID subreqs_from_body(ENUM which)
BOOL is_subreq()
VOID bundle(BOOL)
VOID set_level(INT level)
VOID meta(\[STRING name\], \[STRING mode\], \[TIME atime\], \[TIME
mtime\])
:::
# NOTE
This vmod needs
<https://github.com/varnishcache/varnish-cache/pull/3843>
# DESCRIPTION
This module provides a Varnish Delivery Processor (VDP) interface to
Mark Adler\'s [zipflow](https://github.com/madler/zipflow) library to
package and compress responses into the ZIP format.
Example: Send the response body as a zip file containing \"filename\"
: :
import zipflow;
sub vcl_init {
: zipflow.set_level(9);
}
sub vcl_deliver {
: zipflow.meta(\"filename\"); set resp.filters += \" zipflow\";
}
Example: Create a two subrequests for other URLs, which are bundled into
the ZIP response :
import zipflow;
sub vcl_recv {
if (req.url == "/zip") {
return (synth(1200));
}
}
sub synth_zipflow {
if (zipflow.is_subreq()) {
return;
}
# activate zipflow
set resp.filters = "zipflow";
set resp.body = " "; // REQUIRED!
# do not put this body into the zip
zipflow.bundle(false);
# create two subrequests to put into the zip
zipflow.subreq("/file1");
zipflow.subreq("/file2");
return (deliver);
}
sub vcl_synth {
if (resp.status == 1200) {
call synth_zipflow;
}
}
Example: Read URLs to bundle into the ZIP from the request body
: :
import zipflow; import std;
sub vcl_recv {
:
if (req.url == \"/zip\" && req.method == \"POST\") {
:
if (! std.cache_req_body(1M)) {
: return (synth(400, \"Need request body\"));
} return (synth(1200));
}
}
sub synth_zipflow {
:
if (zipflow.is_subreq()) {
: return;
} \# activate zipflow set resp.filters = \"zipflow\"; set
resp.body = \" \"; // REQUIRED!
\# do not put this body into the zip zipflow.bundle(false);
\# read urls from request body
zipflow.subreqs_from_body(req_body); return (deliver);
}
sub vcl_synth {
:
if (resp.status == 1200) {
: call synth_zipflow;
}
}
# VCL INTERFACE REFERENCE
**NOTE** Varnish-Cache does not support content generated by Varnish
Delivery Processors if there is not response body to be sent. This means
that, on the top level, a response body must always be present, as
illustrated by `set resp.body = " ";` in the examples above.
## VOID subreq(STRING url, STRING host=0) {#zipflow.subreq()}
Restricted to: `client`.
Issue a sub requets to *host*/*uri* when the VDP runs, similar to ESI
processing.
If *host* is omitted (default), it is taken from the parent request.
This function can be called any number of times to add multiple files,
it can even be called from a sub request, which is to say that more
files can be added while requests for files are processed.
The sub request can be identified using
[zipflow.is_subreq()](#zipflow.is_subreq()). In the sub request,
[zipflow.set_level()](#zipflow.set_level()) and
[zipflow.meta()](#zipflow.meta()) should be used to control how zipflow
handles the body.
Only sub requests with reponse status 200 will be included in the
resulting zip file.
## VOID subreqs_from_body(ENUM {req_body, resp_body} which) {#zipflow.subreqs_from_body()}
Restricted to: `client`.
*Note* this function should eventually be superseded with something more
versatile.
Parse the given body for tokens in one of the following formats,
separated by any whitespace (`\\r\\n\\t\\s`)
- `http://`*hosturl*
- `https://`*hosturl*
- `//`*hosturl*
- *url*
with *host* containing any non-whitespace character except for `/` and
*url* starting with `/` and run a sub request for each token as if
`subreq(`*url*`,`*host*`)` was invoced, but not using any workspace
memory.
*host* may not be empty. The syntactic ambiguity prevents *urls*
: starting with `///`.
(Note: *host* and *url* are used in varnish-cache terminology, in [RFC
3965](https://www.ietf.org/rfc/rfc3986.txt) parlance, *host* is called
authority and *url* is a path. The `//` scheme is called a network-path
reference)
This function can only be called from the top level, that is, not from a
sub request.
For `which = resp_body`, bundling of the current body is turned off.
For `which = req_body`, `std.cache_req_body()` must have been called
earlier.
The order of subrequests is:
- [zipflow.subreq()](#zipflow.subreq()) initiated by the top level
request
- `resp_body`
- each subreq followed by any
[zipflow.subreq()](#zipflow.subreq()) initiated from it
- `req_body`
- each subreq followed by any
[zipflow.subreq()](#zipflow.subreq()) initiated from it
Only sub requests with reponse status 200 will be included in the
resulting zip file.
## BOOL is_subreq() {#zipflow.is_subreq()}
Restricted to: `client`.
True if the current request is a zipflow sub request.
## VOID bundle(BOOL) {#zipflow.bundle()}
Restricted to: `client`.
By default, any content handled by zipflow, be it in the parent or sub
reuqest, is bundled in the resulting ZIP file. This function can be used
to diable or re-enable bundling of the request\'s body.
## VOID set_level(INT level) {#zipflow.set_level()}
Restricted to: `vcl_init`, `client`.
Set the (default) compression level for the zipflow filter.
Valid values for [level]{.title-ref} are -1 to select the default
compression level `Z_DEFAULT_COMPRESSION`, which is also the overall
default, or between 0 and 9: 1 gives best speed, 9 gives best
compression, 0 gives no compression at all (the input data is simply
copied a block at a time). `Z_DEFAULT_COMPRESSION` requests a default
compromise between speed and compression (currently equivalent to level
6). \[Text copied from zlib.h\]
When used from `vcl_init {}`, sets the default for all tasks from all
VCLs. Any such call affects all VCLs using this vmod, setting the
default per VCL is not currently supported.
When used elsewhere, sets the level for the current task. Calls from
`esi_level` \> 0 have no effect.
## VOID meta(\[STRING name\], \[STRING mode\], \[TIME atime\], \[TIME mtime\]) {#zipflow.meta()}
VOID meta(
[STRING name],
[STRING mode],
[TIME atime],
[TIME mtime]
)
Set zip metadata for the currently streamed object. Defaults apply if
[zipflow.meta()](#zipflow.meta()) is not called, or arguments are not
given.
- \`name\`: Filename
Default: Last component of `req.url` or, if it is empty,
`unnamed_file`.
- \`mode\`: UNIX mode bits
This argument is of string type and interpreted as *octal*.
Default: `0644`
- \`atime\`: Access time
Default: `now`.
- \`mtime\`: Modification time
Default: `Last-Modified` header of the response or, if it is not
available, `now`.
# SEE ALSO
vcl(7), varnishd(1)
# COPYRIGHT
Copyright 2022,2023 UPLEX Nils Goroll Systemoptimierung
All rights reserved
Author: Nils Goroll <nils.goroll@uplex.de>
See LICENSE
#-
# Copyright 2022 UPLEX Nils Goroll Systemoptimierung
# Copyright 2022,2023 UPLEX Nils Goroll Systemoptimierung
# All rights reserved
#
# Author: Nils Goroll <nils.goroll@uplex.de>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment