#-
# Copyright (c) 2010-2017 Varnish Software AS
# All rights reserved.
#
# Author: Poul-Henning Kamp <phk@FreeBSD.org>
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.

$Module std 3 "Varnish Standard Module"
$ABI strict

DESCRIPTION
===========

`vmod_std` contains basic functions which are part and parcel of Varnish,
but which for reasons of architecture fit better in a VMOD.

One particular class of functions in vmod_std is the conversions functions
which all have the form::

	TYPE type(STRING, TYPE)

These functions attempt to convert STRING to the TYPE, and if that fails,
they return the second argument, which must have the given TYPE.

$Function STRING toupper(STRING_LIST s)

Description
	Converts the string *s* to uppercase.
Example
	set beresp.http.scream = std.toupper("yes!");

$Function STRING tolower(STRING_LIST s)

Description
	Converts the string *s* to lowercase.
Example
	set beresp.http.nice = std.tolower("VerY");

$Function VOID set_ip_tos(INT tos)

Description
	Sets the IP type-of-service (TOS) field for the current session
	to *tos*. Silently ignored if the listen address is a Unix
	domain socket.
	Please note that the TOS field is not removed by the end of the
	request so probably want to set it on every request should you
	utilize it.
Example
	| if (req.url ~ "^/slow/") {
	|	std.set_ip_tos(0);
	| }

$Function REAL random(REAL lo, REAL hi)

Description
	Returns a random real number between *lo* and *hi*.
	This function uses the "testable" random generator in varnishd
	which enables determinstic tests to be run (See m00002.vtc).
	This function should not be used for cryptographic applications.
Example
	set beresp.http.random-number = std.random(1, 100);

$Function VOID log(STRING_LIST s)

Description
	Logs the string *s* to the shared memory log, using VSL tag
	*SLT_VCL_Log*.
Example
	std.log("Something fishy is going on with the vhost " + req.http.host);

$Function VOID syslog(INT priority, STRING_LIST s)

Description
	Logs the string *s* to syslog tagged with *priority*. *priority*
	is formed by ORing the facility and level values. See your
	system's syslog.h file for possible values.

	Notice: Unlike VCL and other functions in the std vmod, this
	function will not fail VCL processing for workspace overflows:
	For an out of workspace condition, the ``syslog()`` function
	has no effect.
Example
	std.syslog(9, "Something is wrong");

	This will send a message to syslog using LOG_USER | LOG_ALERT.

$Function STRING fileread(PRIV_CALL, STRING)

Description
	Reads a file and returns a string with the content. The result
	is cached indefinitely per filename.
Example
	synthetic("Response was served by " + std.fileread("/etc/hostname"));

Consider that the entire contents of the file appear in the string
that is returned, including newlines that may result in invalid
headers if ``std.fileread()`` is used to form a header. In that case,
you may need to modify the string, for example with ``regsub()``::

  set beresp.http.served-by = regsub(std.fileread("/etc/hostname"), "\R$", "");

$Function BOOL file_exists(STRING path)

Description
	Returns `true` if path or the file pointed to by path exists,
	`false` otherwise.
Example
	| if (std.file_exists("/etc/return_503")) {
	|	return (synth(503, "Varnish is in maintenance"));
	| }

$Function VOID collect(HEADER hdr, STRING sep=", ")

Description
	Collapses multiple *hdr* headers into one long header. The
	default separator *sep* is the standard comma separator to
	use when collapsing headers, with an additional  whitespace
	for pretty printing.

	Care should be taken when collapsing headers. In particular
	collapsing Set-Cookie will lead to unexpected results on the
	browser side.
Examples
	| std.collect(req.http.accept);
	| std.collect(req.http.cookie, "; ");

$Function DURATION duration(STRING s, DURATION fallback)

Description
	Converts the string *s* to seconds. *s* must be quantified
	with ms (milliseconds), s (seconds), m (minutes), h (hours),
	d (days), w (weeks) or y (years) units. If conversion fails,
	*fallback* will be returned.
Example
	set beresp.ttl = std.duration("1w", 3600s);

$Function INT integer(STRING s, INT fallback)

Description
	Converts the string *s* to an integer. If conversion fails,
	*fallback* will be returned.
Example
	| if (std.integer(req.http.foo, 0) > 5) {
	|	...
	| }

$Function IP ip(STRING s, IP fallback, BOOL resolve = 1, STRING p = "80")

Description
	Converts the string *s* to the first IP number returned by the
	system library function `getaddrinfo(3)`. If conversion fails,
	*fallback* will be returned or VCL failure will happen.

	The IP address includes a port number that can be found with
	``std.port()`` that defaults to 80. The default port can be set
	to a different value with the *p* argument. It will be overriden
	if *s* contains both an IP address and a port number or service
	name.

	When *s* contains both, the syntax is either ``address:port`` or
	``address port``. If the address is a numerical IPv6 address it
	must be enclosed between brackets, for example ``[::1] 80`` or
	``[::1]:http``. The *fallback* may also contain both an address
	and a port, but its default port is always 80.

	If *resolve* is false, `getaddrinfo(3)` is called using
	``AI_NUMERICHOST`` and ``AI_NUMERICSERV`` to avoid network lookups
	depending on the system's `getaddrinfo(3)` or nsswitch configuration.
	This makes "numerical" IP strings and services cheaper to convert.

Example
	| if (std.ip(req.http.X-forwarded-for, "0.0.0.0") ~ my_acl) {
	|	...
	| }

$Function REAL real(STRING s, REAL fallback)

Description
	Converts the string *s* to a real. If conversion fails,
	*fallback* will be returned.
Example
	| if (std.real(req.http.foo, 0.0) > 5.5) {
	|	...
	| }

$Function INT real2integer(REAL r, INT fallback)

Description
	Rounds the real *r* to the nearest integer, but round halfway
	cases away from zero (see round(3)). If conversion fails,
	*fallback* will be returned.
Example
	set req.http.integer = std.real2integer(1140618699.00, 0);
	set req.http.posone = real2integer( 0.5, 0);	# =  1.0
	set req.http.negone = real2integer(-0.5, 0);	# = -1.0

$Function TIME real2time(REAL r, TIME fallback)

Description
	Rounds the real *r* to the nearest integer (see
	`func_real2integer`_) and returns the corresponding time when
	interpreted as a unix epoch. If conversion fails, *fallback*
	will be returned.
Example
	set req.http.time = std.real2time(1140618699.00, now);

$Function INT time2integer(TIME t, INT fallback)

Description
	Converts the time *t* to a integer. If conversion fails,
	*fallback* will be returned.
Example
	set req.http.int = std.time2integer(now, 0);

$Function REAL time2real(TIME t, REAL fallback)

Description
	Converts the time *t* to a real. If conversion fails,
	*fallback* will be returned.
Example
	set req.http.real = std.time2real(now, 1.0);

$Function BOOL healthy(BACKEND be)

Description
	Returns `true` if the backend *be* is healthy.

$Function INT port(IP ip)

Description
	Returns the port number of the IP address *ip*. Always returns
	0 for a ``*.ip`` variable whose value is ``0.0.0.0`` because
	the listen address is a Unix domain socket.

$Function VOID rollback(HTTP h)

Description
	Restores the *h* HTTP headers to their original state.
Example
	std.rollback(bereq);

$Function VOID timestamp(STRING s)

Description
	Introduces a timestamp in the log with the current time, using
	the string *s* as the label. This is useful to time the execution
	of lengthy VCL procedures, and makes the timestamps inserted
	automatically by Varnish more accurate.
Example
	std.timestamp("curl-request");

$Function STRING querysort(STRING)

Description
	Sorts the query string for cache normalization purposes.
Example
	set req.url = std.querysort(req.url);

$Function BOOL cache_req_body(BYTES size)

Description
	Caches the request body if it is smaller than *size*.  Returns
	`true` if the body was cached, `false` otherwise.

	Normally the request body is not available after sending it to
	the backend.  By caching it is possible to retry pass operations,
	e.g. POST and PUT.
Example
	| if (std.cache_req_body(1KB)) {
	|	...
	| }

$Function STRING strstr(STRING s1, STRING s2)

Description
	Returns a string beginning at the first occurrence of the string
	*s2* in the string *s1*, or an empty string if *s2* is not found.

	Note that the comparison is case sensitive.
Example
	| if (std.strstr(req.url, req.http.restrict)) {
	|	...
	| }

	This will check if the content of req.http.restrict occurs
	anywhere in req.url.

$Function TIME time(STRING s, TIME fallback)

Description
	Converts the string *s* to a time. If conversion fails,
	*fallback* will be returned.

	Supported formats:

	| "Sun, 06 Nov 1994 08:49:37 GMT"
	| "Sunday, 06-Nov-94 08:49:37 GMT"
	| "Sun Nov  6 08:49:37 1994"
	| "1994-11-06T08:49:37"
	| "784111777.00"
	| "784111777"
Example
	| if (std.time(resp.http.last-modified, now) < now - 1w) {
	|	...
	| }

$Function STRING getenv(STRING name)

Description
	Return environment variable *name* or the empty string.

	See getenv(3)
Example
	| set req.http.My-Env = std.getenv("MY_ENV");

$Function VOID late_100_continue(BOOL late)

Description
	Controls when varnish reacts to an `Expect: 100-continue` client
	request header.

	Varnish always generates a `100 Continue` response if
	requested by the client trough the `Expect: 100-continue`
	header when waiting for request body data.

	But, by default, the `100 Continue` response is already
	generated immediately after `vcl_recv` returns to reduce
	latencies under the assumption that the request body will be
	read eventually.

	Calling `std.late_100_continue(true)` in `vcl_recv` will cause
	the `100 Continue` response to only be sent when needed. This
	may cause additional latencies for processing request bodies,
	but is the correct behavior by strict interpretation of
	RFC7231.

	This function has no effect outside `vcl_recv` and after
	calling `std.cache_req_body()` or any other function consuming
	the request body.

Example
	| vcl_recv {
	|	std.late_100_continue(true);
	|
	|	if (req.method == "POST") {
	|		std.late_100_continue(false);
	|		return (pass);
	|	}
	|	...
	| }

$Function BOOL syntax(REAL)

Description
	Returns the true if VCL version is at least REAL.

$Function BOOL fnmatch(STRING pattern, STRING subject, BOOL pathname=1,
		       BOOL noescape=0, BOOL period=0)

Description
	Shell-style pattern matching; returns `true` if *subject*
	matches *pattern*, where *pattern* may contain wildcard
	characters such as \* or ?.

	The match is executed by the implementation of `fnmatch(3)` on
	your system. The rules for pattern matching on most systems
	include the following:

	* \* matches any sequence of characters

	* ? matches a single character

	* a bracket expression such as [abc] or [!0-9] is interpreted
	  as a character class according to the rules of basic regular
	  expressions (*not* PCRE regexen), except that ! is used for
	  character class negation instead of ^.

	If *pathname* is `true`, then the forward slash character / is
	only matched literally, and never matches \*, ? or a bracket
	expression. Otherwise, / may match one of those patterns.  By
	default, *pathname* is `true`.

	If *noescape* is `true`, then the backslash character \\ is
	matched as an ordinary character. Otherwise, \\ is an escape
	character, and matches the character that follows it in the
	`pattern`. For example, \\\\ matches \\ when *noescape* is
	`true`, and \\\\ when `false`. By default, *noescape* is
	`false`.

	If *period* is `true`, then a leading period character . only
	matches literally, and never matches \*, ? or a bracket
	expression. A period is leading if it is the first character
	in `subject`; if *pathname* is also `true`, then a period that
	immediately follows a / is also leading (as in "/.").  By
	default, *period* is `false`.

	`fnmatch()` invokes VCL failure and returns `false` if either
	of *pattern* or *subject* is NULL -- for example, if an unset
	header is specified.

Examples
	| # Matches URLs such as /foo/bar and /foo/baz
	| if (std.fnmatch("/foo/\*", req.url)) { ... }
	|
	| # Matches URLs such as /foo/bar/baz and /foo/baz/quux
	| if (std.fnmatch("/foo/\*/\*", bereq.url)) { ... }
	|
	| # Matches /foo/bar/quux, but not /foo/bar/baz/quux
	| if (std.fnmatch("/foo/\*/quux", req.url)) { ... }
	|
	| # Matches /foo/bar/quux and /foo/bar/baz/quux
	| if (std.fnmatch("/foo/\*/quux", req.url, pathname=false)) { ... }
	|
	| # Matches /foo/bar, /foo/car and /foo/far
	| if (std.fnmatch("/foo/?ar", req.url)) { ... }
	|
	| # Matches /foo/ followed by a non-digit
	| if (std.fnmatch("/foo/[!0-9]", req.url)) { ... }


SEE ALSO
========

* :ref:`varnishd(1)`
* :ref:`vsl(7)`
* `fnmatch(3)`
