Add docs and examples for custom VCL.

docs/README.md

@@ -12,6 +12,9 @@ The docs in this folder cover these topics:
 
 * [Varnish as a Kubernetes Ingress](varnish-as-ingress.md)
 
+* [Custom VCL](/docs/custom-vcl.md): restrictions, conventions, and
+  links to further information about VCL
+
 * [Self-sharding Varnish cluster](self-sharding.md): High-level
   discussion of the design
 

docs/custom-vcl.md

+# Custom VCL
+
+You can implement your own logic for caching policy and request
+processing by writing custom
+[VCL](https://varnish-cache.org/docs/6.1/reference/vcl.html),
+configured simply as the string value of ``spec.vcl`` in a
+[VarnishConfig](/docs/ref-varnish-cfg.md#specvcl) manifest. The string
+is appended "as-is" to VCL code that is generated by the controller to
+implement Ingress, VarnishConfig and other features, and loaded by the
+Varnish instances in your cluster. This gives you all of the
+capabilites of VCL, subject to the restrictions described in the
+following.
+
+The Varnish container specialized for Ingress runs Varnish 6.1.1, and
+the generated VCL code is at version 4.0. So your custom VCL must be
+compatible with those versions.
+
+For further information about VCL see:
+
+* [vcl(7)](https://varnish-cache.org/docs/6.1/reference/vcl.html)
+
+* [Varnish Processing States](https://varnish-cache.org/docs/6.1/reference/states.html)
+  in the Varnish reference manual.
+
+* The
+  [VCL chapter](https://varnish-cache.org/docs/6.1/users-guide/vcl.html)
+  in the Varnish User's Guide.
+
+* The
+  [VCL article](https://www.varnish-software.com/wiki/content/tutorials/varnish/vcl.html)
+  at Varnish Software's wiki.
+
+* The
+  [VCL chapter](https://www.oreilly.com/library/view/getting-started-with/9781491972212/ch04.html)
+  in Thijs Feryn's book "Getting Started with Varnish Cache".
+
+* The
+  [VCL Basics](https://book.varnish-software.com/4.0/chapters/VCL_Basics.html)
+  chapter in Varnish Software's "Varnish Book" (written for Varnish
+  4.0, but mostly still correct for version 6.1).
+
+## Use At Your Own Risk
+
+The correctness of the Ingress implementation, viewed broadly,
+encompasses:
+
+* Correct implementation of Ingress rules -- routing requests to
+  Services based on the Host header and URL path, as specified by an
+  Ingress.
+
+* Correct implementation of VarnishConfig configurations, and other
+  features documented for the project.
+
+The VCL code generated by the controller implements these features,
+and is subject to these requirements. Custom VCL could interfere with
+the work of the generated code. In that case you're on your own --
+broken custom VCL is not a bug of the Ingress implementation.
+
+By following some conventions and restrictions, you can mesh custom
+code smoothly with the generated code:
+
+* The prefix ``vk8s`` (case-insensitive) is reserved by generated code
+  for:
+
+    * VCL symbol names
+
+    * Header names
+
+    * String values
+
+So if you don't change anything with the ``vk8s`` prefix, you won't
+change something that the generated code depends on.
+
+* Response status values >= 60000 (for ``resp.status`` and
+  ``beresp.status``) are reserved for generated code.
+
+VCL allows you to set
+[status codes >= 1000](https://varnish-cache.org/docs/6.1/reference/vcl.html#beresp) --
+these are truncated to 3-digit codes in a client response, so as to form
+a valid HTTP response. This is a way to control logic in VCL -- set a
+special response code for which specific behaviors are implemened (for
+example in ``vcl_synth`` or ``vcl_deliver``). By avoiding codes in the
+reserved range, you avoid affecting the work of the generated code.
+
+* Custom VCL is always appended *after* generated VCL.
+
+VCL allows you to define the ``vcl_*`` subroutines
+[more than once](https://varnish-cache.org/docs/6.1/reference/vcl.html#multiple-subroutines) --
+they are concatenated in the order in which they appear in the VCL
+source, and hence have the effect of being executed in that order,
+until one of them calls ``return``, or all of them fall through. If
+none of them calls ``return``, then the built-in versions of the
+subroutines run (in other words,
+[``builtin.vcl``](https://github.com/varnishcache/varnish-cache/blob/varnish-6.1.1/bin/varnishd/builtin.vcl)
+is implicitly concatenated last).
+
+By appending custom VCL after generated VCL, the controller ensures
+that code to implement Ingress and its features is executed before
+anything in custom VCL that may change the flow of control.
+
+## Further restrictions
+
+* It is not possible to use the
+  [``include`` directive](https://varnish-cache.org/docs/6.1/reference/vcl.html#include-statement) --
+  all of your custom VCL must form a single string in the VarnishConfig
+  manifest.
+
+* You cannot
+  [``return(vcl(LABEL))``](https://varnish-cache.org/docs/6.1/users-guide/vcl-built-in-subs.html?highlight=label#vcl-recv)
+  from ``vcl_recv``. The code is already running in a labeled VCL
+  instance, and it is not possible to branch from a label to another
+  label. The controller uses labels for the readiness state and to
+  direct flow to the config for the current desired state, so you
+  should refrain from using labels.

docs/ref-varnish-cfg.md

@@ -459,3 +459,33 @@ when the URL begins with "/tea", and the Host header is exactly
 
 See the [``examples/`` folder](/examples/acl) for working examples
 of ACL configurations.
+
+## ``spec.vcl``
+
+The ``vcl`` element is optional, and if present contains a non-empty
+string that is appended "as-is" to VCL code generated by the
+controller that is loaded by Varnish instances. This provides a means
+to write custom
+[VCL](https://varnish-cache.org/docs/6.1/reference/vcl.html). For
+example:
+
+```
+  vcl: |
+    sub vcl_deliver {
+    	set resp.http.Hello = "world";
+    }
+
+    sub vcl_backend_response {
+    	set beresp.http.Backend = beresp.backend.name;
+    }
+```
+
+Custom VCL currently cannot be validated at apply time. If the VCL is
+invalid, the controller receives error messages from the compiler when
+it attempts to load the code. Check the Varnish Service for
+[Events](/docs/monitor.md) at the Warning level and the controller log
+for error messages. Varnish usually returns status 106 for invalid
+VCL.
+
+See the [docs](/docs/custom-vcl.md) for conventions and restrictions
+that apply to custom VCL, and for links to more information about VCL.

examples/README.md

@@ -13,6 +13,8 @@ requirements.
 * [Customizing the Pod template](/examples/varnish_pod_template)
   for Varnish
 
+* [Custom VCL](/examples/custom-vcl)
+
 * [Self-sharding Varnish cluster](/examples/self-sharding)
   ([docs](/docs/self-sharding.md))
 

examples/custom-vcl/README.md

+# Custom VCL example
+
+Custom VCL can be written simply by setting a string value for the
+``spec.vcl`` field of a [VarnishConfig](/docs/ref-varnish-cfg.md). The
+string is appended "as-is" to code generated by the controller, and
+loaded by Varnish instances in the cluster. The manifest in this folder
+is a simple example.
+
+```
+spec:
+  vcl: |
+    sub vcl_deliver {
+    	set resp.http.Hello = "world";
+    }
+
+    sub vcl_backend_response {
+    	set beresp.http.Backend = beresp.backend.name;
+    }
+```
+
+The sample code sets the client response header ``Hello`` to "world",
+and sets the backend response header ``Backend`` (which is passed
+along to the client response) to the name of backend from which the
+response was fetched.
+
+Notice that the ``|`` notation is used for a block-style multiline
+YAML string, indented so that all of the lines in the string are
+interpreted as the value of ``vcl``. The code can of course be written
+as an unformatted string (it doesn't matter to the VCL compiler), but
+you may want to use one of
+[YAML's multiline formats](https://yaml-multiline.info/) to write
+well-formatted, readable code.
+
+In a cluster where an Ingress and the associated Services are defined
+(assume the ["cafe" example](/examples/hello) in the following), apply
+the VarnishConfig definition:
+
+```
+$ kubectl apply -f custom-vcl.yaml
+```
+
+Custom VCL currently cannot be validated at apply time. Your code must
+of course conform to valid VCL syntax; if not, the controller receives
+error messages from the compiler when it attempts to load the code. In
+that case, [Events](/docs/monitor.md) at the Warning level are
+generated for the Varnish Service, including the error message from
+the compiler. The error message also appears in the controller
+log. Varnish usually returns status 106 for invalid VCL.
+
+When the definition is successfully synced, it can be verified with
+curl (where ``$ADDR`` is the external address of the cluster, and
+``$PORT`` is the port at which requests are received by the Ingress
+implementation):
+
+```
+$ curl -v -x $ADDR:$PORT http://cafe.example.com/coffee
+[...]
+
+< HTTP/1.1 200 OK
+[...]
+< Backend: vk8s_coffee-svc_172_17_0_8
+[...]
+< Hello: world
+[...]
+
+Server address: 172.17.0.8:80
+Server name: coffee-6c47b9cb9c-ntv5r
+Date: 07/Jan/2019:18:30:45 +0000
+URI: /coffee
+Request ID: b9a3abf0c2d378fe7282b614a556f46b
+```
+
+Note that the backend name in the ``Backend`` response header is
+generated by the controller.
+
+See the [docs](/docs/custom-vcl.md) for considerations about writing
+custom VCL for the Ingress implementation, and for links to more
+information about VCL.

examples/custom-vcl/custom-vcl.yaml

+# Sample configuration for adding custom VCL
+
+apiVersion: "ingress.varnish-cache.org/v1alpha1"
+kind: VarnishConfig
+metadata:
+  name: custom-vcl-cfg
+spec:
+  # The services array is required and must have at least one element.
+  # Lists the Service names of Varnish services in the same namespace
+  # to which this config is to be applied.
+  services:
+    - varnish-ingress
+
+  # Custom VCL is appended "as-is" to VCL generated by the controller.
+  # see: https://varnish-cache.org/docs/6.1/reference/vcl.html
+  vcl: |
+    sub vcl_deliver {
+    	set resp.http.Hello = "world";
+    }
+
+    sub vcl_backend_response {
+    	set beresp.http.Backend = beresp.backend.name;
+    }