Add docs & examples for customizing the Varnish Pod template.

deploy/README.md

@@ -161,7 +161,6 @@ properly:
           containerPort: 6081
 ```
 
-  **TO DO**: The ports are currently hard-wired to these port numbers.
   A port for TLS access is currently not supported.
 * ``volumeMounts`` and ``volumes`` must be specified so that the
   Secret defined above is available to Varnish. The ``secretName``
@@ -183,9 +182,7 @@ properly:
           - key: admin
             path: _.secret
 ```
-  **TO DO**: The ``mountPath`` is currently hard-wired to
-  ``/var/run/varnish``. The ``key`` is hard-wired to ``admin``, and
-  ``path`` to ``_.secret``.
+  **TO DO**: The ``key`` is hard-wired to ``admin``.
 
 * The liveness check should determine if the Varnish master process is
   running. Since Varnish is started in the foreground as the entry
@@ -215,60 +212,15 @@ properly:
 
   The port name must match the name given for port 8080 above.
 
-#### varnishd options
-
-Varnish command-line options can be specified using the ``args`` section
-of the ``container`` specfication:
-
-```
-      containers:
-      - image: varnish-ingress/varnish
-        name: varnish-ingress
-        # [...]
-        args:
-        # Starts varnishd with: -l 80M -p default_grace=10
-        - -l
-        - 80M
-        - -p
-        - default_grace=10
-```
-
-See
-[``varnishd(1)``](https://varnish-cache.org/docs/6.1/reference/varnishd.html#options)
-for details about the available options.
-
-Because of the fact that the container starts with a number of options
-in order to implement the role of an Ingress, there are restrictions
-on the options that you can or should set. Some of them result in
-illegal combinations of options, which causes varnishd to terminate
-and the container to crash.  Others will be ignored, since the Varnish
-instance is managed by the controller.  Still others may interfere
-with the operation as an Ingress.
-
-Among these restrictions are:
-
-* You MAY NOT use any of the ``-C``, ``-d``, ``-I``, ``-S``, ``-T``,
-  ``-V``, ``-x`` or ``-?`` options.
-
-* The ``-p vcl_path`` parameter MAY NOT be changed.
-
-* ``-b`` or ``-f`` SHOULD NOT be set, since they will be ignored (but
-  their use does not cause an error).
-
-* ``-a`` CAN be set to define more listener ports for regular HTTP
-  client traffic; to be useful, these must be declared in the
-  ``ports`` specification (as with the port named ``http`` in the
-  example above). The listener name ``vk8s`` MAY NOT be used (it is
-  reserved for the listener used for readiness checks, declared as
-  ``k8sport`` above).
-
-* ``-M`` CAN be set, so that Varnish will connect to an address
-  listening for the administrative interface. The controller will not
-  use that address, but an admin client can use it to monitor the
-  Varnish instance separately. But an admin client MAY NOT call
-  ``vcl.use`` to activate any configuration, or ``vcl.discard`` to
-  unload one, otherwise it interferes with the implementation of
-  Ingress.
+The Deployment configuration in the current folder shows the default
+Pod template for running Varnish, but it can be customized by setting
+[varnishd command-line options](https://varnish-cache.org/docs/6.1/reference/varnishd.html#options)
+in ``args`` and/or environment variables in ``env``. You may need to
+do so, for example, to set the PROXY protocol for the HTTP listener,
+change container port numbers, or configure Varnish tunables. See the
+[documentation](varnish-pod-template.md) for details and requirements,
+and the [``examples/`` folder](/examples/varnish_pod_templates) for
+working examples.
 
 ### Expose the Varnish HTTP and admin ports
 
@@ -300,8 +252,7 @@ The Service definition must fulfill some requirements:
   controller to identify the admin ports by name for the Endpoints
   that realize the Varnish Service.
 
-  **TO DO**: The port number is hard-wired to 6081, and the
-  ``port.name`` is hardwired to ``varnishadm``.
+  **TO DO**: The ``port.name`` is hardwired to ``varnishadm``.
 
 * The Service must be defined so that the cluster API will allow
   Endpoints to be listed when the container is not ready (since

deploy/varnish.yaml

@@ -38,15 +38,6 @@ spec:
           httpGet:
             path: /ready
             port: k8s
-        args:
-        # varnishd command-line options
-        # In this example: varnishd -l 80M -p default_grace=10
-        # These are default values for the given options in Varnish 6.1.
-        # Shown here to demonstrate setting options for Varnish.
-        - -l
-        - 80M
-        - -p
-        - default_grace=10
       volumes:
       - name: adm-secret
         secret:

docs/README.md

@@ -6,6 +6,7 @@ The docs in this folder cover these topics:
 
   * [``VarnishConfig`` Custom Resource](ref-varnish-cfg.md)
   * [controller command-line options](ref-cli-options.md)
+  * [customizing the Pod template](varnish-pod-template.md) for Varnish
 
 * [Logging, Events and the Varnish Service monitor](monitor.md)
 

docs/varnish-pod-template.md

+# Customizing the Varnish Pod template
+
+The [deployment instructions](/deploy) include a
+[sample manifest](/deploy/varnish.yaml) with a default configuration
+of the Pod template for a Varnish instance, which can be used in the
+configuration of a controller for Varnish (such as a Deployment,
+DaemonSet, and so forth). The Pod template can be customized by
+setting command-line arguments in the ``args`` section, and/or
+environment variables in ``env``. You may want to do this in order to:
+
+* require the
+  [PROXY protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)
+  for the Varnish listener
+
+* set
+  [Varnish runtime parameters and tunables](https://varnish-cache.org/docs/6.1/reference/varnishd.html)
+  such as the cache size, default TTLs and timeouts, thread pool
+  dimensions, workspace sizes (and many more)
+
+* change the container configuration from default values, for example
+  to set non-default port numbers, or a non-default mount path for the
+  admin Secret
+
+The [``examples/`` folder](/examples/varnish_pod_templates) has
+working examples that demonstrate such configurations. The present
+document describes what may, and what may not be customized.
+
+## varnishd Command-Line options
+
+See
+[``varnishd(1)``](https://varnish-cache.org/docs/6.1/reference/varnishd.html#options)
+for details about available options.
+
+```
+        args:
+        # varnishd command-line options
+        # In this example:
+        # varnishd -s malloc,256m -t 900 -p workspace_client=256k
+        - -s
+        - malloc,256m
+        - -t
+        - "900"
+        - -p
+        - workspace_client=256k
+```
+
+Because of the fact that the container starts with a number of options
+in order to implement the role of an Ingress, there are restrictions
+on the options that you can or should set. Some of them result in
+illegal combinations of options, which causes varnishd to terminate
+and the container to crash.  Others will be ignored, since the Varnish
+instance is managed by the controller.  Still others may interfere
+with the operation as an Ingress.
+
+Among these restrictions are:
+
+* You MAY NOT use any of the ``-C``, ``-d``, ``-I``, ``-S``, ``-T``,
+  ``-V``, ``-x`` or ``-?`` options.
+
+* The ``-p vcl_path`` parameter MAY NOT be changed.
+
+* ``-b`` or ``-f`` SHOULD NOT be set, since they will be ignored (but
+  their use does not cause an error).
+
+* ``-a`` CAN be set to define more listener ports for regular HTTP
+  client traffic; to be useful, these must be declared in the
+  ``ports`` specification (as with the port named ``http`` in the
+  pod template). The listener name ``vk8s`` MAY NOT be used (it is
+  reserved for the listener used for readiness checks).
+
+* ``-M`` CAN be set, so that Varnish will connect to an address
+  listening for the administrative interface. The controller will not
+  use that address, but an admin client can use it to monitor the
+  Varnish instance separately. But an admin client MAY NOT call
+  ``vcl.use`` to activate any configuration, or ``vcl.discard`` to
+  unload one, otherwise it interferes with the implementation of
+  Ingress.
+
+See the [``examples/`` folder](/examples/varnish_pod_template) for a
+[working manifest](/examples/varnish_pod_template/cli-args.yaml) that
+sets command-line options.
+
+## Environment variables
+
+These environment variables can be used to change the configuration
+from defaults:
+
+* ``PROTO``: sets the
+  [PROTO sub-argument](https://varnish-cache.org/docs/6.1/reference/varnishd.html#basic-options)
+  for the HTTP listener. Legal values are ``HTTP`` or ``PROXY``,
+  default ``HTTP``.
+
+* ``HTTP_PORT``: sets the container port for the HTTP listener,
+  default 80.
+
+* ``READY_PORT``: sets the container port for the listener for
+  readiness checks, default 8080.
+
+* ``ADMIN_PORT``: sets the port at which Varnish listens for
+  [CLI commands](https://varnish-cache.org/docs/6.1/reference/varnish-cli.html),
+  used by the controller; default 6081.
+
+* ``SECRET_PATH``: sets the path mounted to the volume that is
+  populated with the admin Secret; default ``/var/run/varnish``.
+
+* ``SECRET_FILE``: sets the basename of the file in which the admin
+  Secret is stored, default ``_.secret``.
+
+For example:
+
+```
+        env:
+        # PROTO=PROXY causes the listener at the http port to accept
+        # the PROXY protocol (v1 or v2).
+        # see: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
+        - name: PROTO
+          value: PROXY
+
+        # Container port for the HTTP listener.
+        # MUST match the value set for the http containerPort in the
+        # Pod template, and the http targetPort for the Service.
+        - name: HTTP_PORT
+          value: "81"
+
+        # Container port for the HTTP readiness check.
+        # MUST match the value set for the k8s containerPort in the
+        # Pod template.
+        - name: READY_PORT
+          value: "8000"
+```
+
+As indicated in the example, the values set for some of the
+environment variables must match values in the configuration of other
+elements:
+
+* ``HTTP_PORT``: MUST match the value in the Pod template's
+  ``.ports[n].containerPort`` field for the port number of the
+  HTTP listener (named ``http`` in the
+  [Deployment example](/deploy/varnish.yaml)). MUST also match the
+  ``targetPort`` defined in the Service configuration for Varnish
+  (also named ``http`` in the [Nodeport example](/deploy/nodeport.yaml)).
+
+* ``READY_PORT``: MUST match ``.ports[n].containerPort`` for the port
+  used in the http readiness check (``.readinessProbe.httpGet.port``).
+
+* ``ADMIN_PORT``: MUST match ``.ports[n].containerPort`` for the admin
+  listener (named ``varnishadm`` in the
+  [Deployment example](/deploy/varnish.yaml)). MUST also match the
+  ``targetPort`` defined in the Service configuration for Varnish
+  (also ``varnishadm`` in the [Nodeport example](/deploy/nodeport.yaml)).
+
+* ``SECRET_PATH``: MUST match the value of ``.volumeMounts.mountPath``
+  in the Pod template's configuration of the volume mounted to inject
+  the admin Secret.
+
+* ``SECRET_FILE``: MUST match ``.volumes.secret.items[n].path`` in
+  the Pod template's specfication of the file basename for the admin
+  Secret.
+
+See the [``examples/`` folder](/examples/varnish_pod_template) for a
+[manifest](/examples/varnish_pod_template/proxy.yaml) that just turns
+on the PROXY protocol, and
+[another one](/examples/varnish_pod_template/env.yaml) that sets all
+of the environment variables.

examples/README.md

@@ -10,6 +10,9 @@ requirements.
 * Limiting the Ingress controller to
   [a single namespace](/examples/namespace).
 
+* [Customizing the Pod template](/examples/varnish_pod_template)
+  for Varnish
+
 * [Self-sharding Varnish cluster](/examples/self-sharding)
   ([docs](/docs/self-sharding.md))
 

examples/varnish_pod_template/README.md

+# Customizing the Varnish Pod template
+
+The sample manifests in this folder set values in the ``args`` or
+``env`` section of the Pod template for Varnish, to modify the
+configuration from defaults. See the
+[``docs/`` folder](/docs/varnish-pod-template.md) for details and
+requirements.
+
+As in the [example](/deploy/varnish.yaml) from the
+[deployment instructions](/deploy), each of these define a Deployment,
+but the configuration does not depend on that -- Pod templates are
+also used for other types of controllers, such as DaemonSets.
+
+[``cli-args.yaml``](cli-args.yaml) sets varnishd command-line options:
+
+```
+        args:
+        # varnishd command-line options
+        # In this example:
+        # varnishd -s malloc,256m -t 900 -p workspace_client=256k
+        - -s
+        - malloc,256m
+        - -t
+        - "900"
+        - -p
+        - workspace_client=256k
+```
+
+See
+[``varnishd(1)``](https://varnish-cache.org/docs/6.1/reference/varnishd.html#options)
+for details.
+
+[``proxy.yaml``](proxy.yaml) just sets the ``PROTO`` environment
+variable to activate the
+[PROXY protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)
+(likely a common use case):
+
+```
+        env:
+        # PROTO=PROXY causes the listener at the http port to accept
+        # the PROXY protocol (v1 or v2).
+        - name: PROTO
+          value: PROXY
+```
+
+[``env.yaml``](env.yaml) sets all of the relevant environment
+variables.  It also defines the Service for Varnish, since the values
+of variables that set port numbers (``HTTP_PORT`` and ``ADMIN_PORT``)
+must match ``targetPort`` values set for the Service:
+
+```
+apiVersion: extensions/v1beta1
+kind: Deployment
+[...]
+spec:
+[...]
+  template:
+    [...]
+    spec:
+      containers:
+      - image: varnish-ingress/varnish
+        [...]
+        env:
+        [...]
+
+        # Container port for the HTTP listener.
+        # MUST match the http targetPort in the Service below.
+        - name: HTTP_PORT
+          value: "81"
+
+        [...]
+
+        # Container port for the admin listener.
+        # MUST match the varnishadm targetPort in the Service below.
+        - name: ADMIN_PORT
+          value: "7000"
+[...]
+---
+apiVersion: v1
+kind: Service
+[...]
+spec:
+  type: NodePort 
+  ports:
+  - port: 7000
+    targetPort: 7000
+    protocol: TCP
+    name: varnishadm
+  - port: 81
+    targetPort: 81
+    protocol: TCP
+    name: http
+[...]
+
+```

examples/varnish_pod_template/cli-args.yaml

+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: varnish
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: varnish-ingress
+  template:
+    metadata:
+      labels:
+        app: varnish-ingress
+    spec:
+      containers:
+      - image: varnish-ingress/varnish
+        imagePullPolicy: IfNotPresent
+        name: varnish-ingress
+        ports:
+        - name: http
+          containerPort: 80
+        - name: k8s
+          containerPort: 8080
+        - name: varnishadm
+          containerPort: 6081
+        volumeMounts:
+        - name: adm-secret
+          mountPath: "/var/run/varnish"
+          readOnly: true
+        livenessProbe:
+          exec:
+            command:
+            - /usr/bin/pgrep
+            - -P
+            - "0"
+            - varnishd
+        readinessProbe:
+          httpGet:
+            path: /ready
+            port: k8s
+        args:
+        # varnishd command-line options
+        # In this example:
+        # varnishd -s malloc,256m -t 900 -p workspace_client=256k
+        - -s
+        - malloc,256m
+        - -t
+        - "900"
+        - -p
+        - workspace_client=256k
+      volumes:
+      - name: adm-secret
+        secret:
+          secretName: adm-secret
+          items:
+          - key: admin
+            path: _.secret

examples/varnish_pod_template/env.yaml

+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: varnish
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: varnish-ingress
+  template:
+    metadata:
+      labels:
+        app: varnish-ingress
+    spec:
+      containers:
+      - image: varnish-ingress/varnish
+        imagePullPolicy: IfNotPresent
+        name: varnish-ingress
+        ports:
+        - name: http
+          containerPort: 80
+        - name: k8s
+          containerPort: 8000
+        - name: varnishadm
+          containerPort: 7000
+        volumeMounts:
+        - name: adm-secret
+          mountPath: "/var/secret"
+          readOnly: true
+        livenessProbe:
+          exec:
+            command:
+            - /usr/bin/pgrep
+            - -P
+            - "0"
+            - varnishd
+        readinessProbe:
+          httpGet:
+            path: /ready
+            port: k8s
+        env:
+        # Use the PROXY protocol (cf. proxy.yaml).
+        - name: PROTO
+          value: PROXY
+
+        # Container port for the HTTP listener.
+        # MUST match the value set for the http containerPort above, and
+        # the http targetPort in the Service below.
+        - name: HTTP_PORT
+          value: "81"
+
+        # Container port for the HTTP readiness check.
+        # MUST match the value set for the k8s containerPort above.
+        - name: READY_PORT
+          value: "8000"
+
+        # Container port for the admin listener.
+        # MUST match the value set for the varnishadm containerPort above,
+        # and the varnishadm targetPort in the Service below.
+        - name: ADMIN_PORT
+          value: "7000"
+
+        # Path at which the volume for the admin secret is mounted.
+        # MUST match the value of mountPath in volumeMounts above.
+        - name: SECRET_PATH
+          value: /var/secret
+
+        # Name of the file containing the admin secret.
+        # MUST match the value of path in volumes below.
+        - name: SECRET_FILE
+          value: adm.secret
+
+      volumes:
+      - name: adm-secret
+        secret:
+          secretName: adm-secret
+          items:
+          - key: admin
+            path: adm.secret
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: varnish-ingress
+  labels:
+    app: varnish-ingress
+  annotations:
+    service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+spec:
+  type: NodePort 
+  ports:
+  - port: 7000
+    targetPort: 7000
+    protocol: TCP
+    name: varnishadm
+  - port: 81
+    targetPort: 81
+    protocol: TCP
+    name: http
+  selector:
+    app: varnish-ingress
+  publishNotReadyAddresses: true

examples/varnish_pod_template/proxy.yaml

+apiVersion: extensions/v1beta1
+kind: Deployment
+metadata:
+  name: varnish
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: varnish-ingress
+  template:
+    metadata:
+      labels:
+        app: varnish-ingress
+    spec:
+      containers:
+      - image: varnish-ingress/varnish
+        imagePullPolicy: IfNotPresent
+        name: varnish-ingress
+        ports:
+        - name: http
+          containerPort: 80
+        - name: k8s
+          containerPort: 8080
+        - name: varnishadm
+          containerPort: 6081
+        volumeMounts:
+        - name: adm-secret
+          mountPath: "/var/run/varnish"
+          readOnly: true
+        livenessProbe:
+          exec:
+            command:
+            - /usr/bin/pgrep
+            - -P
+            - "0"
+            - varnishd
+        readinessProbe:
+          httpGet:
+            path: /ready
+            port: k8s
+        env:
+        # PROTO=PROXY causes the listener at the http port to accept
+        # the PROXY protocol (v1 or v2).
+        # see: https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
+        - name: PROTO
+          value: PROXY
+      volumes:
+      - name: adm-secret
+        secret:
+          secretName: adm-secret
+          items:
+          - key: admin
+            path: _.secret