The replica scaling test for self-sharding requires the Deployment
name "varnish-ingress" for the viking service.

This is the SNI sent in the client TLS connection to a backend.

We use VMOD dynamic for backends represented by an ExternalName
Service (likely the common use case for TLS onload). VMOD dynamic
does not have the authority field that klarlack makes available
for standard backends. But if the host_header field is set for
a VMOD dynamic director, the VMOD uses that value for the SNI.

So if the BackendConfig authority field is set, we also assign its
value to the host_header field. Since BackendConfig also has a
separate field for host_header, both of them could be conceivably
set to different values. If we find that the two fields are set
to non-empty, conflicting values, the controller emits a
SyncFatalError, and the BackendConfig is not synced.

This uses haproxy for TLS connections to IngressBackends, and the
via feature of the klarlack implementation of Varnish. See:

https://github.com/varnishcache/varnish-cache/pull/3128

Adds the spec.tls object to the BackendConfig CRD, which configures
TLS onload for a backend.

Limitations: currently only verify:false and the maxConn settings
are implemented. Specification of CA certificates and the stick
table configuration for haproxy are not yet implemented. Currently
TLS onload may be only specified for one backend (no more than one
BackendConfig).

Adds the CLI option -varnishImpl to the controller. TLS onload is
only supported if this option is set to "klarlack". Otherwise, the
presence of the tls object in a BackendConfig leads to a SyncFatalError,
with a message that it's only supported for klarlack, and the
BackendConfig is not synced.

If the backend Service specified for TLS onload has type ExternalName,
then 3 server instances are configured for the haproxy backend. This
value is currently hard-wired, and may be made configurable in a future
iteration. For any other Service type, there are as many haproxy server
instances as there are Endpoints (Pods) in the k8s cluster.

If maxConn is not specified in the BackendConfig, it defaults to
2000 (the haproxy default).

See c3f7a9a9

If changes did not trigger a klarlack build, gitlab CI would file this
complaint:

  Found errors in your .gitlab-ci.yml:

    'build:ascn' job needs 'build:klarlack' job
    but 'build:klarlack' is not in any previous stage

despite the fact that the ci file lints just fine.

IIUC, the issue is that for the triggered pipeline, the klarlack build job
just would not exist.

In this case, we can build ascn on top of the previous klarlack build.

Does this help?

because of:

Step 5/20 : RUN go get -d -v github.com/slimhazard/gogitversion &&     cd /go/src/github.com/slimhazard/gogitversion &&     make install
 ---> Running in 8a50294f4b5c
github.com/slimhazard/gogitversion (download)
godoc -html . | pandoc -f html -t markdown_github > README.md
/bin/sh: 1: pandoc: not found
/bin/sh: 1: godoc: not found
make: *** [Makefile:36: README.md] Error 127

see
- e6645a38
- c488a3b1

Dependencies: did not do what I expected it to

I hope this works

Here we are going to move the custom software install (watermarker etc) which we need
for the primary sponsor.

For now, the new container is just the same as the klarlack image

fixes a typo: We accidentally tagged the klarlack image as varnish

For this it is necessary to abandon go get to download k8s-crt-dnldr,
and use git clone && make instead. The reasons for that are detailed
in commit message fe1fc4f of the k8s-crt-dnldr repo.

Only issue a log warning in this case. Verified with an e2e test.

Closes #44

Allows custom vcl_recv to be executed when requests are forwarded
to the primary.

The self-sharding field in VarnishConfig now has a mandatory field
rules. This is an array of at least one object with a mandatory
field shard and an optional field conditions.

spec.self-sharding.rules[n].shard configures sharding in the fields
key, digest and primaryOnly (all optional).

spec.self-sharding.rules[n].conditions, if specified, is an array
of at least one condition, which specifies a boolean expression.
It is the same as conditions used for other contexts such as
rewrites.

The rules array is evaluated in order as an if-elsif-else sequence.
This means that if any element of the array specifies no conditions,
it MUST be the last rule in the array. In an array of length > 1,
it becomes the else case. This constraint is trivially satisfied
if there is only one rule. It is currently *not* enforced (would be
suitable for a webhook validation).

The fields probe and max2ndttl are now top-level fields under
self-sharding, both optional.

Closes #42

Currently only implemented to kubectl-style deployments.
Helm support to be added in the next commits.

This is a breaking change for the self-sharding configuration.
Configs previously expressed in the self-sharding object are
now moved to self-sharding.shard. In addition, the optional
array self-sharding.conditions may express conditions under
which self-sharding logic is executed.

The conditions array has the same syntax and semantics as
for req-disposition.

XXX: there is some DRY that needs to be refactored away.

- We currently have two internal means of expressing
  conditions under which something happens (translated
  to if-clauses with matching operations and the like in
  VCL): Condition and the legacy MatchTerm, in both the
  k8s VarnishConfig class and the spec type for VCL
  templating. These should be unified to Condition.

- The code for interpreting Conditions for the k8s object
  in pkg/controller is in part repeated for self-sharding
  and req-disposition. These should be encapsulated in a
  common function.

- Template generation for the if-clauses implementing
  conditions is repeated in req-dispostiion and currently
  in two places for self-sharding. This should be a
  common text/template object.

While here, move some of the code about self-sharding and
conditions from varnishconfig.go to ingress.go. Because
it's actually about the VarnishConfig Custom Resource, and
in a continuing effort to reduce the oversized ingress.go
source.