README.rst 39.2 KB


access the libgcrypt cryptographic library

Manual section: 3


import gcrypt [from "path"] ;


new OBJECT = gcrypt.symmetric(ENUM cipher, ENUM mode, ENUM padding,
                              BLOB key [, BOOL secure]
                              [, BOOL cbc_cts])
BLOB <OBJ>.encrypt(BLOB plaintext [, BLOB iv] [, BLOB ctr])
BLOB <OBJ>.decrypt(BLOB ciphertext [, BLOB iv] [, BLOB ctr])

BLOB gcrypt.fileread(STRING path)

BLOB gcrypt.random([ENUM quality] [, BYTES n])
INT gcrypt.random_int(ENUM quality [, INT bound])
REAL gcrypt.random_real(ENUM)
BOOL random_bool(ENUM)


STRING gcrypt.version()
STRING gcrypt.gcrypt_version()


This Varnish Module (VMOD) provides access to the libgcrypt library of cryptographic building blocks -- the same library used by the GNU Privacy Guard cryptographic suite (GnuPG or GPG). The VMOD currently supports:

  • symmetric encryption with AES, and with the standard modes of operation
  • generation of pseudo-random data

The VMOD uses the VCL data type BLOB for data that enter into encryption operations -- plaintext, ciphertext, initialization vectors and counter vectors. BLOBs are arbitrary regions of memory; this allows the VMOD to concentrate strictly on cryptographic operations, separate from concerns of binary-to-text encodings. BLOBs are not created by any part of native VCL, and can only be created by other VMODs, so it is necessary to use this VMOD together with another one that does so (such as VMOD blobcode for binary-to-text encodings, see SEE ALSO).

This is a simple usage example:

import gcrypt;
import blobcode;

sub vcl_init {
    # Finalize default initialization of the libgcrypt library.
    # This must be called once during the lifetime of the Varnish
    # child process, as explained below.

    # Create an object for AES-128 in CTR mode using the key just
    # created, and with libgcrypt internal structures in secure memory.
    # The key is read from the given file.
    new aes = gcrypt.symmetric(AES, CTR, secure=true,

# Assume that a plaintext to be encrypted is in the response
# header X-Msg. Assign the hex-encoded encrypted message to the
# response header X-Cipher, and the hex-encoded counter vector to
# the response header X-Ctr; and remove X-Msg from the response.
sub vcl_deliver {
    # Use the blobcode VMOD to convert the contents of X-Msg to a
    # BLOB, and to encode the encrypted ciphertext in hex with
    # lower-case digits. Use the random() function to generate a
    # counter vector as a 128 bit nonce.
    set resp.http.X-Cipher
        = blobcode.encode(HEXLC,
                        ctr=gcrypt.random(NONCE, 16B)));

    # Use the no-argument version of random() to retrieve the
    # counter vector that was just generated, and use the
    # blobcode VMOD to encode it as lower-case hex.
    set resp.http.X-CTR = blobcode.encode(HEXLC, gcrypt.random());

    # Remove the plaintext from the response header.
    unset resp.http.X-Msg;

libgcrypt secure memory

libgcrypt supports a notion of "secure memory" for storing sensitive data, which is used by the library internally when enabled. What this entails is platform-dependent, but on most platforms supported by Varnish:

  • The secure memory pool is locked (with mlock(2)), so that it will not be paged to swap space.
  • The memory pool is allocated in pages that cannot be read by other processes.
  • Memory obtained from the pool is overwritten after use.
  • On some platforms, privileges/capabilities of the running process are lowered when secure memory is initialized. This applies to the varnishd child process.

The use of secure memory is turned off by default for the creation of new objects. With the VMOD, you can specify the use of secure memory by setting the secure flag to true in a constructor for symmetric encryption, and configure the size of the pool, or disable secure memory, using the init() function, as described below.

libgcrypt logging

The libgcrypt library emits log messages that the VMOD directs to the Varnish shared memory log. These messages are logged with the pseudo XID 0, which is only visible when varnishlog(1) is used with raw grouping (command-line option -g raw) -- they do not appear with default grouping. libgcrypt log levels ERROR, FATAL and BUG appear in the Varnish log with the tag Error; all other libgcrypt log messages appear with the tag Debug.

libgcrypt fatal errors

The libgcrypt library has a fatal error handler that causes the process to abort, and hence stops the Varnish child process. The VMOD endeavors to prevent this from happening (with error handling as described below), but if it does:

  • The error message and numeric error code are written to the Varnish log with XID 0 and the Error tag.
  • A Varnish panic is invoked with the error message from libgcrypt.


  • BLOB fileread(PRIV_TASK, STRING)
  • INT random_int(ENUM {STRONG,NONCE}, INT)
  • REAL random_real(ENUM {STRONG,NONCE})
  • BOOL random_bool(ENUM {STRONG,NONCE})
  • VOID wipe(BLOB)
  • STRING version()
  • STRING gcrypt_version()



Initialize the libgcrypt library, currently to manage the use of secure memory. The ENUM specifies an operation for initialization.

Initialization takes place exactly once during the lifetime of the Varnish child process, and must be performed before any objects are created; details below.

With INIT_SECMEM, you can configure the size of the secure memory pool to n bytes (the n parameter is ignored for the other ENUMs). The data type for n is BYTES, so the value must be written with a suffix such as B or KB. Secure memory is enabled by default and set to a default size (32 KiB in libgcrypt 1.6.3), so you don't have to call init() with INIT_SECMEM to use the default.

Setting n to 0B with INIT_SECMEM disables secure memory, and hence has the same effect as calling init(DISABLE_SECMEM). If secure memory is enabled, libgcrypt imposes a minimum size for the pool (16 KiB for libgcrypt 1.6.3), so any value of n that is smaller than the minimum will result in the minimum allocation. Since the default value of n is 1B, you can specify the minimum size by calling init(INIT_SECMEM) without the n parameter.

DISABLE_SECMEM disables secure memory; when secure memory is disabled (either with init(DISABLE_SECMEM) or init(INIT_SECMEM, 0B)), any attempt to create an object with the secure flag results in failure. If secure memory is enabled and then disabled before init() is called with FINISH, or vice versa, the last setting holds.

FINISH indicates that initialization is finalized, and must be called before any of the VMOD's objects can be created. The simplest initialization is simply to call init(FINISH), which results in the default configuration for the libgcrypt library -- secure memory is enabled with the default pool size.

The init() function may only be called in vcl_init; if it is called in any other VCL subroutine, then an error message is logged with the tag VCL_Error, and the call is ignored.

Initialization is evaluated only once during the lifetime of the Varnish child process. If a new instance of VCL is loaded with calls to init() in vcl_init after initialization was already performed for another instance, then an error message is logged (with XID 0 and the Debug tag), and the calls are ignored (but the VCL reload does not fail). This is true even if the prior VCL instance is unloaded with the CLI command vcl.discard. This means in particular that the configuration of secure memory cannot be changed at runtime; for that, you have to restart the Varnish child process, with a new instance of VCL that changes the configuration.

If initialization has been finalized in a previously loaded VCL instance, then a new VCL instance can be loaded that does not include any init() calls and uses the VMOD; in that case, the prior initialization continues to hold. But the simplest strategy is probably to test a configuration that works for your deployment in the long run, and invoke that configuration in every VCL instance that you load at runtime. After the first load, subsequent invocations of init() are ignored, but are harmless.


import gcrypt;

sub vcl_init {
    # Default initialization -- secure memory is enabled with
    # the default size for the pool.

# Note that changed initializations are only effective after a
# restart of the Varnish child process, as explained above.

sub vcl_init {
    # Enable secure memory and allocate a 64KiB pool.
    gcrypt.init(INIT_SECMEM, 64KB);

sub vcl_init {
    # Disable secure memory.


new OBJ = symmetric(ENUM {AES,AES128,RIJNDAEL,RIJNDAEL128,AES192,RIJNDAEL192,AES256,RIJNDAEL256} cipher, ENUM {ECB,CFB,CBC,OFB,CTR} mode, ENUM {PKCS7,ISO7816,X923,NONE} padding="PKCS7", BLOB key, BOOL secure=0, BOOL cbc_cts=0)

Create an object for encryption and decryption with symmetric ciphers. Currently, only AES is supported, with key lengths 128, 192 and 256.

The cipher ENUM specifies the cryptographic algorithm to be used:

  • AES, AES128, RIJNDAEL, RIJNDAEL128: AES-128 (the ENUMs are aliases for one another)
  • AES192, RIJNDAEL192: AES-192
  • AES256, RIJNDAEL256: AES-256

The mode ENUM specifies the mode of operation.

The padding ENUM specifies the padding method, for modes of operation that require it. Padding is required for ECB and CBC; except that padding is not required for CBC with ciphertext stealing, when the cbc_cts flag is set to true. If the mode of operation does not require padding, then the value of padding is ignored.

  • PKCS7: PKCS#7 padding (sometimes called PKCS#5). A padding byte is always added to the message, and the values of all padding bytes are the number of padding bytes added. This is the default value of padding.
  • ISO7816: ISO/IEC 7816-4:2005 padding. Byte 0x80 is always added to the message, and the remaining padding bytes are set to 0x00.
  • X923: ANSI X.923 padding. A padding byte is always added to the message, and the value of the last byte is the number of padding bytes added. The remaining padding bytes are set to 0x00.
  • NONE: no padding. This can be used if you are certain that the length of all messages to be encrypted or decrypted will always be an exact multiple of the cipher's block length, and the other party also uses no padding.

The key BLOB is the cryptographic key to be used. Its size MUST be correct for the cipher that is to be used; for example, 16 bytes for AES-128.

If the secure flag is true, then secure memory is used by the internal libgcrypt structures created for the object. False by default.

If the cbc_cts flag is true and CBC mode is specified, then CBC is used with ciphertext stealing (and padding is not required). cbc_cts is ignored for all other modes of operation. Default is false.

The creation of a symmetric object fails, leading to a failure of the VCL load with an error message, under these conditions:

  • libgcrypt initialization has not been finalized (init(FINISH) has never been called).
  • key is NULL, for example due to failure to create a BLOB object.
  • The size of the key BLOB is larger than the longest key length supported for cipher.
  • The secure flag is true, but secure memory has been disabled.
  • Any failure of the libgcrypt library to initialize internal structures to be used for this object. This happens, for example, if the key length is to short for the chosen cipher.


import gcrypt;
import blobcode;

# Assume in the following that initialization has been finalized.

sub vcl_init {
    # Use the blobcode VMOD to create some BLOBs for the cryptographic
    # keys, whose lengths are 16, 24 and 32 bytes for AES-128, -192
    # and -256, respectively.
    # NOTE: The keys used in this manual's examples are chosen to
    # make the key lengths easy to recognize. DO NOT copy them
    # into production VCL!
    new k128 = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
    new k192 = blobcode.blob(HEX,
    new k256 = blobcode.blob(HEX,

    # Create an object for AES-128 with CTR mode (no padding
    # required), not using secure memory by default.
    new aes128 = gcrypt.symmetric(AES128, CTR, key=k128.get());

    # Create an object for AES-192 with CBC mode and PKCS#7
    # padding, using secure memory.
    new aes192 = gcrypt.symmetric(AES192, CBC, PKCS7, key=k192.get(),

    # Create an object for AES-256 with CBC mode and ciphertext
    # stealing (no padding required), using secure memory.
    new aes256 = gcrypt.symmetric(AES256, CBC, key=k256.get(),
                                  cbc_cts=true, secure=true);


BLOB symmetric.encrypt(BLOB plaintext, BLOB iv=0, BLOB ctr=0)

Encrypt the contents of plaintext according to the parameters specified for this symmetric object, and return the ciphertext as a BLOB. The BLOB plaintext MAY NOT be NULL.

The iv and ctr parameters specify initialization and counter vectors, respectively, and MUST be non-NULL for modes of operation that require them. They MUST also be of the size required for the cipher in use for the symmetric object. An initialization vector is required for CFB, CBC and OFB modes, and a counter vector is required for CTR. iv and ctr are ignored for modes of operation that do not require them, and can be left out in those cases.

The method fails under the following conditions:

  • There is insufficient workspace for the BLOB returned as the ciphertext, or for the padded plaintext, if padding is necessary.
  • The iv or ctr parameters are NULL for modes of operation that require init or counter vectors.
  • Any failure of the libgcrypt library -- setting the init or counter vectors, or encrypting the message.

If the method fails, a message is logged using the VCL_Error tag, and NULL is returned.


# In the following assume:
# - The aes128, aes192 and aes256 objects from the previous examples.
# - The messages to be encrypted are in the response headers
#   X-Msg-128, X-Msg-192 and X-Msg-256.
# - An appropriately sized base64-encoded counter vector is in
#   the response header X-Ctr.
# - Appropriately sized base64-encoded init vectors are in the
#   response headers X-IV-192 and X-IV-256.
sub vcl_deliver {
    # Encrypt X-Msg-128 with AES-128 and the counter vector, and
    # return the ciphertext in a base64-encoded response header.
    set resp.http.X-Cipher-128 = blobcode.encode(BASE64,
                       ctr=blobcode.decode(BASE64, resp.http.X-Ctr)));

    # Encrypt X-Msg-192 with AES-192 and init vector X-IV-192.
    set resp.http.X-Cipher-192 = blobcode.encode(BASE64,
                       iv=blobcode.decode(BASE64, resp.http.X-IV-192)));

    # Encrypt X-Msg-256 with AES-256 and init vector X-IV-256.
    set resp.http.X-Cipher-256 = blobcode.encode(BASE64,
                       iv=blobcode.decode(BASE64, resp.http.X-IV-256)));


BLOB symmetric.decrypt(BLOB ciphertext, BLOB iv=0, BLOB ctr=0)

Decrypt the contents of ciphertext according to the parameters specified for this symmetric object, and return the plaintext as a BLOB. The BLOB ciphertext MAY NOT be NULL.

As with the .encrypt() method, the iv and ctr parameters specify initialization and counter vectors, and MUST be specified for modes of operation that require them, but may be left out otherwise.

The failure conditions of .decrypt() are similar to those for .encrypt():

  • Insufficient workspace for the ciphertext to be returned.
  • Missing iv or ctr parameters for modes of operation that require them.
  • An error in the padding bytes, if padding is specified. For example, if the number of padding bytes added is longer than the cipher's block length (for PKCS7 and X923 padding), or if a value such as 0x80 or 0x00 is not found where the padding scheme requires it.
  • Failure of the libgcrypt library.

If the method fails, a message is logged using the VCL_Error tag, and NULL is returned.


# Assume the aes128, aes192 and aes256 objects from the previous
# examples. Also assume the messages to be decrypted are in the
# headers X-Msg-128, X-Msg-192 and X-Msg-256, as well as the headers
# X-Ctr, X-IV-192 and X-IV-256 as described for the .encrypt()
# example above, except that they are all request headers.
sub vcl_recv {
    # Decrypt X-Msg-128 with AES-128 and the counter vector, and
    # return the plaintext in a base64-encoded request header.
    set req.http.X-Cipher-128 = blobcode.encode(BASE64,
                       ctr=blobcode.decode(BASE64, req.http.X-Ctr)));

    # Decrypt X-Msg-192 with AES-192 and init vector X-IV-192.
    set req.http.X-Cipher-192 = blobcode.encode(BASE64,
                       iv=blobcode.decode(BASE64, req.http.X-IV-192)));

    # Decrypt X-Msg-256 with AES-256 and init vector X-IV-256.
    set req.http.X-Cipher-256 = blobcode.encode(BASE64,
                       iv=blobcode.decode(BASE64, req.http.X-IV-256)));


BLOB fileread(PRIV_TASK, STRING path)

Return the contents of the file at path in a BLOB. This function is provided for the specific purpose of reading sensitive data that are needed during vcl_init, such as cryptographic keys, so that they are not exposed in the VCL source.

If secure memory is enabled, then space for the BLOB contents is allocated from the secure memory pool; otherwise, space is allocated from the heap. The contents are retained in memory for the duration of the current task scope; this means:

  • If fileread() is called in vcl_init (or vcl_fini), then the scope ends when the execution of that subroutine ends.
  • Otherwise (in any other VCL subroutine), the scope ends when the current client or backend transaction ends.

For example, if fileread() is called during one of the vcl_backend_* subroutines, then the scope ends when the current backend transaction is complete.

When the task scope ends, then the BLOB contents are overwritten in memory, and the allocated memory is freed. If the contents were allocated from the secure memory pool, then that space is returned to the pool.

fileread() fails and returns NULL if:

  • libgcrypt initialization is not finished
  • path is NULL
  • path does not denote a regular file. For example, path MAY NOT be a directory, symbolic link or named pipe.
  • path cannot be inspected by stat(2), opened, read or closed, for example if the file does not exist, or if the Varnish child process has insufficient privileges to read it.
  • path is changed at some time between just after invocation of fileread() and just after reading its contents.
  • There is insufficient space for the necessary allocations, for example if the secure memory pool is to be used but does not have enough free memory.

If fileread() fails, it writes an error message to the Varnish log with the tag VCL_Error. If it fails during vcl_init, then the VCL load fails with the error message.

fileread() does not do any binary-to-text conversion, so the contents of the file should be the raw binary data that you intend to read into the BLOB. The file should not, for example, contain a hex or base64 encoding.

If you intend to read sensitive data from a file, consider setting the file's permissions so that only the owner of the Varnish child process can read it. Secure management of the file becomes part of your administrative responsibility that is outside the scope of Varnish and this VMOD.

fileread() can be used to read data from a file in any VCL subroutine, and the use case does not have to be related to cryptography, but it is very inefficient. It reads from file and allocates memory on every invocation, and that memory is wiped and freed at the end of every task scope in which it is invoked, all of which can have a severe impact on performance. See the fileread() funciton of the std VMOD (vmod_std(3)) for an efficient way to read from files.


# Write 16 bytes of binary data to a file.
$ echo 'AAECAwQFBgcICQoLDA0ODw==' | base64 -d > /path/to/key

# Make the file readable only to the Varnish worker process owner.
$ chown vcache:varnish /path/to/key
$ chmod 400 /path/to/key

# In VCL, use the file contents as an encryption key.
sub vcl_init {
    new aes128 = gcrypt.symmetric(AES128, CTR,



Return a BLOB containing n bytes of pseudo-random data. The cryptographic strength of random number generation is determined by the quality ENUM:

  • NONCE (for "number used once"): Pseudo-random bytes for applications where cryptographic security is not required. This level is suitable, for example, for generation of initialization or counter vectors when used with modes of operation that require uniqueness, but do not have strong requirements for unpredictabilty.
  • STRONG: From the libgcrypt manual: "Use this level for session keys and similar purposes". Most applications requiring cryptographically secure pseudo-random data should use this level.
  • VERY_STRONG: From the libgcrypt manual: "Use this level for long term key material".

NOTE: At the VERY_STRONG level, the random() function may take several minutes to execute. This level should not be used when performance is important; STRONG is sufficient for crypto-quality randomness. If at all, VERY_STRONG might be used in a service that occasionally generates new keys, on the understanding that this will take quite some time (and it may be necessary to set long timeouts).

Consider using the NONCE level for applications that do not require strong randomness. With NONCE, performance is better, nothing can be revealed about the internal state of the strong random number generator, and entropy mixed into the strong generator is not consumed.

If n is 0B, then random() returns the same BLOB that was returned by its most recent invocation in the same client or backend context. Since 0B is the default value of n, it can be left out for this purpose, and the quality ENUM can be left out as well. In other words, calling random() with no arguments returns the same BLOB that was returned with arguments in the same client or backend scope.

This makes it possible, for example, to use random() with appropriate arguments to generate an initialization or counter vector in an encryption operation, and then call random() again with no arguments to assign the same value to a header to be sent along with the encrypted message, as illustrated below.

The random() function fails if:

  • n is 0B, but random() was not previously called with n > 0B in the same client or backend context.
  • n is greater than 0B, but the quality ENUM is not set.
  • There is insufficient workspace for the BLOB to be returned.

If random() fails in vcl_init, then the VCL load fails with an error message. If it fails in any other subroutine, then it returns NULL, and an error message is written to the log with the VCL_Error tag.


# Assume the aes192 object shown in the examples above: AES-192
# encryption with CBC mode.

# Encrypt the contents of the X-Msg response header, using the
# random() function to generate an initialization vector, which
# is sent in the reponse header X-IV.

sub vcl_resp {
    # The length of the IV MUST match the block size of the
    # cipher in use -- 128 bits (16 bytes) for AES. For CBC, the IV
    # MUST be unpredictable, so we use quality level STRONG.
    set resp.http.X-Encrypted
        = blobcode.encode(BASE64,
                           iv=gcrypt.random(STRONG, 16B)));

    # Now call random() with no arguments to retrive the IV that
    # was generated, to be sent in the base64-encoded response
    # header X-IV.
    set resp.http.X-IV = blobcode.encode(BASE64, gcrypt.random());

    # Remove the plaintext from the response header.
    unset resp.http.X-Msg;


INT random_int(ENUM {STRONG,NONCE} quality, INT bound=0)

Returns a random integer, using the random number generator with the quality level specified by the ENUM quality. These are the same levels that are described for the random() function above, except that the VERY_STRONG level is not available.

If bound is less than or equal to 0, then the return value lies within the entire possible range for an INT (positive or negative). The default value of bound is 0.

If bound is greater than 0, then the return value lies between 0 (inclusive) and bound (exclusive). To get a random integer between values min and max inclusive with the NONCE generator, use this formula:

gcrypt.random_int(NONCE, (max - min) + 1) + min


# Assign a random group number from 0 to 99 to a request.
set req.http.X-Group = gcrypt.random_int(NONCE, 100);



Returns a random REAL that is greater than or equal to 0.0 and less than 1.0, using the random number generator with the specified quality level. The permitted levels are NONCE and STRONG as described above.

To get a random REAL in the range from min to max inclusive with the STRONG generator, use this formula:

gcrypt.random_real(STRONG) * (max - min) + min

Note that when a REAL is converted to a STRING, for example when it is assigned to a header, the representation may be rounded up to "1.0" if the REAL's value is very close to 1.0. So it may appear as if the range of random_real() includes 1.0, but this is just an artifact of string conversion.


# Assign an unpredictable REAL from -1.0 to 1.0 to a request.
set req.http.X-Real = gcrypt.random_real(STRONG) * 2 - 1;



Returns a random boolean using the randomness generator with the specified quality level. The permitted levels are NONCE and STRONG as described above.

This function is more efficient than, for example, calling random_int() with the bound set to 2 and then checking if the result is 0 or 1 (or something similar with random_real()), since random_bool() calls the randomness generators less often (caching random bytes as bitmaps), and only uses one bit of randomness for each invocation.


# Choose from two courses of action unpredictably.
if (gcrypt.random_bool(STRONG)) {
    call do_this;
else {
    call do_that;



Overwrites the memory region denoted by the BLOB, leaving it with all zeroes.

The BLOB MUST be non-empty. If wipe() is called with an empty BLOB, then an error message is written to the log with the VCL_Error tag. If this happens in vcl_init, then the VCL load fails with the error message.


# After setting a symmetric encryption key, which will be stored in
# secure memory, wipe the BLOB from which the key was read.  This
# ensures that the key is only stored in secure memory.
sub vcl_init {
    new k = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
    new aes = gcrypt.symmetric(AES, CTR, key=k.get(), secure=true);


STRING version()

Returns the version string for this VMOD.


std.log("Using VMOD gcrypt version " + gcrypt.version());


STRING gcrypt_version()

Returns the version string for the libgcrypt library with which the VMOD is linked.


std.log("Using libgcrypt version " + gcrypt.gcrypt_version());


This VMOD requires Varnish version 5.1 or later, and libgcrypt version 1.6.3 or later. It has been tested successfully against libgcrypt versions 1.6.3 through 1.7.7, except for version 1.7.4. (Version 1.7.4 had a bug and was replaced by 1.7.5 soon after its release.)


As noted above, the VMOD uses Varnish workspace for the BLOB objects returned by the .encrypt() and .decrypt() methods, and for padding a plaintext before encryption, if necessary. If you find that these methods are failing, with VCL_Error messages "out of space" in the Varnish log, then increase the varnishd parameters workspace_client and/or workspace_backend.

If you choose to use the libgcrypt secure memory feature, then the size of the pool that is needed depends on the number of VMOD objects that you create and the number of threads in which encryption operations are performed. For each object in each thread, an internal structure for libgcrypt is created the first time an encryption or decryption is attempted; the same structure is then re-used for all other operations in the same thread. These structures are deallocated when the thread exits.

If secure memory has been specified for the VMOD object, then these internal structures are allocated from the secure memory pool. So you may need to configure a large pool if you are running many worker threads to handle a heavy load for Varnish, and more so if you are using many objects.

Recall that the number of worker threads run by Varnish is controlled by the varnishd parameters thread_pools, thread_pool_min and thread_pool_max; this in turn affects how much secure memory will need to be available for each object in each thread. thread_pools determines how many thread pools there are, and for each pool, at least thread_pool_min threads will always be started. More threads may be started to respond to increasing load, up to the maximum given by thread_pool_max for each thread. As load decreases (that is, when threads become idle for time given by thread_pool_timeout), threads will be stopped until the minimum thread_pool_min is reached.

The VMOD uses pthread keys (see pthread_key_create(3)) to locate the thread-specific data for a symmetric encryption object. A key must be created for each object in each active VCL instance, and the number of keys that can be created is subject to the limit defined by PTHREAD_KEYS_MAX (1024 for current Linux versions); so this limits the number of symmetric objects that can exist in active VCL instances. When an object is de-allocated, its key is deleted, and then no longer counts against the maximum.

Note that when a VCL instance is unloaded at runtime (so as to load a new instance), the objects allocated for the unloaded instance are not de-allocated until that instance enters the cold state, by default after 5 minutes in Varnish 5.1 (determined by the varnishd parameter vcl_cooldown); so the number of pthread keys accumulates during this time. You can bring about an earlier de-allocation, and hence reduce the use of the pthread keys, by shortening the vcl_cooldown period, or by using the CLI command vcl.discard to explicitly remove previous VCL instances.


If you intend to use the VMOD to protect sensitive data, it is critical that you have a firm understanding of how to use the cryptographic primitives that it makes available, and of potential vulnerabilities of the tools you employ -- libgcrypt, Varnish, VMOD gcrypt and any other VMODs you deploy.


Both libgcrypt and Varnish have strong security records. Nevertheless, you should stay informed of any security fixes that are released for both packages, and install them promptly when they are. Please inform us (at the developer contact given in SEE ALSO) if you encounter any difficulties using the VMOD with new versions of libgcrypt or Varnish.

VMOD gcrypt, in contrast, is much newer, and has not established a security record. We welcome any bug reports, bug fixes, reviews or other feedback you may have. And it would help to let us know if you are using the VMOD successfully.

Secure and non-secure memory

The secure memory feature of the libgcrypt library gives a certain measure of protection against exposure of sensitive data, but at present that is limited to internal structures used by the library (mainly, it protects the keys stored in those structures). Moreover, the secure memory pool is visible in the address space of the Varnish child process.

VMOD gcrypt, by itself, is neutral as to the sources of data that enter into cryptographic operations -- keys, plaintexts, ciphertexts, IVs and counters. But these are typically obtained from sources that are readable from other elements of Varnish.

As can be seen in the examples above that use VMOD blobcode to create key objects, the contents of a key are readable in the VCL source. If you are using the VMOD that way, consider storing VCL sources so that they are only readable by the owner of the Varnish child process.

Ciphertexts and plaintexts are typically obtained from sources that are stored in Varnish workspaces, for example when they are read from headers, as in the examples above. And the ciphertexts and plaintexts produced by the VMOD on encryption and decryption are stored in workspaces. Workspaces are re-used for the lifetime of a worker thread. They tend to be overwritten, but this does not happen automatically when a workspace is re-used. Nothing prevents another VMOD from reading the workspace used by VMOD gcrypt.

Note that if any such data is stored in the cache, for example in response headers of cached responses, then they are saved for the lifetime of the cached object. If file storage is used for the cache, then that data can be read from the cache file.

All of the data just described, except for the contents of the libgcrypt secure memory pool, could conceivably be paged into swap space, and all of it can be written to a core dump file.

Therefore, if you are working with sensitive data, consider the following measures:

  • Test your Varnish deployment thoroughly with test data, until you are confident that it does not swap or dump core.
  • Then in production, use the means of your operating system to ensure that the Varnish child process does not swap or write core files.
  • Examine any other VMODs that you use to ensure that they do not leak data from workspaces.
  • Consider using the fileread() function to read sensitive data such as cryptographic keys from files that are only readable by the Varnish worker process. If you must expose sensitive data in VCL sources, ensure that those sources are only readable by the worker process.
  • If you are using file storage for the cache, ensure that the cache file is only readable and writable by the child process.

Cryptographic good practice

Finally, make sure that you are using the cryptographic primitives properly and safely. libgcrypt provides cryptographic building blocks, and the VMOD makes some of these available in VCL. But neither the library nor the VMOD can prevent you from using those building blocks improperly, thus undermining the security of your application.

The proper use of cryptography is a subject that is beyond the scope of this manual; when in doubt, consult an expert. We want to emphasize the following points, but this list is by no means exhaustive:

  • The ECB mode of operation is included for testing purposes, but should never be used to protect sensitive data in insecure environments.
  • It is important to fulfill the uniqueness and predictability requirements for the initialization and counter vectors used for symmetric encryption, which depend in part on the mode of operation. IVs and counters MUST be unique in all cases; they must never be re-used with the same encryption key. For CBC mode, the IV must also be unpredictable; for example, the STRONG quality level should be used if the random() function is used to generate IVs for CBC. For the other modes, unpredictability is not required, and the NONCE level is sufficient.
  • libgcrypt permits certain operations that are poor practice, such as setting an init vector whose length is not equal to the block length of a symmetric cipher, but emits a warning message. The VMOD follows the library in allowing these operations to proceed, but you should check the Varnish log for the warnings, and correct any such problems.
  • Make sure that you have a secure procedure in place for generating and storing cryptographic keys, and for changing the keys periodically.


See INSTALL.rst in the source repository.



This document is licensed under the same conditions
as the libvmod-gcrypt project. See LICENSE for details.

Author: Geoffrey Simmons <>