Skip to content

L
libvmod-gcrypt
Commit 361f4fe9 authored by Geoff Simmons's avatar Geoff Simmons
Browse files
Options

Documentation

parent 3bd10f59
Pipeline #174 skipped
Hide whitespace changes
Inline Side-by-side
Showing with 1062 additions and 8 deletions
README.rst
View file @ 361f4fe9
...@@ -26,14 +26,126 @@ import gcrypt [from "path"] ; ...@@ -26,14 +26,126 @@ import gcrypt [from "path"] ;
:: ::
new OBJECT = ... gcrypt.init(ENUM {INIT_SECMEM, DISABLE_SECMEM} [, INT n])
gcrypt.init(FINISH)
new OBJECT = gcrypt.symmetric(ENUM cipher, ENUM mode, ENUM padding,
BLOB key [, BOOL secure]
[, BOOL cbc_cts])
<OBJ>.encrypt(BLOB plaintext [, BLOB iv] [, BLOB ctr])
<OBJ>.decrypt(BLOB plaintext [, BLOB iv] [, BLOB ctr])
gcrypt.version()
gcrypt.gcrypt_version()
DESCRIPTION DESCRIPTION
=========== ===========
This Varnish Module (VMOD) provides access to the libgcrypt library of This Varnish Module (VMOD) provides access to the libgcrypt library of
cryptographic building blocks -- the same library used by the GNU cryptographic building blocks -- the same library used by the GNU
Privacy Guard cryptographic suite (GnuPG or GPG). Privacy Guard cryptographic suite (GnuPG or GPG). The VMOD currently
only supports symmetric encryption with AES, and with the standard
modes of operation.
The VMOD uses the VCL data type BLOB for data that enter into
encryption operations -- plaintext, ciphertext, initialization vectors
and counter vectors. BLOBs are arbitrary regions of memory; this
allows the VMOD to concentrate strictly on cryptographic operations,
separate from concerns of binary-to-text encodings. BLOBs are not
created by any part of native VCL, and can only be created by other
VMODs, so it is necessary to use this VMOD together with another one
that does so (such as VMOD ``blobcode`` for binary-to-text encodings,
see `SEE ALSO`_).
This is a simple usage example::
import gcrypt;
import blobcode;
sub vcl_init {
# Finalize default initialization of the libgcrypt library.
# This must be called once during the lifetime of the Varnish
# child process, as explained below.
gcrypt.init(FINISH);
# Use the blobcode VMOD to create an object that stores a BLOB
# derived from this hex encoding, to be used as the cryptographic
# key. You must ensure that keys and other cryptographic
# elements have the right sizes -- in this case, a 128 bit key
# for the AES-128 cipher.
# NOTE: The keys used in this manual's examples are chosen to
# make the key lengths easy to recognize. DO NOT copy them
# into production VCL!
new k = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
# Create an object for AES-128 in CTR mode with PKCS#7 padding,
# using the key just created, and with libgcrypt internal
# structures in secure memory.
new aes = gcrypt.symmetric(AES, CTR, PKCS7, key=k.get(),
secure=true);
}
# Assume that a ciphertext to be decrypted is in the request
# header X-Msg, and the counter vector is in X-CTR, both
# hex-encoded.
sub vcl_recv {
# Use the blobcode VMOD to decode the two headers, and to
# encode the decrypted plaintext in hex with lower-case
# digits, to be assigned to the request header X-Plain.
set req.http.X-Plain
= blobcode.encode(HEXLC,
aes.decrypt(blobcode.decode(HEX, req.http.X-Msg),
ctr=blobcode.decode(HEX, req.http.X-CTR)));
}
libgcrypt secure memory
-----------------------
libgcrypt supports a notion of "secure memory" for storing sensitive
data, which is used by the library internally when enabled. What this
entails is platform-dependent, but on most platforms supported by
Varnish:
* The secure memory pool is locked (with mlock(2)), so that it will
not be paged to swap space.
* The memory pool is allocated in pages that cannot be read by other
processes.
* Memory obtained from the pool is overwritten after use.
* On some platforms, privileges/capabilities of the running process
are lowered when secure memory is initialized. This applies to the
varnishd child process.
The use of secure memory is enabled by default. With the VMOD, you can
specify the use of secure memory with the ``secure`` flag, and
configure the size of the pool, or disable secure memory, using the
``init()`` function, as described below.
libgcrypt logging
-----------------
The libgcrypt library emits log messages that the VMOD directs to the
Varnish shared memory log. These messages are logged with the pseudo
XID 0, which is only visible when ``varnishlog(1)`` is used with raw
grouping (command-line option ``-g raw``) -- they do not appear with
default grouping. libgcrypt log levels ERROR, FATAL and BUG appear in
the Varnish log with the tag ``Error``; all other libgcrypt log
messages appear with the tag ``Debug``.
libgcrypt fatal errors
----------------------
The libgcrypt library has a fatal error handler that causes the
process to abort, and hence stops the Varnish child process. The VMOD
endeavors to prevent this from happening (with error handling as
described below), but if it does:
* The error message and numeric error code are written to the Varnish
log with XID 0 and the ``Error`` tag.
* A Varnish panic is invoked with the error message from libgcrypt.
CONTENTS CONTENTS
======== ========
...@@ -52,6 +164,89 @@ init ...@@ -52,6 +164,89 @@ init
VOID init(ENUM {INIT_SECMEM,DISABLE_SECMEM,FINISH}, INT n=1) VOID init(ENUM {INIT_SECMEM,DISABLE_SECMEM,FINISH}, INT n=1)
Initialize the libgcrypt library, currently to manage the use of
secure memory. The ENUM specifies an operation for initialization.
Initialization takes place *exactly once* during the lifetime of the
Varnish child process, and must be performed before any objects are
created; details below.
With ``INIT_SECMEM``, you can configure the size of the secure memory
pool to ``n`` bytes (the ``n`` parameter is ignored for the other
ENUMs). Secure memory is enabled by default and set to a default size
(32 KiB in libgcrypt 1.6.3), so you don't have to call ``init()`` with
``INIT_SECMEM`` to use the default.
Setting ``n`` to 0 with ``INIT_SECMEM`` disables secure memory, and
hence has the same effect as calling ``init(DISABLE_SECMEM)``. If
secure memory is enabled, libgcrypt imposes a minimum size for the
pool (16 KiB for libgcrypt 1.6.3), so any value of ``n`` that is
smaller than the minimum will result in the minimum allocation. Since
the default value of ``n`` is 1, you can specify the minimum size by
calling ``init(INIT_SECMEM)`` without the ``n`` parameter.
``DISABLE_SECMEM`` disables secure memory; when secure memory is
disabled (either with ``init(DISABLE_SECMEM)`` or ``init(INIT_SECMEM,
0)``), any attempt to create an object with the ``secure`` flag
results in failure. If secure memory is enabled and then disabled
before ``init()`` is called with ``FINISH``, or vice versa, the last
setting holds.
``FINISH`` indicates that initialization is finalized, and must be
called before any of the VMOD's objects can be created. The simplest
initialization is simply to call ``init(FINISH)``, which results in
the default configuration for the libgcrypt library -- secure memory
is enabled with the default pool size.
The ``init()`` function may only be called in ``vcl_init``; if it is
called in any other VCL subroutine, then an error message is logged
with the tag ``VCL_Error``, and the call is ignored.
Initialization is evaluated only once during the lifetime of the
Varnish child process. If a new instance of VCL is loaded with calls
to ``init()`` in ``vcl_init`` after initialization was already
performed for another instance, then an error message is logged (with
XID 0 and the ``Debug`` tag), and the calls are ignored (but the VCL
reload does not fail). This is true even if the prior VCL instance is
unloaded with the CLI command ``vcl.discard``. This means in
particular that the configuration of secure memory cannot be changed
at runtime; for that, you have to restart the Varnish child process,
with a new instance of VCL that changes the configuration.
If initialization has been finalized in a previously loaded VCL
instance, then a new VCL instance can be loaded that does not include
any ``init()`` calls and uses the VMOD; in that case, the prior
initialization continues to hold. But the simplest strategy is
probably to test a configuration that works for your deployment in the
long run, and invoke that configuration in every VCL instance that you
load at runtime. After the first load, subsequent invocations of
``init()`` are ignored, but are harmless.
Examples::
import gcrypt;
sub vcl_init {
# Default initialization -- secure memory is enabled with
# the default size for the pool.
gcrypt.init(FINISH);
}
# Note that changed initializations are only effective after a
# restart of the Varnish child process, as explained above.
sub vcl_init {
# Enable secure memory and allocate a 64KiB pool.
gcrypt.init(INIT_SECMEM, 65536);
gcrypt.init(FINISH);
}
sub vcl_init {
# Disable secure memory.
gcrypt.init(DISABLE_SECMEM);
gcrypt.init(FINISH);
}
.. _obj_symmetric: .. _obj_symmetric:
symmetric symmetric
...@@ -61,6 +256,108 @@ symmetric ...@@ -61,6 +256,108 @@ symmetric
new OBJ = symmetric(ENUM {AES,AES128,RIJNDAEL,RIJNDAEL128,AES192,RIJNDAEL192,AES256,RIJNDAEL256} cipher, ENUM {ECB,CFB,CBC,OFB,CTR} mode, ENUM {PKCS7,ISO7816,X923,NONE} padding="PKCS7", BLOB key, BOOL secure=1, BOOL cbc_cts=0) new OBJ = symmetric(ENUM {AES,AES128,RIJNDAEL,RIJNDAEL128,AES192,RIJNDAEL192,AES256,RIJNDAEL256} cipher, ENUM {ECB,CFB,CBC,OFB,CTR} mode, ENUM {PKCS7,ISO7816,X923,NONE} padding="PKCS7", BLOB key, BOOL secure=1, BOOL cbc_cts=0)
Create an object for encryption and decryption with symmetric ciphers.
Currently, only AES is supported, with key lengths 128, 192 and 256.
The ``cipher`` ENUM specifies the cryptographic algorithm to be used:
* ``AES``, ``AES128``, ``RIJNDAEL``, ``RIJNDAEL128``: AES-128 (the ENUMs
are aliases for one another)
* ``AES192``, ``RIJNDAEL192``: AES-192
* ``AES256``, ``RIJNDAEL256``: AES-256
The ``mode`` ENUM specifies the mode of operation.
The ``padding`` ENUM specifies the padding method, for modes of
operation that require it. Padding is required for ``ECB`` and
``CBC``; except that padding is not required for ``CBC`` with
ciphertext stealing, when the ``cbc_cts`` flag is set to ``true``. If
the mode of operation does not require padding, then the value of
``padding`` is ignored.
* ``PKCS7``: PKCS#7 padding (sometimes called PKCS#5). A padding byte
is always added to the message, and the values of all padding bytes
are the number of padding bytes added. This is the default value of
``padding``.
* ``ISO7816``: ISO/IEC 7816-4:2005 padding. Byte 0x80 is always added
to the message, and the remaining padding bytes are set to 0x00.
* ``X923``: ANSI X.923 padding. A padding byte is always added to the
message, and the value of the last byte is the number of padding
bytes added. The remaining padding bytes are set to 0x00.
* ``NONE``: no padding. This can be used if you are certain that the
length of all messages to be encrypted or decrypted will always be
an exact multiple of the cipher's block length, and the other party
also uses no padding.
The ``key`` BLOB is the cryptographic key to be used. Its size MUST
be correct for the cipher that is to be used; for example, 16 bytes
for AES-128.
If the ``secure`` flag is true, then secure memory is used by the
internal libgcrypt structures created for the object. True by default.
If the ``cbc_cts`` flag is true and ``CBC`` mode is specified, then
CBC is used with ciphertext stealing (and padding is not required).
``cbc_cts`` is ignored for all other modes of operation. Default is
``false``.
The creation of a ``symmetric`` object fails, leading to a failure
of the VCL load with an error message, under these conditions:
* libgcrypt initialization has not been finalized (``init(FINISH)``
has never been called).
* ``key`` is NULL, for example due to failure to create a BLOB
object.
* The size of the ``key`` BLOB is larger than the longest key length
supported for ``cipher``.
* The ``secure`` flag is ``true``, but secure memory has been
disabled.
* Any failure of the libgcrypt library to initialize internal
structures to be used for this object.
Examples::
import gcrypt;
import blobcode;
# Assume in the following that initialization has been finalized.
sub vcl_init {
# Create some BLOBs for the cryptographic keys, whose lengths
# are 16, 24 and 32 bytes for AES-128, -192 and -256,
# respectively.
# As emphasized above, DON'T copy these keys into production
# VCL!
new k128 = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
new k192 = blobcode.blob(HEX,
"000102030405060708090a0b0c0d0e0f1011121314151617");
new k256 = blobcode.blob(HEX,
"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f");
# Create an object for AES-128 with CTR mode (no padding
# required), using secure memory by default.
new aes128 = gcrypt.symmetric(AES128, CTR, key=k128.get());
# Create an object for AES-192 with CBC mode and PKCS#7
# padding, not using secure memory.
new aes192 = gcrypt.symmetric(AES192, CBC, PKCS7, key=k192.get(),
secure=false);
# Create an object for AES-256 with CBC mode and ciphertext
# stealing (no padding required), using secure memory.
new aes256 = gcrypt.symmetric(AES256, CBC, key=k256.get(),
cbc_cts=true);
}
.. _func_symmetric.encrypt: .. _func_symmetric.encrypt:
symmetric.encrypt symmetric.encrypt
...@@ -70,6 +367,61 @@ symmetric.encrypt ...@@ -70,6 +367,61 @@ symmetric.encrypt
BLOB symmetric.encrypt(BLOB plaintext, BLOB iv=0, BLOB ctr=0) BLOB symmetric.encrypt(BLOB plaintext, BLOB iv=0, BLOB ctr=0)
Encrypt the contents of ``plaintext`` according to the parameters
specified for this ``symmetric`` object, and return the ciphertext as
a BLOB. The BLOB ``plaintext`` MAY NOT be NULL.
The ``iv`` and ``ctr`` parameters specify initialization and counter
vectors, respectively, and MUST be non-NULL for modes of operation
that require them. They MUST also be of the size required for the
cipher in use for the ``symmetric`` object. An initialization vector
is required for CFB, CBC and OFB modes, and a counter vector is
required for CTR. ``iv`` and ``ctr`` are ignored for modes of
operation that do not require them, and can be left out in those
cases.
The method fails under the following conditions:
* There is insufficient workspace for the BLOB returned as the
ciphertext, or for the padded plaintext, if padding is necessary.
* The ``iv`` or ``ctr`` parameters are NULL for modes of operation
that require init or counter vectors.
* Any failure of the libgcrypt library -- setting the init or counter
vectors, or encrypting the message.
If the method fails, a message is logged using the ``VCL_Error`` tag,
and NULL is returned.
Examples::
# In the following assume:
# - The aes128, aes192 and aes256 objects from the previous examples.
# - The messages to be encrypted are in the response headers
# X-Msg-128, X-Msg-192 and X-Msg-256.
# - An appropriately sized base64-encoded counter vector is in
# the response header X-Ctr.
# - Appropriately sized base64-encoded init vectors are in the
# response headers X-IV-192 and X-IV-256.
sub vcl_deliver {
# Encrypt X-Msg-128 with AES-128 and the counter vector, and
# return the ciphertext in a base64-encoded response header.
set resp.http.X-Cipher-128 = blobcode.encode(BASE64,
aes128.encrypt(blobcode.decode(encoded=resp.http.X-Msg-128),
ctr=blobcode.decode(BASE64, resp.http.X-Ctr)));
# Encrypt X-Msg-192 with AES-192 and init vector X-IV-192.
set resp.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.encrypt(blobcode.decode(encoded=resp.http.X-Msg-192),
iv=blobcode.decode(BASE64, resp.http.X-IV-192)));
# Encrypt X-Msg-256 with AES-256 and init vector X-IV-256.
set resp.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.encrypt(blobcode.decode(encoded=resp.http.X-Msg-192),
iv=blobcode.decode(BASE64, resp.http.X-IV-192)));
}
.. _func_symmetric.decrypt: .. _func_symmetric.decrypt:
symmetric.decrypt symmetric.decrypt
...@@ -79,6 +431,52 @@ symmetric.decrypt ...@@ -79,6 +431,52 @@ symmetric.decrypt
BLOB symmetric.decrypt(BLOB ciphertext, BLOB iv=0, BLOB ctr=0) BLOB symmetric.decrypt(BLOB ciphertext, BLOB iv=0, BLOB ctr=0)
Decrypt the contents of ``ciphertext`` according to the parameters
specified for this ``symmetric`` object, and return the plaintext as
a BLOB. The BLOB ``ciphertext`` MAY NOT be NULL.
As with the ``.encrypt()`` method, the ``iv`` and ``ctr`` parameters
specify initialization and counter vectors, and MUST be specified for
modes of operation that require them, but may be left out otherwise.
The failure conditions of ``.decrypt()`` are similar to those for
``.encrypt()``:
* Insufficient workspace for the ciphertext to be returned.
* Missing ``iv`` or ``ctr`` parameters for modes of operation that
require them.
* Failure of the libgcrypt library.
If the method fails, a message is logged using the ``VCL_Error`` tag,
and NULL is returned.
Examples::
# Assume the aes128, aes192 and aes256 objects from the previous
# examples. Also assume the messages to be decrypted are in the
# headers X-Msg-128, X-Msg-192 and X-Msg-256, as well as the headers
# X-Ctr, X-IV-192 and X-IV-256 as described for the .encrypt()
# example above, except that they are all request headers.
sub vcl_recv {
# Decrypt X-Msg-128 with AES-128 and the counter vector, and
# return the plaintext in a base64-encoded request header.
set req.http.X-Cipher-128 = blobcode.encode(BASE64,
aes128.decrypt(blobcode.decode(encoded=req.http.X-Msg-128),
ctr=blobcode.decode(BASE64, req.http.X-Ctr)));
# Decrypt X-Msg-192 with AES-192 and init vector X-IV-192.
set req.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.decrypt(blobcode.decode(encoded=req.http.X-Msg-192),
iv=blobcode.decode(BASE64, req.http.X-IV-192)));
# Decrypt X-Msg-256 with AES-256 and init vector X-IV-256.
set req.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.decrypt(blobcode.decode(encoded=req.http.X-Msg-192),
iv=blobcode.decode(BASE64, req.http.X-IV-192)));
}
.. _func_version: .. _func_version:
version version
...@@ -113,12 +511,136 @@ Example:: ...@@ -113,12 +511,136 @@ Example::
REQUIREMENTS REQUIREMENTS
============ ============
This VMOD requires Varnish ... This VMOD requires Varnish version 5.1 or later, and libgcrypt version
1.6.3 or later.
LIMITATIONS LIMITATIONS
=========== ===========
... As noted above, the VMOD uses Varnish workspace for the BLOB objects
returned by the ``.encrypt()`` and ``.decrypt()`` methods, and for
padding a plaintext before encryption, if necessary. If you find that
these methods are failing, with ``VCL_Error`` messages "out of space"
in the Varnish log, then increase the varnishd parameters
``workspace_client`` and/or ``workspace_backend``.
If you choose to use the libgcrypt secure memory feature, then the
size of the pool that is needed depends on the number of VMOD objects
that you create. The default size should be sufficient for typical
use cases, but you may need to configure a larger pool if you are
using many objects.
Note that when a VCL instance is unloaded at runtime (so as to load a
new instance), the objects allocated for the unloaded instance are not
de-allocated until that instance enters the cold state, by default
after 5 minutes in Varnish 5.1 (determined by the varnishd parameter
``vcl_cooldown``); so the use of the secure memory pool accumulates
during this time. You can bring about an earlier de-allocation, and
hence reduce the use of the secure memory pool, by shortening the
``vcl_cooldown`` period, or by using the CLI command ``vcl.discard``
to explicitly remove previous VCL instances.
SECURITY
========
If you intend to use the VMOD to protect sensitive data, it is
critical that you have a firm understanding of how to use the
cryptographic primitives that it makes available, and of potential
vulnerabilities of the tools you employ -- libgcrypt, Varnish, VMOD
gcrypt and any other VMODs you deploy.
Software
--------
Both libgcrypt and Varnish have strong security records. Nevertheless,
you should stay informed of any security fixes that are released for
both packages, and install them promptly when they are. Please inform
us (at the developer contact given in `SEE ALSO`_) if you encounter
any difficulties using the VMOD with new versions of libgcrypt or
Varnish.
VMOD gcrypt, in contrast, is much newer, and has not established a
security record. We welcome any bug reports, bug fixes, reviews or
other feedback you may have. And it would help to let us know if
you are using the VMOD successfully.
Secure and non-secure memory
----------------------------
The secure memory feature of the libgcrypt library gives a certain
measure of protection against exposure of sensitive data, but at
present that is limited to internal structures used by the library
(mainly, it protects the keys stored in those structures). Moreover,
the secure memory pool is visible in the address space of the Varnish
child process.
VMOD gcrypt, by itself, is neutral as to the sources of data that
enter into cryptographic operations -- keys, plaintexts, ciphertexts,
IVs and counters. But these are typically obtained from sources that
are readable from other elements of Varnish.
As can be seen in the examples above that use VMOD blobcode to create
key objects, the contents of a key are readable in the VCL source. If
you are using the VMOD that way, consider storing VCL sources so that
they are only readable by the owner of the Varnish child process.
Ciphertexts and plaintexts are typically obtained from sources that
are stored in Varnish workspaces, for example when they are read from
headers, as in the examples above. And the ciphertexts and plaintexts
produced by the VMOD on encryption and decryption are stored in
workspaces. Workspaces are re-used for the lifetime of a worker
thread. They tend to be overwritten, but this does not happen
automatically when a workspace is re-used. Nothing prevents another
VMOD from reading the workspace used by VMOD gcrypt.
Note that if any such data is stored in the cache, for example in
response headers of cached responses, then they are saved for the
lifetime of the cached object. If file storage is used for the cache,
then that data can be read from the cache file.
All of the data just described, *except* for the contents of the
libgcrypt secure memory pool, could conceivably be paged into swap
space, and all of it can be written to a core dump file.
Therefore, if you are working with sensitive data, consider the
following measures:
* Test your Varnish deployment thoroughly with test data, until you
are confident that it does not to swap or dump core.
* Then in production, use the means of your operating system to ensure
that the Varnish child process does not swap or write core files.
* Examine any other VMODs that you use to ensure that they do not leak
data from workspaces.
* Ensure that VCL sources are only readable by the Varnish child
process.
* If you are using file storage for the cache, ensure that the cache
file is only readable and writable by the child process.
Cryptographic good practice
---------------------------
Finally, make sure that you are using the cryptographic primitives
properly and safely. That subject is beyond the scope of this manual;
when in doubt, consult an expert. We want to emphasize the following
points, but this list is by no means exhaustive:
* The ECB mode of operation is included for testing purposes, but
should *never* be used to protect sensitive data in insecure
environments.
* Initialization and counter vectors can be sent in the clear, and do
not need to generated in a secure way; for example, they do not need
to be obtained from cryptographic random number generators. But an
IV or counter should never be used twice with the same key, so make
sure that they are generated uniquely.
* Make sure that you have a secure procedure in place for generating
and storing cryptographic keys, and for changing the keys
periodically.
INSTALLATION INSTALLATION
============ ============
...@@ -130,7 +652,12 @@ SEE ALSO ...@@ -130,7 +652,12 @@ SEE ALSO
* varnishd(1) * varnishd(1)
* vcl(7) * vcl(7)
* varnishlog(1)
* libgcrypt: https://gnupg.org/software/libgcrypt/index.html
* source repository: https://code.uplex.de/uplex-varnish/libvmod-gcrypt * source repository: https://code.uplex.de/uplex-varnish/libvmod-gcrypt
* VMOD blobcode: https://code.uplex.de/uplex-varnish/libvmod-blobcode
* developer contact: <varnish-support@uplex.de>, and at the source
repository site
COPYRIGHT COPYRIGHT
========= =========
......
src/vmod_gcrypt.vcc
View file @ 361f4fe9
...@@ -9,26 +9,424 @@ $Module gcrypt 3 access the libgcrypt cryptographic library ...@@ -9,26 +9,424 @@ $Module gcrypt 3 access the libgcrypt cryptographic library
:: ::
new OBJECT = ... gcrypt.init(ENUM {INIT_SECMEM, DISABLE_SECMEM} [, INT n])
gcrypt.init(FINISH)
new OBJECT = gcrypt.symmetric(ENUM cipher, ENUM mode, ENUM padding,
BLOB key [, BOOL secure]
[, BOOL cbc_cts])
<OBJ>.encrypt(BLOB plaintext [, BLOB iv] [, BLOB ctr])
<OBJ>.decrypt(BLOB plaintext [, BLOB iv] [, BLOB ctr])
gcrypt.version()
gcrypt.gcrypt_version()
DESCRIPTION DESCRIPTION
=========== ===========
This Varnish Module (VMOD) provides access to the libgcrypt library of This Varnish Module (VMOD) provides access to the libgcrypt library of
cryptographic building blocks -- the same library used by the GNU cryptographic building blocks -- the same library used by the GNU
Privacy Guard cryptographic suite (GnuPG or GPG). Privacy Guard cryptographic suite (GnuPG or GPG). The VMOD currently
only supports symmetric encryption with AES, and with the standard
modes of operation.
The VMOD uses the VCL data type BLOB for data that enter into
encryption operations -- plaintext, ciphertext, initialization vectors
and counter vectors. BLOBs are arbitrary regions of memory; this
allows the VMOD to concentrate strictly on cryptographic operations,
separate from concerns of binary-to-text encodings. BLOBs are not
created by any part of native VCL, and can only be created by other
VMODs, so it is necessary to use this VMOD together with another one
that does so (such as VMOD ``blobcode`` for binary-to-text encodings,
see `SEE ALSO`_).
This is a simple usage example::
import gcrypt;
import blobcode;
sub vcl_init {
# Finalize default initialization of the libgcrypt library.
# This must be called once during the lifetime of the Varnish
# child process, as explained below.
gcrypt.init(FINISH);
# Use the blobcode VMOD to create an object that stores a BLOB
# derived from this hex encoding, to be used as the cryptographic
# key. You must ensure that keys and other cryptographic
# elements have the right sizes -- in this case, a 128 bit key
# for the AES-128 cipher.
# NOTE: The keys used in this manual's examples are chosen to
# make the key lengths easy to recognize. DO NOT copy them
# into production VCL!
new k = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
# Create an object for AES-128 in CTR mode with PKCS#7 padding,
# using the key just created, and with libgcrypt internal
# structures in secure memory.
new aes = gcrypt.symmetric(AES, CTR, PKCS7, key=k.get(),
secure=true);
}
# Assume that a ciphertext to be decrypted is in the request
# header X-Msg, and the counter vector is in X-CTR, both
# hex-encoded.
sub vcl_recv {
# Use the blobcode VMOD to decode the two headers, and to
# encode the decrypted plaintext in hex with lower-case
# digits, to be assigned to the request header X-Plain.
set req.http.X-Plain
= blobcode.encode(HEXLC,
aes.decrypt(blobcode.decode(HEX, req.http.X-Msg),
ctr=blobcode.decode(HEX, req.http.X-CTR)));
}
libgcrypt secure memory
-----------------------
libgcrypt supports a notion of "secure memory" for storing sensitive
data, which is used by the library internally when enabled. What this
entails is platform-dependent, but on most platforms supported by
Varnish:
* The secure memory pool is locked (with mlock(2)), so that it will
not be paged to swap space.
* The memory pool is allocated in pages that cannot be read by other
processes.
* Memory obtained from the pool is overwritten after use.
* On some platforms, privileges/capabilities of the running process
are lowered when secure memory is initialized. This applies to the
varnishd child process.
The use of secure memory is enabled by default. With the VMOD, you can
specify the use of secure memory with the ``secure`` flag, and
configure the size of the pool, or disable secure memory, using the
``init()`` function, as described below.
libgcrypt logging
-----------------
The libgcrypt library emits log messages that the VMOD directs to the
Varnish shared memory log. These messages are logged with the pseudo
XID 0, which is only visible when ``varnishlog(1)`` is used with raw
grouping (command-line option ``-g raw``) -- they do not appear with
default grouping. libgcrypt log levels ERROR, FATAL and BUG appear in
the Varnish log with the tag ``Error``; all other libgcrypt log
messages appear with the tag ``Debug``.
libgcrypt fatal errors
----------------------
The libgcrypt library has a fatal error handler that causes the
process to abort, and hence stops the Varnish child process. The VMOD
endeavors to prevent this from happening (with error handling as
described below), but if it does:
* The error message and numeric error code are written to the Varnish
log with XID 0 and the ``Error`` tag.
* A Varnish panic is invoked with the error message from libgcrypt.
$Function VOID init(ENUM {INIT_SECMEM, DISABLE_SECMEM, FINISH}, INT n=1) $Function VOID init(ENUM {INIT_SECMEM, DISABLE_SECMEM, FINISH}, INT n=1)
Initialize the libgcrypt library, currently to manage the use of
secure memory. The ENUM specifies an operation for initialization.
Initialization takes place *exactly once* during the lifetime of the
Varnish child process, and must be performed before any objects are
created; details below.
With ``INIT_SECMEM``, you can configure the size of the secure memory
pool to ``n`` bytes (the ``n`` parameter is ignored for the other
ENUMs). Secure memory is enabled by default and set to a default size
(32 KiB in libgcrypt 1.6.3), so you don't have to call ``init()`` with
``INIT_SECMEM`` to use the default.
Setting ``n`` to 0 with ``INIT_SECMEM`` disables secure memory, and
hence has the same effect as calling ``init(DISABLE_SECMEM)``. If
secure memory is enabled, libgcrypt imposes a minimum size for the
pool (16 KiB for libgcrypt 1.6.3), so any value of ``n`` that is
smaller than the minimum will result in the minimum allocation. Since
the default value of ``n`` is 1, you can specify the minimum size by
calling ``init(INIT_SECMEM)`` without the ``n`` parameter.
``DISABLE_SECMEM`` disables secure memory; when secure memory is
disabled (either with ``init(DISABLE_SECMEM)`` or ``init(INIT_SECMEM,
0)``), any attempt to create an object with the ``secure`` flag
results in failure. If secure memory is enabled and then disabled
before ``init()`` is called with ``FINISH``, or vice versa, the last
setting holds.
``FINISH`` indicates that initialization is finalized, and must be
called before any of the VMOD's objects can be created. The simplest
initialization is simply to call ``init(FINISH)``, which results in
the default configuration for the libgcrypt library -- secure memory
is enabled with the default pool size.
The ``init()`` function may only be called in ``vcl_init``; if it is
called in any other VCL subroutine, then an error message is logged
with the tag ``VCL_Error``, and the call is ignored.
Initialization is evaluated only once during the lifetime of the
Varnish child process. If a new instance of VCL is loaded with calls
to ``init()`` in ``vcl_init`` after initialization was already
performed for another instance, then an error message is logged (with
XID 0 and the ``Debug`` tag), and the calls are ignored (but the VCL
reload does not fail). This is true even if the prior VCL instance is
unloaded with the CLI command ``vcl.discard``. This means in
particular that the configuration of secure memory cannot be changed
at runtime; for that, you have to restart the Varnish child process,
with a new instance of VCL that changes the configuration.
If initialization has been finalized in a previously loaded VCL
instance, then a new VCL instance can be loaded that does not include
any ``init()`` calls and uses the VMOD; in that case, the prior
initialization continues to hold. But the simplest strategy is
probably to test a configuration that works for your deployment in the
long run, and invoke that configuration in every VCL instance that you
load at runtime. After the first load, subsequent invocations of
``init()`` are ignored, but are harmless.
Examples::
import gcrypt;
sub vcl_init {
# Default initialization -- secure memory is enabled with
# the default size for the pool.
gcrypt.init(FINISH);
}
# Note that changed initializations are only effective after a
# restart of the Varnish child process, as explained above.
sub vcl_init {
# Enable secure memory and allocate a 64KiB pool.
gcrypt.init(INIT_SECMEM, 65536);
gcrypt.init(FINISH);
}
sub vcl_init {
# Disable secure memory.
gcrypt.init(DISABLE_SECMEM);
gcrypt.init(FINISH);
}
$Object symmetric(ENUM {AES, AES128, RIJNDAEL, RIJNDAEL128, AES192, RIJNDAEL192, $Object symmetric(ENUM {AES, AES128, RIJNDAEL, RIJNDAEL128, AES192, RIJNDAEL192,
AES256, RIJNDAEL256} cipher, ENUM {ECB, CFB, CBC, OFB, CTR} AES256, RIJNDAEL256} cipher, ENUM {ECB, CFB, CBC, OFB, CTR}
mode, ENUM {PKCS7, ISO7816, X923, NONE} padding="PKCS7", mode, ENUM {PKCS7, ISO7816, X923, NONE} padding="PKCS7",
BLOB key, BOOL secure=1, BOOL cbc_cts=0) BLOB key, BOOL secure=1, BOOL cbc_cts=0)
Create an object for encryption and decryption with symmetric ciphers.
Currently, only AES is supported, with key lengths 128, 192 and 256.
The ``cipher`` ENUM specifies the cryptographic algorithm to be used:
* ``AES``, ``AES128``, ``RIJNDAEL``, ``RIJNDAEL128``: AES-128 (the ENUMs
are aliases for one another)
* ``AES192``, ``RIJNDAEL192``: AES-192
* ``AES256``, ``RIJNDAEL256``: AES-256
The ``mode`` ENUM specifies the mode of operation.
The ``padding`` ENUM specifies the padding method, for modes of
operation that require it. Padding is required for ``ECB`` and
``CBC``; except that padding is not required for ``CBC`` with
ciphertext stealing, when the ``cbc_cts`` flag is set to ``true``. If
the mode of operation does not require padding, then the value of
``padding`` is ignored.
* ``PKCS7``: PKCS#7 padding (sometimes called PKCS#5). A padding byte
is always added to the message, and the values of all padding bytes
are the number of padding bytes added. This is the default value of
``padding``.
* ``ISO7816``: ISO/IEC 7816-4:2005 padding. Byte 0x80 is always added
to the message, and the remaining padding bytes are set to 0x00.
* ``X923``: ANSI X.923 padding. A padding byte is always added to the
message, and the value of the last byte is the number of padding
bytes added. The remaining padding bytes are set to 0x00.
* ``NONE``: no padding. This can be used if you are certain that the
length of all messages to be encrypted or decrypted will always be
an exact multiple of the cipher's block length, and the other party
also uses no padding.
The ``key`` BLOB is the cryptographic key to be used. Its size MUST
be correct for the cipher that is to be used; for example, 16 bytes
for AES-128.
If the ``secure`` flag is true, then secure memory is used by the
internal libgcrypt structures created for the object. True by default.
If the ``cbc_cts`` flag is true and ``CBC`` mode is specified, then
CBC is used with ciphertext stealing (and padding is not required).
``cbc_cts`` is ignored for all other modes of operation. Default is
``false``.
The creation of a ``symmetric`` object fails, leading to a failure
of the VCL load with an error message, under these conditions:
* libgcrypt initialization has not been finalized (``init(FINISH)``
has never been called).
* ``key`` is NULL, for example due to failure to create a BLOB
object.
* The size of the ``key`` BLOB is larger than the longest key length
supported for ``cipher``.
* The ``secure`` flag is ``true``, but secure memory has been
disabled.
* Any failure of the libgcrypt library to initialize internal
structures to be used for this object.
Examples::
import gcrypt;
import blobcode;
# Assume in the following that initialization has been finalized.
sub vcl_init {
# Create some BLOBs for the cryptographic keys, whose lengths
# are 16, 24 and 32 bytes for AES-128, -192 and -256,
# respectively.
# As emphasized above, DON'T copy these keys into production
# VCL!
new k128 = blobcode.blob(HEX, "000102030405060708090a0b0c0d0e0f");
new k192 = blobcode.blob(HEX,
"000102030405060708090a0b0c0d0e0f1011121314151617");
new k256 = blobcode.blob(HEX,
"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f");
# Create an object for AES-128 with CTR mode (no padding
# required), using secure memory by default.
new aes128 = gcrypt.symmetric(AES128, CTR, key=k128.get());
# Create an object for AES-192 with CBC mode and PKCS#7
# padding, not using secure memory.
new aes192 = gcrypt.symmetric(AES192, CBC, PKCS7, key=k192.get(),
secure=false);
# Create an object for AES-256 with CBC mode and ciphertext
# stealing (no padding required), using secure memory.
new aes256 = gcrypt.symmetric(AES256, CBC, key=k256.get(),
cbc_cts=true);
}
$Method BLOB .encrypt(BLOB plaintext, BLOB iv=0, BLOB ctr=0) $Method BLOB .encrypt(BLOB plaintext, BLOB iv=0, BLOB ctr=0)
Encrypt the contents of ``plaintext`` according to the parameters
specified for this ``symmetric`` object, and return the ciphertext as
a BLOB. The BLOB ``plaintext`` MAY NOT be NULL.
The ``iv`` and ``ctr`` parameters specify initialization and counter
vectors, respectively, and MUST be non-NULL for modes of operation
that require them. They MUST also be of the size required for the
cipher in use for the ``symmetric`` object. An initialization vector
is required for CFB, CBC and OFB modes, and a counter vector is
required for CTR. ``iv`` and ``ctr`` are ignored for modes of
operation that do not require them, and can be left out in those
cases.
The method fails under the following conditions:
* There is insufficient workspace for the BLOB returned as the
ciphertext, or for the padded plaintext, if padding is necessary.
* The ``iv`` or ``ctr`` parameters are NULL for modes of operation
that require init or counter vectors.
* Any failure of the libgcrypt library -- setting the init or counter
vectors, or encrypting the message.
If the method fails, a message is logged using the ``VCL_Error`` tag,
and NULL is returned.
Examples::
# In the following assume:
# - The aes128, aes192 and aes256 objects from the previous examples.
# - The messages to be encrypted are in the response headers
# X-Msg-128, X-Msg-192 and X-Msg-256.
# - An appropriately sized base64-encoded counter vector is in
# the response header X-Ctr.
# - Appropriately sized base64-encoded init vectors are in the
# response headers X-IV-192 and X-IV-256.
sub vcl_deliver {
# Encrypt X-Msg-128 with AES-128 and the counter vector, and
# return the ciphertext in a base64-encoded response header.
set resp.http.X-Cipher-128 = blobcode.encode(BASE64,
aes128.encrypt(blobcode.decode(encoded=resp.http.X-Msg-128),
ctr=blobcode.decode(BASE64, resp.http.X-Ctr)));
# Encrypt X-Msg-192 with AES-192 and init vector X-IV-192.
set resp.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.encrypt(blobcode.decode(encoded=resp.http.X-Msg-192),
iv=blobcode.decode(BASE64, resp.http.X-IV-192)));
# Encrypt X-Msg-256 with AES-256 and init vector X-IV-256.
set resp.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.encrypt(blobcode.decode(encoded=resp.http.X-Msg-192),
iv=blobcode.decode(BASE64, resp.http.X-IV-192)));
}
$Method BLOB .decrypt(BLOB ciphertext, BLOB iv=0, BLOB ctr=0) $Method BLOB .decrypt(BLOB ciphertext, BLOB iv=0, BLOB ctr=0)
Decrypt the contents of ``ciphertext`` according to the parameters
specified for this ``symmetric`` object, and return the plaintext as
a BLOB. The BLOB ``ciphertext`` MAY NOT be NULL.
As with the ``.encrypt()`` method, the ``iv`` and ``ctr`` parameters
specify initialization and counter vectors, and MUST be specified for
modes of operation that require them, but may be left out otherwise.
The failure conditions of ``.decrypt()`` are similar to those for
``.encrypt()``:
* Insufficient workspace for the ciphertext to be returned.
* Missing ``iv`` or ``ctr`` parameters for modes of operation that
require them.
* Failure of the libgcrypt library.
If the method fails, a message is logged using the ``VCL_Error`` tag,
and NULL is returned.
Examples::
# Assume the aes128, aes192 and aes256 objects from the previous
# examples. Also assume the messages to be decrypted are in the
# headers X-Msg-128, X-Msg-192 and X-Msg-256, as well as the headers
# X-Ctr, X-IV-192 and X-IV-256 as described for the .encrypt()
# example above, except that they are all request headers.
sub vcl_recv {
# Decrypt X-Msg-128 with AES-128 and the counter vector, and
# return the plaintext in a base64-encoded request header.
set req.http.X-Cipher-128 = blobcode.encode(BASE64,
aes128.decrypt(blobcode.decode(encoded=req.http.X-Msg-128),
ctr=blobcode.decode(BASE64, req.http.X-Ctr)));
# Decrypt X-Msg-192 with AES-192 and init vector X-IV-192.
set req.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.decrypt(blobcode.decode(encoded=req.http.X-Msg-192),
iv=blobcode.decode(BASE64, req.http.X-IV-192)));
# Decrypt X-Msg-256 with AES-256 and init vector X-IV-256.
set req.http.X-Cipher-192 = blobcode.encode(BASE64,
aes192.decrypt(blobcode.decode(encoded=req.http.X-Msg-192),
iv=blobcode.decode(BASE64, req.http.X-IV-192)));
}
$Function STRING version() $Function STRING version()
Returns the version string for this VMOD. Returns the version string for this VMOD.
...@@ -49,12 +447,136 @@ Example:: ...@@ -49,12 +447,136 @@ Example::
REQUIREMENTS REQUIREMENTS
============ ============
This VMOD requires Varnish ... This VMOD requires Varnish version 5.1 or later, and libgcrypt version
1.6.3 or later.
LIMITATIONS LIMITATIONS
=========== ===========
... As noted above, the VMOD uses Varnish workspace for the BLOB objects
returned by the ``.encrypt()`` and ``.decrypt()`` methods, and for
padding a plaintext before encryption, if necessary. If you find that
these methods are failing, with ``VCL_Error`` messages "out of space"
in the Varnish log, then increase the varnishd parameters
``workspace_client`` and/or ``workspace_backend``.
If you choose to use the libgcrypt secure memory feature, then the
size of the pool that is needed depends on the number of VMOD objects
that you create. The default size should be sufficient for typical
use cases, but you may need to configure a larger pool if you are
using many objects.
Note that when a VCL instance is unloaded at runtime (so as to load a
new instance), the objects allocated for the unloaded instance are not
de-allocated until that instance enters the cold state, by default
after 5 minutes in Varnish 5.1 (determined by the varnishd parameter
``vcl_cooldown``); so the use of the secure memory pool accumulates
during this time. You can bring about an earlier de-allocation, and
hence reduce the use of the secure memory pool, by shortening the
``vcl_cooldown`` period, or by using the CLI command ``vcl.discard``
to explicitly remove previous VCL instances.
SECURITY
========
If you intend to use the VMOD to protect sensitive data, it is
critical that you have a firm understanding of how to use the
cryptographic primitives that it makes available, and of potential
vulnerabilities of the tools you employ -- libgcrypt, Varnish, VMOD
gcrypt and any other VMODs you deploy.
Software
--------
Both libgcrypt and Varnish have strong security records. Nevertheless,
you should stay informed of any security fixes that are released for
both packages, and install them promptly when they are. Please inform
us (at the developer contact given in `SEE ALSO`_) if you encounter
any difficulties using the VMOD with new versions of libgcrypt or
Varnish.
VMOD gcrypt, in contrast, is much newer, and has not established a
security record. We welcome any bug reports, bug fixes, reviews or
other feedback you may have. And it would help to let us know if
you are using the VMOD successfully.
Secure and non-secure memory
----------------------------
The secure memory feature of the libgcrypt library gives a certain
measure of protection against exposure of sensitive data, but at
present that is limited to internal structures used by the library
(mainly, it protects the keys stored in those structures). Moreover,
the secure memory pool is visible in the address space of the Varnish
child process.
VMOD gcrypt, by itself, is neutral as to the sources of data that
enter into cryptographic operations -- keys, plaintexts, ciphertexts,
IVs and counters. But these are typically obtained from sources that
are readable from other elements of Varnish.
As can be seen in the examples above that use VMOD blobcode to create
key objects, the contents of a key are readable in the VCL source. If
you are using the VMOD that way, consider storing VCL sources so that
they are only readable by the owner of the Varnish child process.
Ciphertexts and plaintexts are typically obtained from sources that
are stored in Varnish workspaces, for example when they are read from
headers, as in the examples above. And the ciphertexts and plaintexts
produced by the VMOD on encryption and decryption are stored in
workspaces. Workspaces are re-used for the lifetime of a worker
thread. They tend to be overwritten, but this does not happen
automatically when a workspace is re-used. Nothing prevents another
VMOD from reading the workspace used by VMOD gcrypt.
Note that if any such data is stored in the cache, for example in
response headers of cached responses, then they are saved for the
lifetime of the cached object. If file storage is used for the cache,
then that data can be read from the cache file.
All of the data just described, *except* for the contents of the
libgcrypt secure memory pool, could conceivably be paged into swap
space, and all of it can be written to a core dump file.
Therefore, if you are working with sensitive data, consider the
following measures:
* Test your Varnish deployment thoroughly with test data, until you
are confident that it does not to swap or dump core.
* Then in production, use the means of your operating system to ensure
that the Varnish child process does not swap or write core files.
* Examine any other VMODs that you use to ensure that they do not leak
data from workspaces.
* Ensure that VCL sources are only readable by the Varnish child
process.
* If you are using file storage for the cache, ensure that the cache
file is only readable and writable by the child process.
Cryptographic good practice
---------------------------
Finally, make sure that you are using the cryptographic primitives
properly and safely. That subject is beyond the scope of this manual;
when in doubt, consult an expert. We want to emphasize the following
points, but this list is by no means exhaustive:
* The ECB mode of operation is included for testing purposes, but
should *never* be used to protect sensitive data in insecure
environments.
* Initialization and counter vectors can be sent in the clear, and do
not need to generated in a secure way; for example, they do not need
to be obtained from cryptographic random number generators. But an
IV or counter should never be used twice with the same key, so make
sure that they are generated uniquely.
* Make sure that you have a secure procedure in place for generating
and storing cryptographic keys, and for changing the keys
periodically.
INSTALLATION INSTALLATION
============ ============
...@@ -66,6 +588,11 @@ SEE ALSO ...@@ -66,6 +588,11 @@ SEE ALSO
* varnishd(1) * varnishd(1)
* vcl(7) * vcl(7)
* varnishlog(1)
* libgcrypt: https://gnupg.org/software/libgcrypt/index.html
* source repository: https://code.uplex.de/uplex-varnish/libvmod-gcrypt * source repository: https://code.uplex.de/uplex-varnish/libvmod-gcrypt
* VMOD blobcode: https://code.uplex.de/uplex-varnish/libvmod-blobcode
* developer contact: <varnish-support@uplex.de>, and at the source
repository site
$Event event $Event event
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment