postpone GCM authentication tag requirement until finalization (#3421)

* postpone GCM authentication tag requirement until finalization

Add a .finalize_with_tag() variant of the .finalize() function of
the GCM context. At the same time, do not enforce the requirement
of supplying the tag with the mode ctor. This facilitates
streamed decryption when the MAC is appended to the ciphertext
and cannot be efficiently retrieved ahead of decryption.

According to the GCM spec (section 7.2: “Algorithm for the
Authenticated Decryption Function”), the tag itself is not needed
until the ciphertext has been decrypted.

Addresses #3380

Signed-off-by: Philipp Gesang <philipp.gesang@intra2net.com>

* disallow delayed GCM tag passing for legacy OpenSSL

Old versions of Ubuntu supported by Cryptography ship a v1.0.1 of
OpenSSL which is no longer supported by upstream. This library
seems to cause erratic test failures with the delayed GCM tag
functionality which are not reproducible outside the CI.
Unfortunately OpenSSL v1.0.1 does not even document the required
API (``EVP_EncryptInit(3)``) so there is no by-the-book fix.

For backends of version 1.0.1 and earlier, verify the GCM tag
at the same stage as before.

Also, indicate to the user that late passing of GCM tags is
unsupported by throwing ``NotImplementedError`` for these backend
versions if

    - the method ``finalize_with_tag()`` is invoked, or
    - the mode ctor is called without passing a tag.

Unit tests have been adapted to account for different backend
versions.

7 files changed, 208 insertions, 33 deletions

diff --git a/docs/hazmat/primitives/symmetric-encryption.rst b/docs/hazmat/primitives/symmetric-encryption.rst
index 17d91091..5f4d7bf9 100644
--- a/docs/hazmat/primitives/symmetric-encryption.rst
+++ b/docs/hazmat/primitives/symmetric-encryption.rst
@@ -292,7 +292,10 @@ Modes
     .. danger::
 
         When using this mode you **must** not use the decrypted data until
-        :meth:`~cryptography.hazmat.primitives.ciphers.CipherContext.finalize`
+        the appropriate finalization method
+        (:meth:`~cryptography.hazmat.primitives.ciphers.CipherContext.finalize`
+        or
+        :meth:`~cryptography.hazmat.primitives.ciphers.AEADDecryptionContext.finalize_with_tag`)
         has been called. GCM provides **no** guarantees of ciphertext integrity
         until decryption is complete.
 
@@ -326,7 +329,10 @@ Modes
             truncation down to ``4`` bytes was always allowed.
 
     :param bytes tag: The tag bytes to verify during decryption. When
-        encrypting this must be ``None``.
+        encrypting this must be ``None``. When decrypting, it may be ``None``
+        if the tag is supplied on finalization using
+        :meth:`~cryptography.hazmat.primitives.ciphers.AEADDecryptionContext.finalize_with_tag`.
+        Otherwise, the tag is mandatory.
 
     :param bytes min_tag_length: The minimum length ``tag`` must be. By default
         this is ``16``, meaning tag truncation is not allowed. Allowing tag
@@ -334,6 +340,9 @@ Modes
 
     :raises ValueError: This is raised if ``len(tag) < min_tag_length``.
 
+    :raises NotImplementedError: This is raised if the version of the OpenSSL
+        backend used is 1.0.1 or earlier.
+
     An example of securely encrypting and decrypting data with ``AES`` in the
     ``GCM`` mode looks like:
 
@@ -516,12 +525,12 @@ Interfaces
     with an AEAD mode (e.g.
     :class:`~cryptography.hazmat.primitives.ciphers.modes.GCM`) the result will
     conform to the ``AEADCipherContext`` and ``CipherContext`` interfaces. If
-    it is an encryption context it will additionally be an
-    ``AEADEncryptionContext`` instance. ``AEADCipherContext`` contains an
-    additional method :meth:`authenticate_additional_data` for adding
-    additional authenticated but unencrypted data (see note below). You should
-    call this before calls to ``update``. When you are done call ``finalize``
-    to finish the operation.
+    it is an encryption or decryption context it will additionally be an
+    ``AEADEncryptionContext`` or ``AEADDecryptionContext`` instance,
+    respectively. ``AEADCipherContext`` contains an additional method
+    :meth:`authenticate_additional_data` for adding additional authenticated
+    but unencrypted data (see note below). You should call this before calls to
+    ``update``. When you are done call ``finalize`` to finish the operation.
 
     .. note::
 
@@ -551,6 +560,37 @@ Interfaces
         :raises: :class:`~cryptography.exceptions.NotYetFinalized` if called
             before the context is finalized.
 
+.. class:: AEADDecryptionContext
+
+    .. versionadded:: 1.9
+
+    When creating an encryption context using ``decryptor`` on a ``Cipher``
+    object with an AEAD mode such as
+    :class:`~cryptography.hazmat.primitives.ciphers.modes.GCM` an object
+    conforming to both the ``AEADDecryptionContext`` and ``AEADCipherContext``
+    interfaces will be returned.  This interface provides one additional method
+    :meth:`finalize_with_tag` that allows passing the authentication tag for
+    validation after the ciphertext has been decrypted.
+
+    .. method:: finalize_with_tag(tag)
+
+        :param bytes tag: The tag bytes to verify after decryption.
+        :return bytes: Returns the remainder of the data.
+        :raises ValueError: This is raised when the data provided isn't
+            a multiple of the algorithm's block size, if ``min_tag_length`` is
+            less than 4, or if ``len(tag) < min_tag_length``.
+        :raises NotImplementedError: This is raised if the version of the
+            OpenSSL backend used is 1.0.1 or earlier.
+
+        If the authentication tag was not already supplied to the constructor
+        of the :class:`~cryptography.hazmat.primitives.ciphers.modes.GCM` mode
+        object, this method must be used instead of
+        :meth:`~cryptography.hazmat.primitives.ciphers.CipherContext.finalize`.
+
+    .. note::
+
+        This method is not supported when compiled against OpenSSL 1.0.1.
+
 .. class:: CipherAlgorithm
 
     A named symmetric encryption algorithm.
diff --git a/src/cryptography/hazmat/backends/commoncrypto/ciphers.py b/src/cryptography/hazmat/backends/commoncrypto/ciphers.py
index b59381cb..85ec9e76 100644
--- a/src/cryptography/hazmat/backends/commoncrypto/ciphers.py
+++ b/src/cryptography/hazmat/backends/commoncrypto/ciphers.py
@@ -213,11 +213,15 @@ class _GCMCipherContext(object):
         self._backend._check_cipher_response(res)
         self._backend._release_cipher_ctx(self._ctx)
         self._tag = self._backend._ffi.buffer(tag_buf)[:]
-        if (self._operation == self._backend._lib.kCCDecrypt and
-                not constant_time.bytes_eq(
-                    self._tag[:len(self._mode.tag)], self._mode.tag
-                )):
-            raise InvalidTag
+        if self._operation == self._backend._lib.kCCDecrypt:
+            if self._mode.tag is None:
+                raise ValueError(
+                    "Authentication tag must be provided when decrypting."
+                )
+            if not constant_time.bytes_eq(
+                self._tag[:len(self._mode.tag)], self._mode.tag
+            ):
+                raise InvalidTag
         return b""
 
     def authenticate_additional_data(self, data):
diff --git a/src/cryptography/hazmat/backends/openssl/ciphers.py b/src/cryptography/hazmat/backends/openssl/ciphers.py
index 0e0918af..b6058150 100644
--- a/src/cryptography/hazmat/backends/openssl/ciphers.py
+++ b/src/cryptography/hazmat/backends/openssl/ciphers.py
@@ -78,7 +78,13 @@ class _CipherContext(object):
                 len(iv_nonce), self._backend._ffi.NULL
             )
             self._backend.openssl_assert(res != 0)
-            if operation == self._DECRYPT:
+            if operation == self._DECRYPT and \
+                    self._backend.openssl_version_number() < 0x10002000:
+                if mode.tag is None:
+                    raise NotImplementedError(
+                        "delayed passing of GCM tag requires OpenSSL >= 1.0.2."
+                        " To use this feature please update OpenSSL"
+                    )
                 res = self._backend._lib.EVP_CIPHER_CTX_ctrl(
                     ctx, self._backend._lib.EVP_CTRL_GCM_SET_TAG,
                     len(mode.tag), mode.tag
@@ -134,6 +140,20 @@ class _CipherContext(object):
         if isinstance(self._mode, modes.GCM):
             self.update(b"")
 
+        if self._operation == self._DECRYPT and \
+                isinstance(self._mode, modes.ModeWithAuthenticationTag) and \
+                self._backend.openssl_version_number() >= 0x10002000:
+            tag = self._mode.tag
+            if tag is None:
+                raise ValueError(
+                    "Authentication tag must be provided when decrypting."
+                )
+            res = self._backend._lib.EVP_CIPHER_CTX_ctrl(
+                self._ctx, self._backend._lib.EVP_CTRL_GCM_SET_TAG,
+                len(tag), tag
+            )
+            self._backend.openssl_assert(res != 0)
+
         buf = self._backend._ffi.new("unsigned char[]", self._block_size_bytes)
         outlen = self._backend._ffi.new("int *")
         res = self._backend._lib.EVP_CipherFinal_ex(self._ctx, buf, outlen)
diff --git a/src/cryptography/hazmat/primitives/ciphers/base.py b/src/cryptography/hazmat/primitives/ciphers/base.py
index e9d55a10..9e0d0051 100644
--- a/src/cryptography/hazmat/primitives/ciphers/base.py
+++ b/src/cryptography/hazmat/primitives/ciphers/base.py
@@ -76,6 +76,16 @@ class AEADCipherContext(object):
 
 
 @six.add_metaclass(abc.ABCMeta)
+class AEADDecryptionContext(object):
+    @abc.abstractmethod
+    def finalize_with_tag(self, tag):
+        """
+        Returns the results of processing the final block as bytes and allows
+        delayed passing of the authentication tag.
+        """
+
+
+@six.add_metaclass(abc.ABCMeta)
 class AEADEncryptionContext(object):
     @abc.abstractproperty
     def tag(self):
@@ -115,11 +125,6 @@ class Cipher(object):
         return self._wrap_ctx(ctx, encrypt=True)
 
     def decryptor(self):
-        if isinstance(self.mode, modes.ModeWithAuthenticationTag):
-            if self.mode.tag is None:
-                raise ValueError(
-                    "Authentication tag must be provided when decrypting."
-                )
         ctx = self._backend.create_symmetric_decryption_ctx(
             self.algorithm, self.mode
         )
@@ -169,6 +174,7 @@ class _CipherContext(object):
 
 @utils.register_interface(AEADCipherContext)
 @utils.register_interface(CipherContext)
+@utils.register_interface(AEADDecryptionContext)
 class _AEADCipherContext(object):
     def __init__(self, ctx):
         self._ctx = ctx
@@ -214,6 +220,16 @@ class _AEADCipherContext(object):
         self._ctx = None
         return data
 
+    def finalize_with_tag(self, tag):
+        if self._ctx._backend.name == "openssl" and \
+                self._ctx._backend.openssl_version_number() < 0x10002000:
+            raise NotImplementedError(
+                "finalize_with_tag requires OpenSSL >= 1.0.2. To use this "
+                "method please update OpenSSL"
+            )
+        self._ctx._mode._set_tag(tag)
+        return self.finalize()
+
     def authenticate_additional_data(self, data):
         if self._ctx is None:
             raise AlreadyFinalized("Context was already finalized.")
diff --git a/src/cryptography/hazmat/primitives/ciphers/modes.py b/src/cryptography/hazmat/primitives/ciphers/modes.py
index 802e544a..5b28157a 100644
--- a/src/cryptography/hazmat/primitives/ciphers/modes.py
+++ b/src/cryptography/hazmat/primitives/ciphers/modes.py
@@ -161,21 +161,22 @@ class GCM(object):
         # len(initialization_vector) must in [1, 2 ** 64), but it's impossible
         # to actually construct a bytes object that large, so we don't check
         # for it
-        if min_tag_length < 4:
-            raise ValueError("min_tag_length must be >= 4")
-        if tag is not None and len(tag) < min_tag_length:
-            raise ValueError(
-                "Authentication tag must be {0} bytes or longer.".format(
-                    min_tag_length)
-            )
-
         if not isinstance(initialization_vector, bytes):
             raise TypeError("initialization_vector must be bytes")
-
-        if tag is not None and not isinstance(tag, bytes):
-            raise TypeError("tag must be bytes or None")
-
         self._initialization_vector = initialization_vector
+        self._set_tag(tag, min_tag_length)
+
+    def _set_tag(self, tag, min_tag_length=16):
+        if tag is not None:
+            if not isinstance(tag, bytes):
+                raise TypeError("tag must be bytes or None")
+            if min_tag_length < 4:
+                raise ValueError("min_tag_length must be >= 4")
+            if len(tag) < min_tag_length:
+                raise ValueError(
+                    "Authentication tag must be {0} bytes or longer.".format(
+                        min_tag_length)
+                )
         self._tag = tag
 
     tag = utils.read_only_property("_tag")
diff --git a/tests/hazmat/primitives/test_aes.py b/tests/hazmat/primitives/test_aes.py
index 8826aae8..392a847f 100644
--- a/tests/hazmat/primitives/test_aes.py
+++ b/tests/hazmat/primitives/test_aes.py
@@ -303,3 +303,99 @@ class TestAESModeGCM(object):
         assert encryptor._aad_bytes_processed == 8
         encryptor.authenticate_additional_data(b"0" * 18)
         assert encryptor._aad_bytes_processed == 26
+
+    def test_gcm_tag_decrypt_none(self, backend):
+        key = binascii.unhexlify(b"5211242698bed4774a090620a6ca56f3")
+        iv = binascii.unhexlify(b"b1e1349120b6e832ef976f5d")
+        aad = binascii.unhexlify(b"b6d729aab8e6416d7002b9faa794c410d8d2f193")
+
+        encryptor = base.Cipher(
+            algorithms.AES(key),
+            modes.GCM(iv),
+            backend=backend
+        ).encryptor()
+        encryptor.authenticate_additional_data(aad)
+        encryptor.finalize()
+
+        if backend.name == "openssl" and \
+                backend.openssl_version_number() < 0x10002000:
+            with pytest.raises(NotImplementedError):
+                decryptor = base.Cipher(
+                    algorithms.AES(key),
+                    modes.GCM(iv),
+                    backend=backend
+                ).decryptor()
+        else:
+            decryptor = base.Cipher(
+                algorithms.AES(key),
+                modes.GCM(iv),
+                backend=backend
+            ).decryptor()
+            decryptor.authenticate_additional_data(aad)
+            with pytest.raises(ValueError):
+                decryptor.finalize()
+
+    def test_gcm_tag_decrypt_mode(self, backend):
+        key = binascii.unhexlify(b"5211242698bed4774a090620a6ca56f3")
+        iv = binascii.unhexlify(b"b1e1349120b6e832ef976f5d")
+        aad = binascii.unhexlify(b"b6d729aab8e6416d7002b9faa794c410d8d2f193")
+
+        encryptor = base.Cipher(
+            algorithms.AES(key),
+            modes.GCM(iv),
+            backend=backend
+        ).encryptor()
+        encryptor.authenticate_additional_data(aad)
+        encryptor.finalize()
+        tag = encryptor.tag
+
+        decryptor = base.Cipher(
+            algorithms.AES(key),
+            modes.GCM(iv, tag),
+            backend=backend
+        ).decryptor()
+        decryptor.authenticate_additional_data(aad)
+        decryptor.finalize()
+
+    def test_gcm_tag_decrypt_finalize(self, backend):
+        key = binascii.unhexlify(b"5211242698bed4774a090620a6ca56f3")
+        iv = binascii.unhexlify(b"b1e1349120b6e832ef976f5d")
+        aad = binascii.unhexlify(b"b6d729aab8e6416d7002b9faa794c410d8d2f193")
+
+        encryptor = base.Cipher(
+            algorithms.AES(key),
+            modes.GCM(iv),
+            backend=backend
+        ).encryptor()
+        encryptor.authenticate_additional_data(aad)
+        encryptor.finalize()
+        tag = encryptor.tag
+
+        if backend.name == "openssl" and \
+                backend.openssl_version_number() < 0x10002000:
+            with pytest.raises(NotImplementedError):
+                decryptor = base.Cipher(
+                    algorithms.AES(key),
+                    modes.GCM(iv),
+                    backend=backend
+                ).decryptor()
+            decryptor = base.Cipher(
+                algorithms.AES(key),
+                modes.GCM(iv, tag=encryptor.tag),
+                backend=backend
+            ).decryptor()
+        else:
+            decryptor = base.Cipher(
+                algorithms.AES(key),
+                modes.GCM(iv),
+                backend=backend
+            ).decryptor()
+        decryptor.authenticate_additional_data(aad)
+
+        if backend.name == "openssl" and \
+                backend.openssl_version_number() < 0x10002000:
+            with pytest.raises(NotImplementedError):
+                decryptor.finalize_with_tag(tag)
+            decryptor.finalize()
+        else:
+            decryptor.finalize_with_tag(tag)
diff --git a/tests/hazmat/primitives/utils.py b/tests/hazmat/primitives/utils.py
index d0e87a78..59326367 100644
--- a/tests/hazmat/primitives/utils.py
+++ b/tests/hazmat/primitives/utils.py
@@ -304,8 +304,6 @@ def aead_tag_exception_test(backend, cipher_factory, mode_factory):
         mode_factory(binascii.unhexlify(b"0" * 24)),
         backend
     )
-    with pytest.raises(ValueError):
-        cipher.decryptor()
 
     with pytest.raises(ValueError):
         mode_factory(binascii.unhexlify(b"0" * 24), b"000")