foundationdb/flow/EncryptUtils.h
Ata E Husain Bohra a5d91fe18a
KmsConnector implementation to support KMS driven CipherKey TTL (#7334)
* KmsConnector implementation to support KMS driven CipherKey TTL

Description

KMS CipherKeys can be of two types:
1. Revocable CipherKeys: having a finite lifetime, after which the CipherKey
shouldn't be used by the FDB.
2. Non-revocable CipherKeys: ciphers are not revocable, however, FDB would
still want to refresh ciphers to support KMS cipher rotation feature.

Patch proposes following change to incorporate support for above defined cipher-key
types:
1. Extend KmsConnector response to include optional 'refreshAfter' & 'expireAfter'
time intervals. EncryptKeyProxy (EKP) cache would define corresponding absolute refresh &
expiry timestamp for a given cipherKey. On an event of transient KMS connectivity outage,
a caller of EKP API for a non-revocable key should continue using cached cipherKey until
it expires.
2. Simplify KmsConnector API arena handling by using VectorRef to represent component
structs and manage associated memory allocation/lifetime.

Testing

1. EncryptKeyProxyTest
2. RESTKmsConnectorTest
3. SimKmsConnectorTest

* KmsConnector implementation to support KMS driven CipherKey TTL

Description

  diff-1: Set expireTS for baseCipherId indexed cache

KMS CipherKeys can be of two types:
1. Revocable CipherKeys: having a finite lifetime, after which the CipherKey
shouldn't be used by the FDB.
2. Non-revocable CipherKeys: ciphers are not revocable, however, FDB would
still want to refresh ciphers to support KMS cipher rotation feature.

Patch proposes following change to incorporate support for above defined cipher-key
types:
1. Extend KmsConnector response to include optional 'refreshAfter' & 'expireAfter'
time intervals. EncryptKeyProxy (EKP) cache would define corresponding absolute refresh &
expiry timestamp for a given cipherKey. On an event of transient KMS connectivity outage,
a caller of EKP API for a non-revocable key should continue using cached cipherKey until
it expires.
2. Simplify KmsConnector API arena handling by using VectorRef to represent component
structs and manage associated memory allocation/lifetime.

Testing

1. EncryptKeyProxyTest
2. RESTKmsConnectorTest
3. SimKmsConnectorTest

* KmsConnector implementation to support KMS driven CipherKey TTL

Description

  diff-2: Fix Valgrind issues discovered runnign tests
  diff-1: Set expireTS for baseCipherId indexed cache

KMS CipherKeys can be of two types:
1. Revocable CipherKeys: having a finite lifetime, after which the CipherKey
shouldn't be used by the FDB.
2. Non-revocable CipherKeys: ciphers are not revocable, however, FDB would
still want to refresh ciphers to support KMS cipher rotation feature.

Patch proposes following change to incorporate support for above defined cipher-key
types:
1. Extend KmsConnector response to include optional 'refreshAfter' & 'expireAfter'
time intervals. EncryptKeyProxy (EKP) cache would define corresponding absolute refresh &
expiry timestamp for a given cipherKey. On an event of transient KMS connectivity outage,
a caller of EKP API for a non-revocable key should continue using cached cipherKey until
it expires.
2. Simplify KmsConnector API arena handling by using VectorRef to represent component
structs and manage associated memory allocation/lifetime.

Testing

1. EncryptKeyProxyTest
2. RESTKmsConnectorTest
3. SimKmsConnectorTest

* KmsConnector implementation to support KMS driven CipherKey TTL

Description

  diff-3: Address review comment
  diff-2: Fix Valgrind issues discovered runnign tests
  diff-1: Set expireTS for baseCipherId indexed cache

KMS CipherKeys can be of two types:
1. Revocable CipherKeys: having a finite lifetime, after which the CipherKey
shouldn't be used by the FDB.
2. Non-revocable CipherKeys: ciphers are not revocable, however, FDB would
still want to refresh ciphers to support KMS cipher rotation feature.

Patch proposes following change to incorporate support for above defined cipher-key
types:
1. Extend KmsConnector response to include optional 'refreshAfter' & 'expireAfter'
time intervals. EncryptKeyProxy (EKP) cache would define corresponding absolute refresh &
expiry timestamp for a given cipherKey. On an event of transient KMS connectivity outage,
a caller of EKP API for a non-revocable key should continue using cached cipherKey until
it expires.
2. Simplify KmsConnector API arena handling by using VectorRef to represent component
structs and manage associated memory allocation/lifetime.

Testing

1. EncryptKeyProxyTest
2. RESTKmsConnectorTest
3. SimKmsConnectorTest
2022-06-13 13:25:01 -07:00

93 lines
3.5 KiB
C++

/*
* EncryptUtils.h
*
* This source file is part of the FoundationDB open source project
*
* Copyright 2013-2022 Apple Inc. and the FoundationDB project authors
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#ifndef ENCRYPT_UTILS_H
#define ENCRYPT_UTILS_H
#pragma once
#include "flow/Arena.h"
#include <cstdint>
#include <limits>
#include <string>
#include <string_view>
#define ENCRYPT_INVALID_DOMAIN_ID 0
#define ENCRYPT_INVALID_CIPHER_KEY_ID 0
#define ENCRYPT_INVALID_RANDOM_SALT 0
#define AUTH_TOKEN_SIZE 16
#define SYSTEM_KEYSPACE_ENCRYPT_DOMAIN_ID -1
#define ENCRYPT_HEADER_DOMAIN_ID -2
const std::string FDB_DEFAULT_ENCRYPT_DOMAIN_NAME = "FdbDefaultEncryptDomain";
using EncryptCipherDomainId = int64_t;
using EncryptCipherDomainName = StringRef;
using EncryptCipherBaseKeyId = uint64_t;
using EncryptCipherRandomSalt = uint64_t;
typedef enum {
ENCRYPT_CIPHER_MODE_NONE = 0,
ENCRYPT_CIPHER_MODE_AES_256_CTR = 1,
ENCRYPT_CIPHER_MODE_LAST = 2
} EncryptCipherMode;
static_assert(EncryptCipherMode::ENCRYPT_CIPHER_MODE_LAST <= std::numeric_limits<uint8_t>::max(),
"EncryptCipherMode value overflow");
// EncryptionHeader authentication modes
// 1. NONE - No 'authentication token' generation needed for EncryptionHeader i.e. no protection against header OR
// cipherText 'tampering' and/or bit rot/flip corruptions.
// 2. Single/Multi - Encryption header would generate one or more 'authentication tokens' to protect the header against
// 'tempering' and/or bit rot/flip corruptions. Refer to BlobCipher.h for detailed usage recommendations.
// 3. LAST - Invalid mode, used for static asserts.
typedef enum {
ENCRYPT_HEADER_AUTH_TOKEN_MODE_NONE = 0,
ENCRYPT_HEADER_AUTH_TOKEN_MODE_SINGLE = 1,
ENCRYPT_HEADER_AUTH_TOKEN_MODE_MULTI = 2,
ENCRYPT_HEADER_AUTH_TOKEN_LAST = 3 // Always the last element
} EncryptAuthTokenMode;
static_assert(EncryptAuthTokenMode::ENCRYPT_HEADER_AUTH_TOKEN_LAST <= std::numeric_limits<uint8_t>::max(),
"EncryptHeaderAuthToken value overflow");
constexpr std::string_view ENCRYPT_DBG_TRACE_CACHED_PREFIX = "Chd";
constexpr std::string_view ENCRYPT_DBG_TRACE_QUERY_PREFIX = "Qry";
constexpr std::string_view ENCRYPT_DBG_TRACE_INSERT_PREFIX = "Ins";
constexpr std::string_view ENCRYPT_DBG_TRACE_RESULT_PREFIX = "Res";
// Utility interface to construct TraceEvent key for debugging
std::string getEncryptDbgTraceKey(std::string_view prefix,
EncryptCipherDomainId domainId,
StringRef domainName,
Optional<EncryptCipherBaseKeyId> baseCipherId = Optional<EncryptCipherBaseKeyId>());
std::string getEncryptDbgTraceKeyWithTS(std::string_view prefix,
EncryptCipherDomainId domainId,
StringRef domainName,
EncryptCipherBaseKeyId baseCipherId,
int64_t refAfterTS,
int64_t expAfterTS);
#endif