Use customer-managed encryption keys (CMEK)
This page describes how to perform tasks related to customer-managed encryption keys (CMEK) for Firestore. For more information about CMEK in general, including when and why to enable it, see the Cloud KMS documentation.
Prepare your CMEK keys
Before you can create a CMEK-protected Firestore database, you must complete the following steps:
- Request access to the Firestore CMEK feature.
- Create (or retrieve) a Firestore service agent.
- Create a CMEK key.
- Configure IAM settings for that key.
Complete these steps for each project that will contain CMEK-protected Firestore databases. If you later create a new CMEK key, you must configure IAM settings for that key.
Request access
Before you create a Firestore service agent, request access to the CMEK feature by filling in this form.
Create a Firestore service agent
Before you create a CMEK key, you must have a Firestore service agent, which is a type of Google-managed service account that Firestore uses to access the key.
Run the services identity create command to create the service agent that Firestore uses to access the CMEK key on your behalf. This command creates the service account if it does not already exist, then displays it.
gcloud beta services identity create \ --service=firestore.googleapis.com \ --project FIRESTORE_PROJECT
Replace FIRESTORE_PROJECT
with the project you plan
to use for your Firestore databases.
The command displays the service agent ID, which is formatted like an email address. Record the output email string, because you'll use it in a later step.
Service identity created: service-xxx@gcp-sa-firestore.iam.gserviceaccount.com
Create a key
You can use a key created directly in Cloud KMS or an externally managed key that you make available with Cloud External Key Manager.
The Cloud KMS key location must be the same as the location of the Firestore database that it will be used with.
For regional database locations, use the same location name for key ring, key, and database because the location names have a one-to-one mapping.
For example, if you want to create a CMEK-protected database in
us-west1
, create a key ring and key inus-west1
.For multi-region database locations, use the location name of the KMS multi-region location:
- Use the Cloud KMS
us
multi-region location for the Firestorenam5
multi-region location. - Use the Cloud KMS
europe
multi-region location for the Firestoreeur3
multi-region location.
- Use the Cloud KMS
In the Google Cloud project where you want to manage your keys, complete the following:
Create a key ring and a key using one of the following options:
- Create the key ring and key directly in Cloud KMS.
- Use an externally managed key. Create the external key and then create an Cloud EKM key to make the key available through Cloud KMS.
Configure IAM settings for the key
Console
To grant an Cloud KMS role to your service agent, do the following. You are also able to grant permission at the key or key-ring level if you want lower granularity.
In the Google Cloud console, go to the IAM page.
Click Add.
Enter the email-formatted ID for your Firestore service agent.
Select the Cloud KMS CryptoKey Encrypter/Decrypter role.
Click Save.
gcloud
Grant the
cloudkms.cryptoKeyEncrypterDecrypter
role to your service agent:gcloud kms keys add-iam-policy-binding KMS_KEY \ --keyring KMS_KEYRING\ --location KMS_LOCATION \ --member serviceAccount:SERVICE_AGENT_EMAIL \ --role roles/cloudkms.cryptoKeyEncrypterDecrypter \ --project KMS_PROJECT
Provide the following:
KMS_KEY
: the name you assigned to the keyKMS_KEYRING
: the KMS key ring that contains the keyKMS_LOCATION
: the region that contains the key ringSERVICE_AGENT_EMAIL
: the email-formatted identifier for the service agent that you are granting access toKMS_PROJECT
: the project that contains the key
The terminal should display a response similar to the following:
Updated IAM policy for key KMS_KEY. bindings: - members: - serviceAccount: service-{project-number}@gcp-sa-firestore.iam.gserviceaccount.com role: roles/cloudkms.cryptoKeyEncrypterDecrypter
Create a CMEK-enabled database
After your CMEK keys are created and configured, you can create a CMEK-protected database. Existing Firestore database that are protected by Google default encryption cannot be converted to use CMEK; you can only choose an encryption type and key at the time of creation.
gcloud
gcloud alpha firestore databases create --location=FIRESTORE_DATABASE_LOCATION \
--database=DATABASE_ID \
--kms-key-name=KMS_KEY_NAME \
--project=FIRESTORE_PROJECT
Provide the following:
FIRESTORE_DATABASE_LOCATION
: the Firestore location for the databaseDATABASE_ID
: an ID for the databaseKMS_KEY_NAME
: the name you assigned to the key. Use the full resource name for the key in the format of:projects/KMS_PROJECT/locations/KMS_LOCATION/keyRings/KMS_KEYRING_ID/cryptoKeys/KMS_KEY_ID
FIRESTORE_PROJECT
: the project to use for your Firestore database
REST API
HTTP request:
POST https://firestore.googleapis.com/v1/projects/{FIRESOTRE_PROJECT}/databases
In the request body configure CMEK in the cmek_config.kms_key_name
field.
Set to the full resource ID of a Cloud KMS key. Only a key in the same location as this database is allowed.
This value should be the Cloud KMS key resource ID in the format of
projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}
For more details for other fields, see the database create
page.
Example request:
curl -X POST 'https://firestore.googleapis.com/v1/projects/FIRESTORE_PROJECT/databases?databaseId={DATABASE_ID}' \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-type: application/json" \
-d '{
"type":"FIRESTORE_NATIVE",
"locationId":"{FIRESTORE_DATABASE_LOCATION}",
"cmekConfig": {
"kmsKeyName":"projects/KMS_PROJECT/locations/KMS_LOCATION/keyRings/KMS_KEYRING_ID/cryptoKeys/KMS_KEY_ID"
}
}'
Terraform
To create a CMEK-enabled database, use the google_firestore_database
resource. For more information and examples, see
google_firestore_database
.
resource "google_firestore_database" "database" {
project = "FIRESTORE_PROJECT"
name = "DATABASE_ID"
location_id = "FIRESTORE_DATABASE_LOCATION"
type = "DATABASE_TYPE"
cmek_config {
kms_key_name = "KMS_KEY_NAME"
}
}
Provide the following:
FIRESTORE_PROJECT
: the project to use for your Firestore databaseDATABASE_ID
: an ID for the databaseFIRESTORE_DATABASE_LOCATION
: the Firestore location for the databaseDATABASE_TYPE
: either FIRESTORE_NATIVE for Native mode or DATASTORE_MODE for Datastore mode.KMS_KEY_NAME
: the name you assigned to the key. Use the full resource name for the key in the format of:projects/KMS_PROJECT/locations/KMS_LOCATION/keyRings/KMS_KEYRING_ID/cryptoKeys/KMS_KEY_ID
Access a CMEK-protected database
All the read, write, and query operations sent to a CMEK-protected database should function the same as with a Google default encrypted database. For example, you don't need to provide a key for each request.
View the key in use
gcloud
You can use the databases describe gcloud CLI command to confirm database CMEK configuration:
gcloud firestore databases describe --database=DATABASE_ID --project=FIRESTORE_PROJECT
You should see CMEK information in the cmekConfig
field in the response
similar to the following:
cmekConfig:
activeKeyVersion:
- projects/PROJECT_ID/locations/us/keyRings/KEYRING_NAME/cryptoKeys/KEY_NAME/cryptoKeyVersions/1
kmsKeyName: projects/PROJECT_ID/locations/us/keyRings/KEYRING_NAME/cryptoKeys/KEY_NAME
locationId: nam5
name: projects/PROJECT_ID/databases/DATABASE_ID
The response includes the following information:
kmsKeyName
: the full key resource name of the key that's used to encrypt your CMEK-protected database.activeKeyVersion
: a list of all key versions currently in use by the CMEK-protected database. During key rotation, there can be multiple active key versions.
REST API
HTTP request:
GET https://firestore.googleapis.com/v1/{name=projects/FIRESTORE_PROJECT/databases/DATABASE_ID}
In the request body configure CMEK in the cmek_config.kms_key_name
field.
Set to the full resource ID of a Cloud KMS key. Only a key in the same
location as this database is allowed.
This value should be the Cloud KMS key resource ID in the format of
projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}
For more details for other fields, see the database create
page.
Example request and response:
curl 'https://firestore.googleapis.com/v1/projects/FIRESTORE_PROJECT/databases/{DATABASE_ID}' \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-type: application/json"
—----------------------------------------- Response —--------------------------------------------
{
"name": "projects/FIRESTORE_PROJECT/databases/{DATABASE_ID}",
"locationId": "{FIRESTORE_DATABASE_LOCATION}",
"type": "FIRESTORE_NATIVE",
"cmekConfig": {
"kmsKeyName": "projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}",
"activeKeyVersion": [
"projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}/cryptoKeyVersions/1"
]
},
……
}
Disable a key
To disable a key associated with a database, complete the following:
- View the key versions in use for a database
- Disable those key versions
- Wait for the change to take effect and check if the data is no longer accessible. Changes typically take effect within minutes, but can take up to 3 hours.
When a key used by a database is disabled, expect to receive a
FAILED_PRECONDITION
exception with additional details in the error message,
for example:
{ "error": { "code": 400, "message": "The customer-managed encryption key required by the requested resource is not accessible. Error reason: generic::permission_denied: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/FIRESTORE_PROJECT/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}' (or it may not exist).", "status": "FAILED_PRECONDITION", "details": [ { "@type": "type.googleapis.com/google.rpc.DebugInfo", "detail": "The customer-managed encryption key required by the requested resource is not accessible. Error reason: generic::permission_denied: Permission 'cloudkms.cryptoKeyVersions.useToEncrypt' denied on resource 'projects/FIRESTORE_PROJECT/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}' (or it may not exist)" } ] } }
Enable a key
To re-enable a key associated with a database, complete the following:
- View the key versions in use for a database
- Enable those key versions
- Wait for the change to take effect and check if the data is no longer accessible. Changes typically take effect within minutes, but can take up to 3 hours.
View audit logs for a Cloud KMS key
Before you enable Cloud KMS Data Access audit logs, you should be familiar with Cloud Audit Logs.
Cloud KMS Data Access audit logs show you when Firestore or any other products that are configured to use your CMEK key make encrypt/decrypt calls to Cloud KMS. Firestore does not issue an encrypt/decrypt call on every data request, but instead maintains a poller that checks the key periodically. The polling results appear in the audit logs.
You can set up and interact with the audit logs in the Google Cloud console:
Make sure that logging is enabled for the Cloud KMS API in your project.
Go to Cloud Logging in the Google Cloud console.
Limit the log entries to your Cloud KMS key by adding the following lines to the Query builder:
resource.type="cloudkms_cryptokey" resource.labels.key_ring_id = KMS_KEYRING resource.labels.crypto_key_id = KMS_KEY resource.labels.location=KMS_LOCATION
Provide the following:
KMS_KEY
: the name of the CMEK keyKMS_KEYRING
: the KMS key ring that contains the keyKMS_LOCATION
: the location of the key and key ring
The log shows a couple log entries about every five minutes per database. The log entries look similar to these examples:
Info 2021-03-20 08:02:24.869 EDT Cloudkms.googleapis.com Decrypt projects/cloud-kms-project/locations/us-central1/keyRings/firestore-keys/cryptoKeys/my-cmek-key service-123456789123@gcp-sa-firestore.iam.gserviceaccount.com audit_log, method: "Decrypt", principal_email: "service-1234567891011@gcp-sa-firestore.iam.gserviceaccount.com" Info 2021-03-20 08:02:24.913 EDT Cloudkms.googleapis.com Encrypt projects/cloud-kms-project/locations/us-central1/keyRings/firestore-keys/cryptoKeys/my-cmek-key service-123456789123@gcp-sa-firestore.iam.gserviceaccount.com audit_log, method: "Encrypt", principal_email: "service-123456789123@gcp-sa-firestore.iam.gserviceaccount.com"
See Understanding audit logs for details about interpreting audit logs.
Configure a CMEK organization policy
To specify encryption compliance requirements for Firestore databases in your organization, use a CMEK organization policy constraint.
Require CMEK protection
Configure constraints/gcp.restrictNonCmekServices
to require CMEK for
Firestore database creation. Set the constraint to deny
and
add firestore.googleapis.com
to the deny list, for example:
gcloud resource-manager org-policies deny gcp.restrictNonCmekServices is:firestore.googleapis.com --project=FIRESTORE_PROJECT
Replace FIRESTORE_PROJECT
with the project to restrict.
To learn more about configuring organization policies, see Creating and editing policies.
After the policy takes effect, you receive a FAILED_PRECONDITION
exception and error message if you try to create a non-CMEK database under the affected project. For example, an exception looks like:
{ "error": { "code": 400, "message": "Constraint 'constraints/gcp.restrictNonCmekServices' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'firestore.googleapis.com'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information.", "status": "FAILED_PRECONDITION", "details": [ { "@type": "type.googleapis.com/google.rpc.PreconditionFailure", "violations": [ { "type": "constraints/gcp.restrictNonCmekServices", "subject": "orgpolicy:projects/FIRESTORE_PROJECT", "description": "Constraint 'constraints/gcp.restrictNonCmekServices' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'firestore.googleapis.com'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." } ]
Limit the use of keys for CMEK
To limit which Cloud KMS keys are used for CMEK protection,
configure the constraints/gcp.restrictCmekCryptoKeyProjects
constraint.
As a list constraint, the accepted values are resource hierarchy indicators (for
example, projects/PROJECT_ID
, under:folders/FOLDER_ID
, and
under:organizations/ORGANIZATION_ID
). Use this constraint by configuring a
list of resource hierarchy indicators and setting the constraint to Allow.
This configuration restricts supported services so that CMEK keys can be chosen
only from the listed projects, folders, and organizations. Requests to create
CMEK-protected resources in configured services don't succeed without a
Firestore key from one of the allowed resources.
The following example allows only keys from the ALLOWED_KEY_PROJECT_ID for CMEK-protected databases in the specified project:
gcloud resource-manager org-policies allow gcp.restrictCmekCryptoKeyProjects \ under:projects/ALLOWED_KEY_PROJECT_ID \ --project=FIRESTORE_PROJECT
After the policy takes effect, you receive a FAILED_PRECONDITION
exception
and an error message if you violate the constraint. An exception
looks like the following:
{ "error": { "code": 400, "message": "Constraint 'constraints/gcp.restrictCmekCryptoKeyProjects' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'projects/{NOT_ALLOWED_KEY_PROJECT}'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information.", "status": "FAILED_PRECONDITION", "details": [ { "@type": "type.googleapis.com/google.rpc.PreconditionFailure", "violations": [ { "type": "constraints/gcp.restrictCmekCryptoKeyProjects", "subject": "orgpolicy:projects/FIRESTORE_PROJECT", "description": "Constraint 'constraints/gcp.restrictCmekCryptoKeyProjects' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'projects/{NOT_ALLOWED_KEY_PROJECT}'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." } ] } ] } }