Importing/Exporting Google Cloud Datastore between Cross-Projects

This post explains how to copy a Datastore entity from Project A to Project B in Google Cloud Platform.

The steps needed to complete this task are the following:

• Creating a Cloud Storage bucket in Project A
• Adding permissions to the service account that will perform the operation. I will be using my AppEngine service account (PROJECTID@appspot.gserviceaccount.com)
• Using Google Cloud Shell or Google SDK to run the commands

Creating a Cloud Storage Bucket

Make sure you select Project A (source project for Datastore to be copied) in Google Cloud Platform. Create a new Cloud Storage bucket and give it a name, I called mine backups. . You will need a Cloud Storage bucket because the process of exporting the Datastore entit(y/ies) requires a destination for dumping the entit(y/ies) files. It is recommended to use Cloud Storage as the destination.

Permissions for Export

Make sure the service account you will use for exporting have the following permissions in Project A by setting them in the IAM & admin section of the Google Cloud Platform console.

• Storage Admin
• Cloud Datastore Import Export Admin

Permissions for Import

Next, assign the service account you will use for importing the cloud storage file with the following permissions in Project B, by setting them in the IAM & admin section of the Google Cloud Platform console.

• Storage Object Viewer
• Cloud Datastore Import Export Admin

Exporting Data

Now you are ready to export your Datastore entity from Project A -> Project A’s Cloud Storage -> Project B.

Run the following command on Google Cloud Shell or your own CLI if you have Google SDK installed to export in Project A:

gcloud datastore export --kinds="<KIND1>" gs://<BUCKETNAME>

replace <BUCKETNAME> with the name of your bucket in my example it is backups.

Once the command runs you can view your bucket and you will find files have been created from the export.

Importing Data

Next run the import command in Project B:

gcloud datastore import gs://<BUCKETNAME>/<PATH>/<FILE>.overall_export_metadata

For more information visit the official Google Cloud documentation here: Exporting and Importing Entities