Jaliya's Blog: Copying Azure Cosmos DB NoSQL Containers to a Database in another Account

In this post, let's see how to copy Azure Cosmos DB NoSQL containers from one database to another database in a different Azure Cosmos DB NoSQL account.

Currently within the Azure Portal itself, only Container Copy within the same account is supported.

Container Copy

There are other options like Azure Data Factory etc, but it's much easier to do this using az cosmosdb copy from az cli cosmosdb-preview extension.

Let's have a look.

First step is adding cosmosdb-preview extension.

az extension add --name cosmosdb-preview

Now I am declaring script level variables.

$sourceSubsciptionId = "<sourceSubscriptionId>"
$sourceAccountResourceGroup = "<sourceAccountResourceGroup>"
$sourceAccountName = "<sourceAccountName>"
 
$destinationSubsciptionId = "<destinationSubscriptionId>"
$destinationAccountResourceGroup = "<destinationResourceGroup>"
$destinationAccountName = "<destinationAccountName>"
 
# ManagedIdentity of destination Cosmos DB account
$destinationUserAssignedManagedIdentityName = "<destinationUserAssignedManagedIdentityName>"
 
$destinationUserAssignedManagedIdentityPrincipalId = az identity show `
    --resource-group $destinationAccountResourceGroup `
    --name $destinationUserAssignedManagedIdentityName `
    --query principalId `
    --output tsv
 
$destinationUserAssignedManagedIdentityResourceId = az identity show `
    --resource-group $destinationAccountResourceGroup `
    --name $destinationUserAssignedManagedIdentityName `
    --query id `
    --output tsv

We can copy containers in two modes: Online copy and Offline copy. With Offline copy, we need to stop operations on source container, I don't want to do that. So in this post, we are going to be doing Online copy. I am also using User-Assigned managed identities for database access.

Requirements

Source Account

Enable continuous backup on source Azure Cosmos DB account.
Enable All version and delete change feed mode (preview) feature on the source account.

Features: All version and delete change feed mode (preview)
Add capability: EnableOnlineContainerCopy

# Existing capabilities of your account.
$cosmosdb = az cosmosdb show `
    --resource-group $sourceAccountResourceGroup `
    --name $sourceAccountName
 
$capabilities = (($cosmosdb | ConvertFrom-Json).capabilities)
 
# Append EnableOnlineContainerCopy capability in the list of capabilities.
$capabilitiesToAdd = @()
foreach ($item in $capabilities) {
    $capabilitiesToAdd += $item.name
}
$capabilitiesToAdd += "EnableOnlineContainerCopy"
 
# Update Cosmos DB account
az cosmosdb update `
    --resource-group $sourceAccountResourceGroup
    --name $sourceAccountName `
    --capabilities $capabilitiesToAdd

Verify the capabilities and ensure EnableOnlineContainerCopy is added.

Source Account Capabilities

Destination Account

Currently cross account container copy is only supported for accounts with System-Assigned or User-Assigned default identity. So make sure in the destination database, default identity is set to destination User-Assigned managed identity.

# Show the current default identity of the destination Cosmos DB account
az cosmosdb show `
    --resource-group $destinationAccountResourceGroup `
    --name $destinationAccountName `
    --query defaultIdentity
 
# Update default identity for the destination Cosmos DB account
az cosmosdb update `
    --resource-group $destinationAccountResourceGroup `
    --name $destinationAccountName `
    --default-identity=UserAssignedIdentity=$destinationUserAssignedManagedIdentityResourceId

Now we need to grant the destination Cosmos DB account’s managed identity read-only access to the source Cosmos DB account, so we can read the data.

az account set --subscription $sourceSubsciptionId
 
$roleDefinitionId = "00000000-0000-0000-0000-000000000001" # Read-Only Role Definition Id
az cosmosdb sql role assignment create `
    --resource-group $sourceAccountResourceGroup `
    --account-name $sourceAccountName `
    --role-definition-id $roleDefinitionId `
    --scope "/" `
    --principal-id $destinationUserAssignedManagedIdentityPrincipalId

Now we are all set. Next step is creating a job to copy container.

Create Container Copy Job:

az account set --subscription $destinationSubsciptionId
 
$jobName = "<jobName>"
 
$sourceDatabase = "<sourceDatabase>"
$sourceContainer = "<sourceContainer>"
 
$destinationDatabase = "<destinationDatabase>"
$destinationContainer = "<destinationContainer>"
 
az cosmosdb copy create `
    --resource-group $destinationAccountResourceGroup `
    --job-name $jobName `
    --src-account $sourceAccountName `
    --src-nosql database=$sourceDatabase container=$sourceContainer `
    --dest-account $destinationAccountName `
    --dest-nosql database=$destinationDatabase container=$destinationContainer `
    --mode Online

az cosmosdb copy create

Once it's started, we can query the job status.

Query Job Status:

az cosmosdb copy show `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName `
    --job-name $jobName

az cosmosdb copy show

When the processedCount becomes greater than or equal to the totalCount, complete the job.

Complete the Job:

az cosmosdb copy complete `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName `
    --job-name $jobName

After some time, query the status again and make sure it's Completed.

az cosmosdb copy show

That's it.

Some useful commands

# List all copy jobs
az cosmosdb copy list `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName
 
# Pause copy job
az cosmosdb copy pause `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName `
    --job-name $jobName
 
# Resume copy job
az cosmosdb copy resume `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName `
    --job-name $jobName
 
# Cancel copy job
az cosmosdb copy cancel `
    --resource-group $destinationAccountResourceGroup `
    --account-name $destinationAccountName `
    --job-name $jobName