Create and manage snapshots in Aiven for OpenSearch®
Create, list, retrieve, or delete snapshots in your Aiven for OpenSearch custom repositories.
Prerequisites
- Aiven Console
- Aiven API
- OpenSearch API
- Running Aiven for OpenSearch service
- Access to the Aiven Console
- Access to a supported object storage service (AWS S3, GCS, or Azure)
- Credentials for the selected storage provider
- Configured custom repository
- Running Aiven for OpenSearch service
- Aiven API and authentication token
- Access to a supported object storage service (AWS S3, GCS, or Azure)
- Credentials for the selected storage provider
- Configured custom repository
- Configured custom repository
- Maintenance updates applied for your service
- Security management enabled for your service
- Snapshot permissions and snapshot repository permissions configured
Automatic snapshot scheduling is not supported. Create, list, and delete snapshots manually.
Limitations
- Aiven Console
- Aiven API
- OpenSearch API
Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
- Supported storage services are:
- Amazon S3
- Google Cloud Storage (GCS)
- Microsoft Azure Blob Storage
- Restore from snapshot has a couple of security-related restrictions.
- Create a snapshot
and
delete a snapshot
are not supported for snapshots in Aiven-managed repositories (prefixed with
aiven_repo
).
Create a snapshot
Create a snapshot in a custom repository.
- Aiven Console
- Aiven API
- OpenSearch API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, do one of the following:
- Click Create snapshot.
- Find your custom repository and click Actions > Create snapshot.
- In the Create snapshot window:
- Select a destination repository.
- Enter a snapshot name.
- Specify which indices to include.
- Optionally, enable the following:
- Ignore unavailable indices
- Include global state
- Partial snapshot
- Click Create.
Your snapshot is being created. Monitor its status until it shows Success.
curl -s -X POST \
--url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/{repository_name}/{snapshot_name}/_restore" \
--header "Authorization: Bearer $TOKEN" \
--header "Content-Type: application/json" \
-d '{"indices": "test*"}'
Example response:
{
"accepted": true
}
Use the Create Snapshot native OpenSearch API endpoint.
Restore from snapshots
Refrain from actions such as updating firewalls, changing index settings, or modifying security configurations during the restore process as it can cause restore failures.
- Aiven Console
- Aiven API
- OpenSearch API
-
Log in to the Aiven Console, go to your project, and open your service's page.
-
Click Snapshots in the sidebar.
-
On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
-
Find the snapshot to restore and click Actions > Restore to this service.
-
In the Restore snapshot window:
-
In the Indices field, enter the indices to include in the snapshot, separated by commas. These indices will be closed automatically before the restore begins.
Existing data may be overwritten as a result of the restoration process. To prevent this, use advanced configuration to add suffixes to the included indices.
-
Toggle Enable advanced configuration to set the following:
- Rename pattern: a regular expression that matches the original index name from the snapshot
- Rename replacement: a string that replaces the matched part of the index name
- Ignore unavailable indices
- Include aliases
-
Click Continue > Close indices.
This triggers the closing of the selected indices. Wait for Indices closed to be displayed, and click Continue.
-
Check the box labeled I understand the effects of this action.
warningThe restoration process you're about to start cannot be interrupted. Refrain from changes during this process: Updating firewalls, index settings, or security configuration during restore may cause failures.
-
Click Start restore.
This triggers the restoration process. It may take time, and it's length depends on the snapshot size.
-
Shut down the Restore snapshot window by clicking Close either during the restore process or when it completes.
tipIf you close the Restore snapshot window before the restore process is complete, you can preview its status on the Snapshots page.
-
curl -s -X POST \
"https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/{repository_name}/{snapshot_name}/_restore" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"indices": "test-index-*",
"include_aliases": true,
"ignore_unavailable": false,
"rename_pattern": "index_(.+)",
"rename_replacement": "restored_index_$1"
}'
Example response:
{
"accepted": true
}
To restore data from a snapshot, use the Restore Snapshot native OpenSearch API endpoint.
List snapshots in progress
Preview snapshots that are still being created in a repository.
- Aiven Console
- Aiven API
- OpenSearch API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
- In the column with snapshot status information, find in progress values.
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/_status" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"repository": "aws-repo",
"snapshot": "second-snapshot",
"state": "SUCCESS",
"shards_stats": {
"done": 1,
"failed": 0,
"total": 1
},
"uuid": "osmCbdF-RMyyUKpWD-4bJA"
}
]
}
Use one of the following variants of the native OpenSearch Get Snapshot Status API:
- For snapshot statuses in a specified repository:
GET /_snapshot/REPOSITORY_NAME/_status
- For snapshot statuses in all repositories:
GET /_snapshot/_status
List snapshots in a repository
Preview all snapshots, including completed and failed ones.
- Aiven Console
- Aiven API
- OpenSearch API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository and click to expand the list of snapshots inside.
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/_all" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"snapshot": "first-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "7cdWedW7RC6FMSktlZTCDw"
},
{
"snapshot": "second-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "osmCbdF-RMyyUKpWD-4bJA"
}
]
}
Use native OpenSearch API endpoint GET /_snapshot/REPOSITORY_NAME/_all
, replacing the
REPOSITORY_NAME
with the actual name of your repository.
Example response:
{
"snapshots" : [
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-infrequent",
"uuid" : "-abCabCabC-abCabCabCab",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.619Z",
"start_time_in_millis" : 1234567891234,
"end_time" : "YYY-MM-DDTHH:MM:SS.624Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
},
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-frequent",
"uuid" : "-abCabCabC-abCabCabCab",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.219Z",
"start_time_in_millis" : 12345678912345,
"end_time" : "YYY-MM-DDTHH:MM:SS.220Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
},
{
"snapshot" : "opensearch-123qaz456wsx789edc123q-frequent",
"uuid" : "-abCabCabC-abCabCabCabQ",
"version_id" : 123456789,
"version" : "N.NN.N",
"remote_store_index_shallow_copy" : false,
"indices" : [
".plugins-ml-config",
".opensearch-sap-log-types-config",
".kibana_N",
".opensearch-observability"
],
"data_streams" : [ ],
"include_global_state" : true,
"state" : "SUCCESS",
"start_time" : "YYY-MM-DDTHH:MM:SS.088Z",
"start_time_in_millis" : 12345678912345,
"end_time" : "YYY-MM-DDTHH:MM:SS.890Z",
"end_time_in_millis" : 1234567891234,
"duration_in_millis" : 1234,
"failures" : [ ],
"shards" : {
"total" : 4,
"failed" : 0,
"successful" : 4
}
}
]
}
View snapshot details
Get details of a specific snapshot.
- Aiven Console
- Aiven API
- OpenSearch API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository, click to expand the list of snapshots inside, find a snapshot to be previewed, and click Actions > View snapshot details.
curl -s --url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/first-snapshot" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"snapshots": [
{
"snapshot": "first-snapshot",
"state": "SUCCESS",
"indices": ["test"],
"uuid": "7cdWedW7RC6FMSktlZTCDw"
}
]
}
Use the native OpenSearch API endpoints: Get Snapshot or Get Snapshot Status.
Delete a snapshot
Delete a snapshot from a repository.
- Aiven Console
- Aiven API
- OpenSearch API
- Log in to the Aiven Console, go to your project, and open your service's page.
- Click Snapshots in the sidebar.
- On the Snapshots page, find your custom repository, click to expand the list of snapshots inside, find a snapshot to be deleted, click Actions > Delete snapshot > Delete.
curl -s -X DELETE \
--url "https://api.aiven.io/v1/project/{project_name}/service/{service_name}/opensearch/_snapshot/aws-repo/first-snapshot" \
--header "Authorization: Bearer $TOKEN"
--header "Content-Type: application/json"
Example response:
{
"acknowledged": true
}
Use the Delete Snapshot native OpenSearch API endpoint.
Error handling
The Aiven API returns OpenSearch errors as they are.
Exceptions:
- 502: OpenSearch did not respond.
- 409: The service is not powered on or does not support this feature.
Related pages