Support of snapshot copy to primary storage in different zones. #9478

slavkap · 2024-08-01T13:47:53Z

Description

This PR allows copying a snapshot from a primary storage pool in one zone to a primary storage pool in different zones without involving secondary storage.
This feature is currently implemented only in the StorPool storage plugin. Other storage plugins can add this feature if the storage pools support direct copies of snapshots from one pool to another.

Added additional API param usestoragereplication and one more for Admin users storageids in CloudStack API calls:

createSnapshot
copySnapshot
createSnapshotPolicy

The option snapshot.backup.to.secondary does not apply to the copy functionality. The snapshots will be copied only to the required primary storage in a different zone.
The user can create volumes/templates from the copied snapshots. The user can make copies only to a primary or to a secondary storage - at the moment, there is no option to do the copy on both.

The destination zone is a mandatory parameter if the users want to copy a snapshot, and the usestoragereplication if the copy has to be on the primary storage

For Admin users:
Those users can define the primary destination storage for the copy or to enable the setting use.storage.replication in the Primary storage settings.

For regular Users:
The Admin should enable the configuration setting use.storage.replication in the Primary storage settings.

For other storage plugins that want to adopt this functionality:

The Primary storage driver should have the capability CAN_COPY_SNAPSHOT_BETWEEN_ZONES
the respective plugin needs to implement the copySnapshot method in their SnapshotStrategy and that the driver can handle the COPY operation

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale

Major
Minor

Screenshots (if appropriate):

How Has This Been Tested?

Manual and smoke tests with StorPool primary storage on multiple zones

codecov · 2024-08-01T13:56:23Z

Codecov Report

❌ Patch coverage is 6.54971% with 799 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.15%. Comparing base (8429223) to head (4c91d28).
⚠️ Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
...om/cloud/storage/snapshot/SnapshotManagerImpl.java	8.08%	174 Missing and 8 partials ⚠️
...ack/storage/snapshot/StorPoolSnapshotStrategy.java	0.00%	154 Missing ⚠️
...age/collector/StorPoolAbandonObjectsCollector.java	0.00%	69 Missing ⚠️
...tastore/driver/StorPoolPrimaryDataStoreDriver.java	0.00%	56 Missing ⚠️
...org/apache/cloudstack/snapshot/SnapshotHelper.java	16.94%	43 Missing and 6 partials ⚠️
...loudstack/storage/datastore/util/StorPoolUtil.java	0.00%	37 Missing ⚠️
...n/java/com/cloud/storage/VolumeApiServiceImpl.java	20.51%	29 Missing and 2 partials ⚠️
...oudstack/storage/snapshot/SnapshotServiceImpl.java	0.00%	27 Missing ⚠️
...e/wrapper/StorPoolModifyStorageCommandWrapper.java	0.00%	21 Missing ⚠️
...udstack/storage/datastore/util/StorPoolHelper.java	0.00%	21 Missing ⚠️
... and 25 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #9478       +/-   ##
===========================================
+ Coverage        0   17.15%   +17.15%     
- Complexity      0    14998    +14998     
===========================================
  Files           0     5869     +5869     
  Lines           0   522449   +522449     
  Branches        0    63624    +63624     
===========================================
+ Hits            0    89618    +89618     
- Misses          0   422752   +422752     
- Partials        0    10079    +10079

Flag	Coverage Δ
uitests	`3.75% <ø> (?)`
unittests	`18.13% <6.54%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2024-08-21T08:21:53Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

DaanHoogland · 2024-08-22T13:48:15Z

@blueorangutan package

blueorangutan · 2024-08-22T13:50:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-08-22T15:24:36Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10756

DaanHoogland · 2024-08-23T07:43:27Z

@blueorangutan test

blueorangutan · 2024-08-23T07:44:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-08-23T22:40:23Z

[SF] Trillian test result (tid-11155)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 51322 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9478-t11155-kvm-ol8.zip
Smoke tests completed. 137 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_snapshot_usage	`Failure`	38.32	test_usage.py
test_01_vpc_site2site_vpn	`Failure`	308.08	test_vpc_vpn.py

github-actions · 2024-08-29T08:41:48Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

api/src/main/java/org/apache/cloudstack/api/command/user/snapshot/CopySnapshotCmd.java

.../src/main/java/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreCapabilities.java

api/src/main/java/org/apache/cloudstack/api/command/user/snapshot/CopySnapshotCmd.java

github-actions · 2024-09-04T08:40:33Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

.../src/main/java/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreCapabilities.java

github-actions · 2025-07-29T14:59:04Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…n zones Added support to copy a snapshot to another StorPool primary storage in different zones.

Added drop down to choose the primary storage pools to copy a snapshot Small fixes

hide the primary storage from the users in the UI refactor smoke test

fix copy when reccuring snapshot fix UI after rebasing

Pool type to volumes that are created from snapshots Added StorPool tags to snapshots that are copied from remote

blueorangutan · 2025-07-29T15:16:21Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14444

slavkap · 2025-07-29T15:19:09Z

@blueorangutan package

blueorangutan · 2025-07-29T15:20:05Z

@slavkap a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-07-29T15:28:25Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 14447

sureshanaparti · 2025-07-30T05:07:02Z

@blueorangutan package

blueorangutan · 2025-07-30T05:08:03Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-07-30T07:32:48Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 14460

sureshanaparti · 2025-07-30T07:39:31Z

@blueorangutan test

blueorangutan · 2025-07-30T07:40:03Z

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

sureshanaparti

clgtm

boring-cyborg bot added component:api component:ceph component:orchestration component:primary-storage component:storage labels Aug 1, 2024

slavkap force-pushed the support-snapshot-copy-on-primary branch from 8cc7b49 to 0df6764 Compare August 14, 2024 09:14

slavkap marked this pull request as ready for review August 14, 2024 09:51

slavkap changed the title ~~[WIP] Support of snapshot copy to StorPool primary storage in different zones~~ Support of snapshot copy to StorPool primary storage in different zones Aug 14, 2024

github-actions bot added the status:has-conflicts label Aug 21, 2024

slavkap force-pushed the support-snapshot-copy-on-primary branch from 0df6764 to e0f4283 Compare August 22, 2024 12:49

github-actions bot removed the status:has-conflicts label Aug 22, 2024

github-actions bot added the status:has-conflicts label Aug 29, 2024

slavkap force-pushed the support-snapshot-copy-on-primary branch from e0f4283 to 6f209d8 Compare August 30, 2024 09:08

github-actions bot removed the status:has-conflicts label Aug 30, 2024

sureshanaparti reviewed Aug 30, 2024

View reviewed changes

api/src/main/java/org/apache/cloudstack/api/command/user/snapshot/CopySnapshotCmd.java Show resolved Hide resolved

sureshanaparti reviewed Aug 30, 2024

View reviewed changes

.../src/main/java/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreCapabilities.java Outdated Show resolved Hide resolved

sureshanaparti reviewed Aug 30, 2024

View reviewed changes

api/src/main/java/org/apache/cloudstack/api/command/user/snapshot/CopySnapshotCmd.java Outdated Show resolved Hide resolved

rohityadavcloud added this to the 4.20.0.0 milestone Sep 4, 2024

github-actions bot added the status:has-conflicts label Sep 4, 2024

sureshanaparti reviewed Sep 4, 2024

View reviewed changes

.../src/main/java/org/apache/cloudstack/engine/subsystem/api/storage/DataStoreCapabilities.java Outdated Show resolved Hide resolved

slavkap added 14 commits July 29, 2025 18:11

Support of snapshot copy to different StorPool primary storage betwee…

6b89a33

…n zones Added support to copy a snapshot to another StorPool primary storage in different zones.

added option in the UI for the storage pools

240cea2

Added drop down to choose the primary storage pools to copy a snapshot Small fixes

Addressed comments

a9f3270

Address reviews

054ef2d

fix snapshot deletion

a587f92

Address comments and fix conflicts

3e807d5

small rearranges

c4dad60

removed unused imports

6ae7a1e

hide the primary storage when a user copies the snapshots

01484d7

hide the primary storage from users in UI

4077614

hide the primary storage from the users in the UI refactor smoke test

Addressed reviews

0b0a065

Fix of publish event for deleted snapshot

81dbc3c

small fixes

da2a083

fix copy when reccuring snapshot fix UI after rebasing

Added pool type to volumes created from snapshots

4c91d28

Pool type to volumes that are created from snapshots Added StorPool tags to snapshots that are copied from remote

slavkap force-pushed the support-snapshot-copy-on-primary branch from edf4397 to 4c91d28 Compare July 29, 2025 15:12

github-actions bot removed the status:has-conflicts label Jul 29, 2025

sureshanaparti closed this Jul 30, 2025

github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.21.0 Jul 30, 2025

sureshanaparti reopened this Jul 30, 2025

sureshanaparti approved these changes Jul 30, 2025

View reviewed changes

Support of snapshot copy to primary storage in different zones. #9478

Are you sure you want to change the base?

Support of snapshot copy to primary storage in different zones. #9478

Uh oh!

Conversation

slavkap commented Aug 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

codecov bot commented Aug 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Aug 21, 2024

Uh oh!

DaanHoogland commented Aug 22, 2024

Uh oh!

blueorangutan commented Aug 22, 2024

Uh oh!

blueorangutan commented Aug 22, 2024

Uh oh!

DaanHoogland commented Aug 23, 2024

Uh oh!

blueorangutan commented Aug 23, 2024

Uh oh!

blueorangutan commented Aug 23, 2024

Uh oh!

github-actions bot commented Aug 29, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2024

Uh oh!

Uh oh!

github-actions bot commented Jul 29, 2025

Uh oh!

blueorangutan commented Jul 29, 2025

Uh oh!

slavkap commented Jul 29, 2025

Uh oh!

blueorangutan commented Jul 29, 2025

Uh oh!

blueorangutan commented Jul 29, 2025

Uh oh!

sureshanaparti commented Jul 30, 2025

Uh oh!

blueorangutan commented Jul 30, 2025

Uh oh!

blueorangutan commented Jul 30, 2025

Uh oh!

sureshanaparti commented Jul 30, 2025

Uh oh!

blueorangutan commented Jul 30, 2025

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

slavkap commented Aug 1, 2024 •

edited

Loading

codecov bot commented Aug 1, 2024 •

edited

Loading