added online/offline copy method for Primera storage adapter #11298

shrikantjoshi-hpe · 2025-07-25T19:49:59Z

Description

Enhanced the Primera storage adapter's volume copy functionality to implement copy method selection based on volume size requirements. The adapter now automatically determines whether to use online copy (fast clone) or offline copy to optimize performance and resource utilization.

Key Changes:

• Added automatic copy method detection based on source and target volume size comparison
• Implemented online copy for same-size volumes (immediate cloning with deduplication)
• Implemented offline copy for volume resize operations (background task with high priority)
• Enhanced logging with detailed copy method selection rationale and operation progress
• Added proper task monitoring and completion handling for offline copy operations
• Optimized copy parameters (provisioning type settings)

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

N/A - Backend storage adapter enhancement

How Has This Been Tested?

Copy Operation Test Scenarios:

Same-Size Volume Copy (Online Copy Path):
• Verified fast cloning with immediate completion
• Confirmed deduplication and compression enabled
Different-Size Volume Copy (Offline Copy Path):
• Tested volume resize during copy operations
• Verified background task creation with high priority
• Confirmed proper target volume pre-creation with correct size
• Validated task completion monitoring and timeout handling
Copy Method Selection Logic:
• Tested size comparison logic (newSize vs sourceSize)
• Verified online copy selection when sizes match exactly
• Confirmed offline copy selection when sizes differ
Copy Configuration Validation:
• Verified online copy parameters
• Verified offline copy parameters, background execution
• Tested proper request parameter assignment
Error Handling and Logging
• Verified detailed logging for copy method selection decisions

How did you try to break this feature and the system with this change?

Copy Operation Stress Testing
• Initiated multiple concurrent copy operations to test Primera API limits
• Tested copy operations with very large volumes (>100GB)

Edge Cases and Boundary Conditions:
• Tested with minimum volume sizes

Integration Testing:
• Tested copy operations within CloudStack volume lifecycle (templates)

The enhancement maintains full backward compatibility while providing significant performance improvements for volume copy operations through method selection.

sureshanaparti

clgtm

sureshanaparti · 2025-07-26T06:49:06Z

@blueorangutan package

blueorangutan · 2025-07-26T06:50:03Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-07-26T06:53:30Z

Codecov Report

❌ Patch coverage is 0% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 16.76%. Comparing base (0d4147f) to head (5622a9c).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...rage/datastore/adapter/primera/PrimeraAdapter.java	0.00%	33 Missing ⚠️
.../datastore/driver/AdaptiveDataStoreDriverImpl.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11298      +/-   ##
============================================
- Coverage     16.76%   16.76%   -0.01%     
+ Complexity    14285    14283       -2     
============================================
  Files          5767     5767              
  Lines        512657   512685      +28     
  Branches      62359    62363       +4     
============================================
- Hits          85950    85942       -8     
- Misses       417108   417145      +37     
+ Partials       9599     9598       -1

Flag	Coverage Δ
uitests	`3.85% <ø> (ø)`
unittests	`17.68% <0.00%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull Request Overview

This PR enhances the Primera storage adapter to implement intelligent volume copy method selection based on size requirements. The adapter now automatically chooses between online copy (fast clone) for same-size volumes and offline copy for resize operations.

Added automatic copy method detection that compares source and target volume sizes
Implemented online copy for same-size volumes using immediate cloning with deduplication
Implemented offline copy for volume resize operations with background task monitoring

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
PrimeraAdapter.java	Enhanced copy method with size-based online/offline copy selection and detailed logging
FlashArrayAdapter.java	Updated method signature to include newSize parameter (not yet implemented)
AdaptiveDataStoreDriverImpl.java	Modified to pass destination size to adapter copy method
ProviderAdapter.java	Updated interface to include newSize parameter in copy method signature

Copilot · 2025-07-26T07:28:58Z

...ra/src/main/java/org/apache/cloudstack/storage/datastore/adapter/primera/PrimeraAdapter.java

+                        targetVolumeInfo.getName(), sourceVolume.getAllocatedSizeInBytes());
+                this.create(context, targetVolumeInfo, null, sourceVolume.getAllocatedSizeInBytes());


For offline copy mode, the target volume is being created with the source volume size instead of the newSize parameter. This contradicts the logic that offline copy is used when sizes differ, and the target should be created with the desired newSize.

Suggested change

targetVolumeInfo.getName(), sourceVolume.getAllocatedSizeInBytes());

this.create(context, targetVolumeInfo, null, sourceVolume.getAllocatedSizeInBytes());

targetVolumeInfo.getName(), newSize);

this.create(context, targetVolumeInfo, null, newSize);

Copilot · 2025-07-26T07:28:58Z

...ra/src/main/java/org/apache/cloudstack/storage/datastore/adapter/primera/PrimeraAdapter.java

+        if (!onlineCopy) {
+            // Offline copy requires waiting for task completion
+            logger.debug("PrimeraAdapter: Offline copy initiated - waiting for task completion (TaskID: {})", taskref.getTaskid());
+            waitForTaskToComplete(taskref.getTaskid(), "copy volume " + sourceVolumeInfo.getExternalName() + " to " +
+                targetVolumeInfo.getExternalName(), taskWaitTimeoutMs);
+            logger.debug("PrimeraAdapter: Offline copy operation completed successfully");
+        } else {


[nitpick] The waitForTaskToComplete call is only executed for offline copy, but it should be within the offline copy conditional block for better code organization and readability.

Suggested change

if (!onlineCopy) {

// Offline copy requires waiting for task completion

logger.debug("PrimeraAdapter: Offline copy initiated - waiting for task completion (TaskID: {})", taskref.getTaskid());

waitForTaskToComplete(taskref.getTaskid(), "copy volume " + sourceVolumeInfo.getExternalName() + " to " +

targetVolumeInfo.getExternalName(), taskWaitTimeoutMs);

logger.debug("PrimeraAdapter: Offline copy operation completed successfully");

} else {

if (onlineCopy) {

Copilot · 2025-07-26T07:28:58Z

.../main/java/org/apache/cloudstack/storage/datastore/adapter/flasharray/FlashArrayAdapter.java

@@ -367,7 +367,8 @@ public ProviderSnapshot getSnapshot(ProviderAdapterContext context, ProviderAdap

    @Override
    public ProviderVolume copy(ProviderAdapterContext context, ProviderAdapterDataObject sourceDataObject,
-            ProviderAdapterDataObject destDataObject) {
+            ProviderAdapterDataObject destDataObject, Long newSize) {
+        // Add new parameter as newSize to match method declaration but not used anywhere


[nitpick] The comment indicates the newSize parameter is not implemented yet. Consider adding a TODO comment or proper documentation about the planned implementation to make the intention clearer for future development.

Suggested change

// Add new parameter as newSize to match method declaration but not used anywhere

// TODO: The newSize parameter is currently unused. It may be implemented in the future

// to allow resizing the destination volume during the copy operation.

blueorangutan · 2025-07-26T08:12:17Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14386

sureshanaparti · 2025-07-26T08:30:43Z

@blueorangutan test

blueorangutan · 2025-07-26T08:32:04Z

@sureshanaparti a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

added online/offline copy method for Primera storage adapter

5622a9c

boring-cyborg bot added component:primary-storage component:storage labels Jul 25, 2025

sureshanaparti approved these changes Jul 26, 2025

View reviewed changes

sureshanaparti added this to the 4.21.0 milestone Jul 26, 2025

sureshanaparti added this to Apache CloudStack 4.21.0 Jul 26, 2025

sureshanaparti moved this to In Progress in Apache CloudStack 4.21.0 Jul 26, 2025

sureshanaparti requested review from DaanHoogland and Copilot July 26, 2025 07:28

Copilot AI reviewed Jul 26, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

added online/offline copy method for Primera storage adapter #11298

added online/offline copy method for Primera storage adapter #11298

shrikantjoshi-hpe commented Jul 25, 2025

Uh oh!

sureshanaparti left a comment

Uh oh!

sureshanaparti commented Jul 26, 2025

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

codecov bot commented Jul 26, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jul 26, 2025

Uh oh!

Copilot AI Jul 26, 2025

Uh oh!

Copilot AI Jul 26, 2025

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

sureshanaparti commented Jul 26, 2025

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

		targetVolumeInfo.getName(), sourceVolume.getAllocatedSizeInBytes());
		this.create(context, targetVolumeInfo, null, sourceVolume.getAllocatedSizeInBytes());

	// Add new parameter as newSize to match method declaration but not used anywhere
	// TODO: The newSize parameter is currently unused. It may be implemented in the future
	// to allow resizing the destination volume during the copy operation.

added online/offline copy method for Primera storage adapter #11298

Are you sure you want to change the base?

added online/offline copy method for Primera storage adapter #11298

Conversation

shrikantjoshi-hpe commented Jul 25, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

sureshanaparti commented Jul 26, 2025

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

codecov bot commented Jul 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 26, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 26, 2025

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

sureshanaparti commented Jul 26, 2025

Uh oh!

blueorangutan commented Jul 26, 2025

Uh oh!

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

codecov bot commented Jul 26, 2025 •

edited

Loading