hal/riscv-rvv: implement FAST keypoint detection #27391
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 11 commits
March 20, 2025 22:40
asmorkalov
requested changes
Jun 2, 2025
| @param threshold Threshold for keypoint | ||
| @param nonmax_suppression Indicates if make nonmaxima suppression or not. | ||
| @param type FAST type | ||
| */ | ||
| inline int hal_ni_FAST(const uchar* src_data, size_t src_step, int width, int height, uchar* keypoints_data, size_t* keypoints_count, int threshold, bool nonmax_suppression, int /*cv::FastFeatureDetector::DetectorType*/ type) { return CV_HAL_ERROR_NOT_IMPLEMENTED; } | ||
| inline int hal_ni_FAST(const uchar* src_data, size_t src_step, int width, int height, std::vector<cv::KeyPoint>& keypoints, int threshold, bool nonmax_suppression, int /*cv::FastFeatureDetector::DetectorType*/ type) { return CV_HAL_ERROR_NOT_IMPLEMENTED; } |
There was a problem hiding this comment.
It's dangerous replacement. HAL uses C style API without std::vector on purpose. HAL may be a binary built with own STL. It means that std::vector on HAL side and OpenCV side may be different with different memory layout.
There was a problem hiding this comment.
Indeed. But I haven't figured out a way to modify the member size in std::vector<cv::KeyPoint>& keypoints by simply reinterpreting keypoints.data() to uchar*. The original CALL_HAL, causes wrong keypoint count in accuracy test since it compares the member size of the output std::vector<cv::KeyPoint>. Is there any solution for the problem?
There was a problem hiding this comment.
A simple solution is that you can extract the macro of calling HAL, manually call HAL function and do a resize on the vector before returning. E.g.
diff --git a/modules/features2d/src/fast.cpp b/modules/features2d/src/fast.cpp
index a9615da5bd..b00b80818d 100644
--- a/modules/features2d/src/fast.cpp
+++ b/modules/features2d/src/fast.cpp
@@ -438,8 +438,12 @@ void FAST(InputArray _img, std::vector<KeyPoint>& keypoints, int threshold, bool
size_t keypoints_count = 10000;
keypoints.clear();
keypoints.resize(keypoints_count);
- CALL_HAL(fast, cv_hal_FAST, img.data, img.step, img.cols, img.rows,
- (uchar*)(keypoints.data()), &keypoints_count, threshold, nonmax_suppression, type);
+ int hal_ret = cv_hal_FAST(img.data, img.step, img.cols, img.rows, (uchar *)(keypoints.data()),
+ &keypoints_count, threshold, nonmax_suppression, type);
+ if (hal_ret == CV_HAL_ERROR_OK) {
+ keypoints.resize(keypoints_count);
+ return;
+ }
switch(type) {
case FastFeatureDetector::TYPE_5_8:64415e0 to
2ed698e
Compare
fengyuentau
reviewed
Jun 3, 2025
There was a problem hiding this comment.
Also take a look at the trailing whitespaces:
hal/riscv-rvv/CMakeLists.txt:20: trailing whitespace.
+ ${CMAKE_SOURCE_DIR}/modules/imgproc/include
hal/riscv-rvv/src/features2d/fast.cpp:60: trailing whitespace.
+inline uint8_t cornerScore(const uint8_t* ptr, const vuint16m2_t& v_offset, int64_t row_stride)
hal/riscv-rvv/src/features2d/fast.cpp:64: trailing whitespace.
+
hal/riscv-rvv/src/features2d/fast.cpp:76: trailing whitespace.
+
hal/riscv-rvv/src/features2d/fast.cpp:108: trailing whitespace.
+inline int fast_16(const uchar* src_data, size_t src_step, int width, int height, std::vector<KeyPoint>& keypoints, int threshold, bool nonmax_suppression)
hal/riscv-rvv/src/features2d/fast.cpp:[19](https://github.com/opencv/opencv/actions/runs/15389955155/job/43296989535?pr=27391#step:14:20)0: trailing whitespace.
+
hal/riscv-rvv/src/features2d/fast.cpp:[20](https://github.com/opencv/opencv/actions/runs/15389955155/job/43296989535?pr=27391#step:14:21)7: trailing whitespace.
fengyuentau
reviewed
Jun 3, 2025
added 2 commits
June 3, 2025 23:04
fengyuentau
reviewed
Jun 4, 2025
fengyuentau
reviewed
Jun 5, 2025
fengyuentau
reviewed
Jun 9, 2025
fengyuentau
reviewed
Jun 10, 2025
There was a problem hiding this comment.
Checked on my side and found more warnings
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/3rdparty/libtiff/tif_hash_set.c:370:37: note: earlier argument should specify number of elements, later size of each element
[352/653] Building CXX object hal/riscv-rvv/CMakeFiles/rvv_hal.dir/src/features2d/fast.cpp.o
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp: In function 'void cv::rvv_hal::features2d::makeOffsets(int16_t*, vuint16m2_t&, int64_t, int)':
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp:11:89: warning: unused parameter 'patternSize' [-Wunused-parameter]
11 | inline void makeOffsets(int16_t pixel[], vuint16m2_t& v_offset, int64_t row_stride, int patternSize)
| ~~~~^~~~~~~~~~~
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp: In function 'uint8_t cv::rvv_hal::features2d::cornerScore(const uint8_t*, const vuint16m2_t&, int64_t)':
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp:48:14: warning: unused variable 'k' [-Wunused-variable]
48 | uint32_t k, v = ptr[0];
| ^
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp: In function 'int cv::rvv_hal::features2d::fast_16(const uchar*, size_t, int, int, uchar*, size_t*, int, bool)':
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp:211:38: warning: unused variable 'debug' [-Wunused-variable]
211 | bool debug = false;
| ^~~~~
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp:212:37: warning: unused variable 'debug_x' [-Wunused-variable]
212 | int debug_x = -1;
| ^~~~~~~
/home/tao/workspace/fytao/k1/worktree/qhx-hal_rvv-features2d-fast/opencv/hal/riscv-rvv/src/features2d/fast.cpp:213:37: warning: unused variable 'debug_y' [-Wunused-variable]
213 | int debug_y = -1;
|
fengyuentau
reviewed
Jun 10, 2025
There was a problem hiding this comment.
My performance results:
K1 GCC:
Name of Test base-gcc patch-gcc patch-gcc
vs
base-gcc
(x-factor)
detect::Fast_Params::(20, 2, false, "cv/cameracalibration/chess9.png") 22.308 23.161 0.96
detect::Fast_Params::(20, 2, false, "cv/inpaint/orig.png") 4.816 6.888 0.70
detect::Fast_Params::(20, 2, true, "cv/cameracalibration/chess9.png") 26.150 24.789 1.05
detect::Fast_Params::(20, 2, true, "cv/inpaint/orig.png") 7.114 7.497 0.95
detect::Fast_Params::(30, 2, false, "cv/cameracalibration/chess9.png") 19.625 21.090 0.93
detect::Fast_Params::(30, 2, false, "cv/inpaint/orig.png") 3.572 5.179 0.69
detect::Fast_Params::(30, 2, true, "cv/cameracalibration/chess9.png") 22.323 22.283 1.00
detect::Fast_Params::(30, 2, true, "cv/inpaint/orig.png") 4.873 5.627 0.87
detect::Fast_Params::(100, 2, false, "cv/cameracalibration/chess9.png") 14.156 8.319 1.70
detect::Fast_Params::(100, 2, false, "cv/inpaint/orig.png") 2.582 1.170 2.21
detect::Fast_Params::(100, 2, true, "cv/cameracalibration/chess9.png") 14.257 8.521 1.67
detect::Fast_Params::(100, 2, true, "cv/inpaint/orig.png") 2.618 1.192 2.20
K1 Clang:
Name of Test base-clang patch-clang patch-clang
vs
base-clang
(x-factor)
detect::Fast_Params::(20, 2, false, "cv/cameracalibration/chess9.png") 25.594 23.853 1.07
detect::Fast_Params::(20, 2, false, "cv/inpaint/orig.png") 5.453 7.196 0.76
detect::Fast_Params::(20, 2, true, "cv/cameracalibration/chess9.png") 29.278 24.837 1.18
detect::Fast_Params::(20, 2, true, "cv/inpaint/orig.png") 7.782 7.888 0.99
detect::Fast_Params::(30, 2, false, "cv/cameracalibration/chess9.png") 23.346 21.688 1.08
detect::Fast_Params::(30, 2, false, "cv/inpaint/orig.png") 4.332 5.609 0.77
detect::Fast_Params::(30, 2, true, "cv/cameracalibration/chess9.png") 25.912 22.294 1.16
detect::Fast_Params::(30, 2, true, "cv/inpaint/orig.png") 5.524 5.957 0.93
detect::Fast_Params::(100, 2, false, "cv/cameracalibration/chess9.png") 17.603 8.718 2.02
detect::Fast_Params::(100, 2, false, "cv/inpaint/orig.png") 3.256 1.216 2.68
detect::Fast_Params::(100, 2, true, "cv/cameracalibration/chess9.png") 17.665 8.637 2.05
detect::Fast_Params::(100, 2, true, "cv/inpaint/orig.png") 3.267 1.222 2.67
K1 vs. RK3568:
TBD
asmorkalov
reviewed
Jun 16, 2025
|
Muse Pi v30 board (./opencv_test_features2d): |
I've checked this test. This cause of this failure is that the count of keypoints in this case is greater than 10000 which is set in the initialization in |
|
Looks like we need some API change here. |
vpisarev
reviewed
Jun 26, 2025
modules/features2d/src/fast.cpp
Outdated
| keypoints.resize(keypoints_count); | ||
| CALL_HAL(fast, cv_hal_FAST, img.data, img.step, img.cols, img.rows, | ||
| (uchar*)(keypoints.data()), &keypoints_count, threshold, nonmax_suppression, type); | ||
| uchar* kps = (uchar*)malloc(sizeof(KeyPoint) * keypoints_count); |
There was a problem hiding this comment.
you don't need extra variable. Use the following code instead:
KeyPoint* kps = (KeyPoint*)malloc(sizeof(kps[0]) * keypoints_count);
int hal_ret = cv_hal_FAST(img.data, img.step, img.cols, img.rows, (void**)&kps,
&keypoints_count, threshold, nonmax_suppression, type, realloc);
if (hal_ret == CV_HAL_ERROR_OK) {
keypoints.assign(kps, kps + keypoints_count);
}
free(kps);
if (hal_ret == CV_HAL_ERROR_OK) {
return;
}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An implementation of FAST keypoint detection with NMS/noNMS version.
A new perf test is written, and the perf test is evaluated in two platforms: K1/K230.
Accelaration is achieved when threshold is high, however, weird stat shows that the acceleration doesn't work when threshold is low (the number of keypoint candidates is high).
K1:
K230
Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.