Skip to content

Fix wrong clamping of pixels for fisheye::undistortPoints #27199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: 4.x
Choose a base branch
from
Open

Fix wrong clamping of pixels for fisheye::undistortPoints #27199

wants to merge 3 commits into from

Conversation

mokrueger
Copy link

Pull Request Readiness Checklist

Fixes: #26174
Feel free to adjust if required.

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

@mokrueger mokrueger mentioned this pull request Apr 5, 2025
Open
@asmorkalov
Copy link
Contributor

Test failures:

[ RUN      ] fisheyeTest.CalibrationWithDifferentPointsNumber
unknown file: Failure
C++ exception with description "OpenCV(4.12.0-dev) /Users/opencv-cn/GHA-OCV-3/_work/opencv/opencv/opencv/modules/calib3d/src/fisheye.cpp:1439: error: (-215:Assertion failed) fabs(norm_u1) > 0 in function 'InitExtrinsics'
" thrown in the test body.
[  FAILED  ] fisheyeTest.CalibrationWithDifferentPointsNumber (1 ms)
[ RUN      ] fisheyeTest.stereoCalibrateWithPerViewTransformations
[       OK ] fisheyeTest.stereoCalibrateWithPerViewTransformations (539 ms)
[ RUN      ] fisheyeTest.estimateNewCameraMatrixForUndistortRectify
/Users/opencv-cn/GHA-OCV-3/_work/opencv/opencv/opencv/modules/calib3d/test/test_fisheye.cpp:1195: Failure
The max difference between matrices "K_new" and "K_new_truth" is 961.206 at (1, 2), which exceeds "1e-6", where "K_new" at (1, 2) evaluates to (1500), "K_new_truth" at (1, 2) evaluates to (538.794), "1e-6" evaluates to 1e-06
[  FAILED  ] fisheyeTest.estimateNewCameraMatrixForUndistortRectify (0 ms)

Most probably regression data update is required.

@mokrueger
Copy link
Author

@asmorkalov thanks for the feedback. I'll take a look

@mokrueger
Copy link
Author

@asmorkalov The test case:

opencv/modules/calib3d/test/test_fisheye.cpp

Line 1153 in 09a85e9

TEST_F(fisheyeTest, estimateNewCameraMatrixForUndistortRectify)

is degenerate. The intrinsics + image shape do not produce a valid (reversible) fisheye projection.
image
There are pixels which are mapped beyond 180 degree FOV, which is currently unsupported.

I will adjust it accordingly later this week.

@mokrueger
Copy link
Author

@asmorkalov Also one more note I also want to make. Currently 'invalid' pixels are mapped to [-1000000.0, -1000000.0]]. Since there are also valid pixels that can get mapped to that. Why does one not switch to Nan?

opencv/modules/calib3d/src/fisheye.cpp

Lines 501 to 502 in 09a85e9

// Vec2d fi(std::numeric_limits<double>::quiet_NaN(), std::numeric_limits<double>::quiet_NaN());
Vec2d fi(-1000000.0, -1000000.0);

@asmorkalov
Copy link
Contributor

Why does one not switch to Nan?

I do not know unfortunately. Feel free propose alternative alternative solution with nans.

@asmorkalov
Copy link
Contributor

Looks like you need too tune related java test in modules/calib3d/misc/java/test/Calib3dTest.java too. (see the last test case in the file).

@asmorkalov
Copy link
Contributor

@vpisarev @ivashmak Could you take a look on the solution?

@mokrueger
Copy link
Author

I think there are some more issues with fisheye in general. I am currently also taking a deeper look but it might take some time.

@mokrueger
Copy link
Author

Update: fisheyeTest.CalibrationWithDifferentPointsNumber fails because of 1) numerical instability and/or 2) because we produce invalid points at the beginning (points with FOV > 180). Also we have some issues with calibration (because the old + wrong version rarely produces invalid points).

I am not 100% sure how this could be fixed.

Currently invalid points will be mapped to -> -10000,-10000
We could:

  1. just not mark these points invalid -> would produce a negative scale variable -> not sure if this is mathematically valid but it may very well be -> invalid/>180 points could be filtered by checking if x,y sign is flipped
  2. filter invalid points during calibration(?) -> I am no expert on the current calibration

tldr: we should check how points with >180 deg can be handled for undistortPoints (because gradients are used in calibration) and then adjust the calibration accordingly.

@asmorkalov
Copy link
Contributor 

[ RUN      ] fisheyeTest.CalibrationWithDifferentPointsNumber
unknown file: Failure
C++ exception with description "OpenCV(4.12.0-dev) /Users/opencv-cn/GHA-OCV-2/_work/opencv/opencv/opencv/modules/calib3d/src/fisheye.cpp:1439: error: (-215:Assertion failed) fabs(norm_u1) > 0 in function 'InitExtrinsics'
" thrown in the test body.
[  FAILED  ] fisheyeTest.CalibrationWithDifferentPointsNumber (2 ms)
[ RUN      ] fisheyeTest.stereoCalibrateWithPerViewTransformations
[       OK ] fisheyeTest.stereoCalibrateWithPerViewTransformations (563 ms)
[ RUN      ] fisheyeTest.estimateNewCameraMatrixForUndistortRectify
/Users/opencv-cn/GHA-OCV-2/_work/opencv/opencv/opencv/modules/calib3d/test/test_fisheye.cpp:1195: Failure
The max difference between matrices "K_new" and "K_new_truth" is 2.53784e-05 at (0, 2), which exceeds "1e-6", where "K_new" at (0, 2) evaluates to (1009.89), "K_new_truth" at (0, 2) evaluates to (1009.89), "1e-6" evaluates to 1e-06
[  FAILED  ] fisheyeTest.estimateNewCameraMatrixForUndistortRectify (0 ms)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
bug category: calib3d
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

undistortPoints function implementation is not working the near 180 degree points
2 participants
@mokrueger @asmorkalov
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy