@s-trinh Is the PR ready for review?

Hello @ivashmak

In OpenCV 4, SOLVEPNP_DLS and SOLVEPNP_UPNP methods (for cv::solvePnP()) have been deactivated and silently remapped to SOLVEPNP_EPNP, see #3828
It has been the case for more than 10 years now.
OpenCV 5 is the opportunity to remove these two flags.

When searching, it looks like the DLS method suffers from a singularity in the rotation parameterization, eg. from or this paper:

I have seen that in USAC:

My questions are:

• any comment, objection about the removal of the SOLVEPNP_DLS or SOLVEPNP_UPNP methods in cv::solvePnP() methods?
• should the USAC PnP use another method instead of the DLS method?
• any reason to not use directly the SOLVEPNP_AP3P (or SOLVEPNP_P3P) in OpenCV as a P3P solver in USAC?

Thanks.

Hi @s-trinh,

• If SOLVEPNP_DLS and SOLVEPNP_UPNP flags are internally mapped to SOLVEPNP_EPNP, then it is probably safe to remove them. I do not have any objection about it.
• USAC PnP uses DLS on a non-minimal (>3) sample. After using DLS solver, USAC verifies that the DLS solution is better (in terms of inliers) than the so-far-the-best. If DLS has problems with singularity, which results in poor estimation, then this solution will not be accepted in USAC. However, if DLS returns a solution with a high number of inliers, but is degenerate, then this is a potential risk. Empirically, have you noticed this problem with running experiments? There is a least-squares PnP solver implemented in USAC, which should not return a degenerate result, but it often gives numerically worse models than DLS.
• P3P solver in USAC is an optimized version of P3P solver, which is faster than the one in OpenCV. AP3P solver is a potentially better solver than P3P. However, I am not sure if it is worth to replace it. First, USAC P3P is very fast, which is very important in RANSAC/USAC paradigm, when many samples are drawn. Second, the final solution is optimized with a non-minimal sample. It is very rare that USAC will return a solution directly from P3P solver (a minimal sample).

Thanks for the answers @ivashmak

However, if DLS returns a solution with a high number of inliers, but is degenerate, then this is a potential risk. Empirically, have you noticed this problem with running experiments?

I have not encountered these issues, at least not situations where the DLS singularity could have been specifically pinpointed.
Main reason is to clean-up the code and avoid issue where someone thinks he is using SOLVEPNP_DLS or SOLVEPNP_UPNP while it is SOLVEPNP_EPNP under the hood and to try to improve the library.

There is a least-squares PnP solver implemented in USAC, which should not return a degenerate result, but it often gives numerically worse models than DLS.

Is it this one?

P3P solver in USAC is an optimized version of P3P solver, which is faster than the one in OpenCV. AP3P solver is a potentially better solver than P3P.

I have experienced with poor accuracy with the P3P solver in cv::solvePnP() and when looking online I have found:

which seem to corroborate with my experiments.

So actually it could be a good idea to replace the P3P in cv::solvePnP() with the USAC one in the future.

@asmorkalov

Is there somewhere something to have a list of API breaking changes in OpenCV 5?
I think the removal of SOLVEPNP_DLS and SOLVEPNP_UPNP flags should be documented in the doc for instance.

If the change of default values of enum SolvePnPMethod is not an issue, the PR is ready.

I have not encountered these issues, at least not situations where the DLS singularity could have been specifically pinpointed. Main reason is to clean-up the code and avoid issue where someone thinks he is using SOLVEPNP_DLS or SOLVEPNP_UPNP while it is SOLVEPNP_EPNP under the hood and to try to improve the library.

Cleaning up code makes sense. I would leave DLS in USAC, because it can significantly improves noisy models from the P3P algorithm. In the documentation, we can make it clear that USAC has 2 solvers.

Is it this one?

opencv/modules/3d/src/usac/pnp_solver.cpp

Line 16 in 2e6a0ca

class PnPMinimalSolver6PtsImpl : public PnPMinimalSolver6Pts {

Almost, P6P works in the same way as the non-minimal PnP solver, which is some lines below:

I have experienced with poor accuracy with the P3P solver in cv::solvePnP() and when looking online I have found:

opencv/modules/calib3d/src/p3p.cpp

Lines 211 to 221 in afe5b22

	/// Given 3D distances between three points and cosines of 3 angles at the apex, calculates
	/// the lengths of the line segments connecting projection center (P) and the three 3D points (A, B, C).
	/// Returned distances are for |PA|, |PB|, |PC| respectively.
	/// Only the solution to the main branch.
	/// Reference : X.S. Gao, X.-R. Hou, J. Tang, H.-F. Chang; "Complete Solution Classification for the Perspective-Three-Point Problem"
	/// IEEE Trans. on PAMI, vol. 25, No. 8, August 2003
	/// \param lengths Lengths of line segments up to four solutions.
	/// \param distances Distance between 3D points in pairs |BC|, |AC|, |AB|.
	/// \param cosines Cosine of the angles /_BPC, /_APC, /_APB.
	/// \returns Number of solutions.
	/// WARNING: NOT ALL THE DEGENERATE CASES ARE IMPLEMENTED

which seem to corroborate with my experiments.

So actually it could be a good idea to replace the P3P in cv::solvePnP() with the USAC one in the future.

I know that my P3P version is faster, I am not sure (do not remember) about accuracy from direct comparison with OpenCV P3P. However, I tested before (around 3 years ago) that USAC PnP (overall) is faster and more accurate than the other OpenCV PnP methods.