Skip to content

Add TryHarder and MinLineCount hints for python wrapper #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

michelerenzullo
Copy link

Added 2 hints to the python wrapper, MinLineCount and TryHarder

Added 2 hints to the python wrapper, MinLineCount and TryHarder
@axxel
Copy link
Collaborator

axxel commented Nov 24, 2022

Thanks for your offer to improve the code. I'm hesitant to merge your PR, though. I deliberately did not include the tryHarder flag in the python API as this is basically only useful for real-time video frame scanning on mobile devices. What is your motivation to add that to the python API?

The same question generally applies to the minLineCount property as well. Do you need something else than the default of '2' here?

@michelerenzullo
Copy link
Author

michelerenzullo commented Nov 24, 2022

I use kinda real time implementation on android through the NDK, yes.

The reason is in my project, I have a unit test in python and I have to pass the same settings of the cpp implementation to check that is the same.

I use MinLineCount to 1 and try harder false

@axxel
Copy link
Collaborator

axxel commented Nov 25, 2022

So you run the same images through your cpp and your python implementation to see if both return the same results? Those can only be different if the two are based on different versions of the code base and I'm not convinced that this justifies the bloating of the python API, especially not the addition of the tryHarder flag, which I've had my issues with for years (#104).

@michelerenzullo
Copy link
Author

We use in production the cpp with android NDK and iOS.

While python in a separate module to test barcodes joined with other pre-processing routine.

Maybe it's ok about tryHarder, but at least MinLineCount? The latter is crucial for some barcodes in our project, and we want to test using the old behaviour of v1.2.0 when was setted to 1

@axxel
Copy link
Collaborator

axxel commented Nov 25, 2022

minLineCount is indeed a little different. But before we simply add that, I'd like to use the opportunity to point out that a value of 2 is in my experience really a very reasonable minimum value to combat false positives and I'd recommend to use that instead of 1. If you can share one of those samples from your project, where this makes a difference, I'd be interested to have a look.

@michelerenzullo
Copy link
Author

michelerenzullo commented Nov 25, 2022

Sure, I have many "strange examples" that might be useful for you, with this image we can spot 2 things: the slightly imperfect alignment make fail the detecting, I addressed this issue with an iterative rotation of step for 0.02 degree till +1.0 and from -0.02 to -1.0 --> SUCCESS but only if we use MinLineCount = 1 ! Try by yourself to set = 2 and it will fail, note, this is not an edge-case it's a very common one, I have so many like this
rotated
original

Testing and Pre-processing

//compile with -lZXing -lopencv_imgcodecs -lopencv_imgproc -lopencv_core
#include <algorithm>
#include <ZXing/ReadBarcode.h>
#include <opencv2/opencv.hpp>
#include <chrono>

void sharpen(cv::Mat &image, float strength)
{
    cv::Mat sharpened;
    image.convertTo(sharpened, CV_32F);
    cv::medianBlur(sharpened, sharpened, 3);
    cv::Mat laplacian;
    cv::Laplacian(sharpened, laplacian, CV_32F);
    sharpened = sharpened - strength * laplacian;

    sharpened.convertTo(image, CV_8U);
}

std::string read_barcode(const cv::Mat &image)
{
    auto imageBuffer = ZXing::ImageView(image.ptr(), image.cols, image.rows, ZXing::ImageFormat::Lum);
    auto hints = ZXing::DecodeHints()
                     .setMinLineCount(1)                              // default is 2 since commit https://github.com/zxing-cpp/zxing-cpp/commit/11fc1d889071562a9eb4fb7ac94e65f220dd86ab
                     .setTryHarder(true)                             // If false, skip some lines/pixels during detection (faster)
                     .setTryRotate(false)                             // If true, try 90, 180 and 270 degree rotation. Already handled by transform.
                     .setBinarizer(ZXing::Binarizer::GlobalHistogram) // Choose from fixed, histogram, local average, and bool
                     .setIsPure(false)                                // Assumes the image contains a perfectly aligned code (faster)
                     .setFormats(ZXing::BarcodeFormat::Code128)       // Limits the amount of formats to check for (faster)
                     .setTryDownscale(false);                         // If false, it doesn't downscale the image

    auto found_barcode = ZXing::ReadBarcode(imageBuffer, hints);
    if (!found_barcode.isValid())
    {
        return "";
    }
    const auto barcode_text = found_barcode.text();
    printf("Barcode detected: %s (%s)\n", barcode_text.c_str(), ZXing::ToString(found_barcode.error()).c_str());
    return barcode_text;
}

int main()
{

    cv::Mat image = cv::imread("original.png");
    if (image.channels() == 3)
        cv::cvtColor(image, image, cv::COLOR_RGB2GRAY);

    assert(image.channels() == 1);

    auto barcode_text = read_barcode(image);

    if (barcode_text.empty()) {
      printf("Trying to read barcode with sharpened image and Otsu's thresholding.\n");
      sharpen(image, 1.0);
      cv::Mat Otsu;
      cv::threshold(image, Otsu, 0, 255, cv::THRESH_BINARY | cv::THRESH_OTSU);

      barcode_text = read_barcode(Otsu);
      cv::imwrite("sharpened_thresholded.jpg", Otsu);
    }

    if (barcode_text.empty())
    {
        printf("Trying to read barcode with different degrees of rotation.\n");
        auto start = std::chrono::steady_clock::now();

        // Generate a lookup of degrees
        // with step of 0.02 and sorted for probabilities
        // of wrong cardboard alignment
        // [-0.02, -0.04 ... -1.0, +0.02, +0.04 ... +1.0]
        const float step = 0.02f;
        const int elements = 2 / step;
        auto degrees = std::make_unique<float[]>(elements);
        float degree = 0.f;
        std::generate_n(degrees.get(), elements / 2, [&degree, &step]
                        { return degree -= step; });
        degree = 0.f;
        std::generate_n(&degrees[elements / 2], elements / 2, [&degree, &step]
                        { return degree += step; });

        const cv::Point2f center(int(image.cols / 2.f), int(image.rows / 2.f));
        for (int i = 0; i < elements && barcode_text.empty(); ++i)
        {
            cv::Mat rotation_matrix = getRotationMatrix2D(center, degrees[i], 1.0);
            cv::Mat rotated_image;
            cv::warpAffine(image, rotated_image, rotation_matrix, image.size());
            barcode_text = read_barcode(rotated_image);
            if (!barcode_text.empty())
            {
                printf("%f\n", std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - start).count());
                cv::imwrite("rotated.jpg", rotated_image);
                printf("rotation angle fixed %f after %i iterations\n", degrees[i], i + 1);
            }
        }
    }
}

@michelerenzullo
Copy link
Author

I think that the iterative rotation emulates the real-time detection with frames, that's why it works always when previous attempts fail

@michelerenzullo
Copy link
Author

I have a question without digging in the math formula, your GlobalHistogram is doing the same type of thresholding of Otsu's method, why alone is not enough? Is GlobalHistogram "less" accurate than Otsu? Shouldn't be the same? In the above code I'm doing Otsu to read the barcode, without it would fail

@axxel
Copy link
Collaborator

axxel commented Nov 29, 2022

I'm a bit low on time but just to give you a quick update:

  • GlobalHistogram is a port of the original Java code. It is not meant to be an implementation of Otsu but tries to do something similar. I actually experimented with Otsu myself a while back and while it helps in some situations, it breaks other that currently work, hence I did not simply switch the algorithms.
  • Both your samples should work out-of-the box (with minLineCount >> 1) in my opinion. They currently don't because the 'black' bars are too wide compared to the 'white' ones due to sharpness/lighting issues. I see how a different binarizer implementation (e.g. Otsu) would help here. The real fix for this is a change in the pattern-to-character decoding though. I was having this on my todo list for a while and was looking into it, just now. I believe to have a promising way forward, but don't know how long this 'real' fix will take (see 'low on time' comment above).
  • You can of course use Otsu prior to calling the library.
  • This whole discussion further convinces me that the fix for your issue is not the addition of the minLineCount hint to the python API.

@michelerenzullo
Copy link
Author

michelerenzullo commented Nov 29, 2022

Thank you! I didn't notice that are wide and the aspect ratio is not correct. I'm gonna fix

@axxel
Copy link
Collaborator

axxel commented Nov 29, 2022

Thank you! I didn't notice that are wide and the aspect ratio is not correct. I'm gonna fix

I don't know what you mean by 'aspect ratio not correct'. There is nothing wrong with that. What I meant is that the image is somewhat underexposed, which makes the black bars wider compared to the white ones, but I personally would like to see the library being capable of dealing with that as is and I am hopeful that it will in some not-too-far-away-future.

@michelerenzullo
Copy link
Author

michelerenzullo commented Nov 29, 2022

I want to clarify what I said before, ignoring the exposure situation (because even fixing that is not enough): so the aspect ratio is modified in my project, the original barcode has thin black lines because the height is 15% of the width, but for some problem after doing a warp, the height becomes the 10.9% of the width, so it's like stretched out(wider), after resizing with opencv multiply for a factor of 1.375 the "Impossible" barcode is detected at first attempt . I think that stretching out barcodes, for mistake, make more difficult the detection sometimes. Therefore I'm trying to fix the warp function that I use in my project

@axxel
Copy link
Collaborator

axxel commented Nov 29, 2022

Don't worry about the aspect ratio of the image, it is irrelevant for the detection of 1D barcodes like Code128 here (completely different situation for 2D symbologies, of course). Important for the detection is the relative widths of the black vs. the white bars. If you made all of them twice as wide, the result would be the same.

The reason why your symbol might still be detected after this rescaling is basically noise, same as your tiny rotation operations you mentioned earlier. Random minor changes might make the difference between 1 or 2 detected lines while this input should actually be found with 100 lines (or whatever the height of the symbol is in pixels).

axxel added a commit that referenced this pull request Jan 4, 2023
This improves the reliability in situations with under/over exposed images.
It basically adopts the reference algorithm from the specification.

This 'fixes' the image discussed in #444.
@axxel
Copy link
Collaborator

axxel commented Jan 4, 2023

The linked commit fixes your issue with the above images. They are now both detected with 40+ lines. I decided to trust my work and put this into the next release, even if it is very last minute ;).

@axxel axxel closed this Jan 4, 2023
@michelerenzullo
Copy link
Author

Thank you very much!

@michelerenzullo
Copy link
Author

I confirm, now is absolutely perfect with my "difficult" 100 barcode datasets. Could you explain what was exactly the cause of the issue and how you addressed it? Just to keep in my mind, even in short words. The black lines "too thick", right?

@axxel
Copy link
Collaborator

axxel commented Jan 5, 2023

Say you have a b/w pattern (in widths) as follows: bwwwbwww (so 1, 3, 1, 3). now if you capture that slightly blurred and underexposed, then threshold it, you can end up with the pattern bbwwbbww (or 2, 2, 2, 2) and they will not match. If you count from edge-to-edge (start of black to next start of black), though, then both patterns are the same (4, 4, 4). That is basically the idea.

I confirm, now is absolutely perfect with my "difficult" 100 barcode datasets.

That is very good to hear. They are all Code128?

@michelerenzullo
Copy link
Author

Yes all code128, do you want the archive?

@axxel
Copy link
Collaborator

axxel commented Jan 6, 2023

No thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy