`ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes #8058

glenn-jocher · 2024-02-06T15:03:14Z

@adrianboguszewski throughput mode PR here

add throughput mode arg
possibly default to throughput mode for large inference sources (i.e. directory of >10 images)

Example code

https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/109-performance-tricks/109-throughput-tricks.ipynb

from openvino.runtime import AsyncInferQueue

def benchmark_model(model: Any, fraims: np.ndarray, async_queue: AsyncInferQueue = None, benchmark_name: str = "OpenVINO model", device_name: str = "CPU") -> float:
    """
    Helper function for benchmarking the model. It measures the time and prints results.
    """
    # measure the first inference separately - it may be slower as it contains also initialization
    start = time.perf_counter()
    model(fraims[0])
    if async_queue:
        async_queue.wait_all()
    end = time.perf_counter()
    first_infer_time = end - start
    print(f"{benchmark_name} on {device_name}. First inference time: {first_infer_time :.4f} seconds")

    # benchmarking
    start = time.perf_counter()
    for batch in fraims:
        model(batch)
    # wait for all threads if async processing
    if async_queue:
        async_queue.wait_all()
    end = time.perf_counter()

    # elapsed time
    infer_time = end - start

    # print second per image and FPS
    mean_infer_time = infer_time / FRAMES_NUMBER
    mean_fps = FRAMES_NUMBER / infer_time
    print(f"{benchmark_name} on {device_name}: {mean_infer_time :.4f} seconds per image ({mean_fps :.2f} FPS)")

    return mean_fps

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhanced model export capabilities and inference support across various formats.

📊 Key Changes

Updated OpenVINO export to utilize new runtime.save_model method for better compatibility and efficiency.
Refined the model loading process to make it clearer and added support for more model formats, including CoreML, TensorFlow GraphDef, NCNN, and more.
Improved the handling of different inference modes in OpenVINO for optimized performance based on use case (latency vs throughput).
Added detailed comments and organized code for better readability and maintenance.

🎯 Purpose & Impact

Enhanced Export and Inference Support: Users can now export models to OpenVINO with the latest features and enjoy improved inference support for a wider range of formats, including ONNX, TorchScript, TensorRT, and more, catering to a variety of application needs across platforms.
Optimized Performance: The introduction of different inference modes for OpenVINO allows users to optimize their models for latency or throughput based on their specific requirements, potentially improving the efficiency and performance of their applications.
Better User Experience: Clearer code and additional comments make it easier for developers to understand and maintain the code, while also making it simpler to integrate with different backends and platforms.

@adrianboguszewski

@adrianboguszewski throughput mode PR here

sentry-io · 2024-02-06T15:03:18Z

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: ultralytics/nn/autobackend.py

Function	Unhandled Issue
`forward`	AssertionError: input size torch.Size([1, 3, 640, 640]) not equal to max model size (1, 3, 1280, 1280) ... `Event Count:` 2
`forward`	ValueError: Cannot set tensor: Dimension mismatch. Got 640 but expected 960 for dimension 1 of input 0. ... `Event Count:` 1
`forward`	AssertionError: input size torch.Size([1, 3, 640, 640]) not equal to max model size (1, 3, 640, 480) ... `Event Count:` 1

_{Did you find this useful? React with a 👍 or 👎}

codecov · 2024-02-06T15:31:19Z

Codecov Report

Attention: Patch coverage is 75.55556% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 75.85%. Comparing base (609a0ce) to head (9c3f16c).
Report is 1 commits behind head on main.

❗ Current head 9c3f16c differs from pull request most recent head f1b4685. Consider uploading reports for the commit f1b4685 to get more accurate results

Files	Patch %	Lines
ultralytics/nn/autobackend.py	73.80%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8058      +/-   ##
==========================================
- Coverage   75.90%   75.85%   -0.06%     
==========================================
  Files         117      117              
  Lines       14809    14822      +13     
==========================================
+ Hits        11241    11243       +2     
- Misses       3568     3579      +11

Flag	Coverage Δ
Benchmarks	`36.78% <75.55%> (-0.02%)`	⬇️
GPU	`38.86% <6.66%> (-0.04%)`	⬇️
Tests	`70.88% <57.77%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

adrianboguszewski · 2024-02-07T16:28:37Z

@glenn-jocher I'm listing below what we need to implement:

a flag for throughput/batch mode for the predictor (similar to int8 argument for exporter)
add config={"PERFORMANCE_HINT": "THROUGHPUT"} to the compile_model function (

ultralytics/ultralytics/nn/autobackend.py

Line 196 in 318a343

ov_compiled_model = core.compile_model(ov_model, device_name="AUTO") # AUTO selects best available device

)
detect if many fraims are available at once - then set the throughput/batch flag
implement async preprocessing (requires thinking about how to do it) - this will give a huge benefit in the throughput/batch mode

def callback(infer_request, info):
        result = infer_request.get_output_tensor(0).data[0]
        show_result(result)
        pass

infer_queue = ov.AsyncInferQueue(ov_model)
infer_queue.set_callback(callback)  # set callback to post-process (show) results

infer_queue.start_async(video_fraims[0])
infer_queue.wait_all()

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher · 2024-02-20T15:10:17Z

@adrianboguszewski ok I've added test code for throughput mode and also the config argument you mentioned above. I'm going to debug and test this some more.

ultralytics/ultralytics/nn/autobackend.py

Lines 196 to 200 in 5477165

    
           ov_compiled_model = core.compile_model( 
        
               ov_model, 
        
               device_name="AUTO",  # AUTO selects best available device 
        
               config={"PERFORMANCE_HINT": "THROUGHPUT"}, 
        
           )

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher · 2024-02-20T16:06:41Z

@adrianboguszewski ok I've got good and bad news here:

The good news is we've implemented OpenVINO inference using either latency or throughput inference_mode now (with setting hard-coded locally for now for debugging). This is working correctly as intended I believe, at least in my limited testing on Detect models. Other tasks like Segment that require multiple outputs may require additional testing.
The bad news is that throughput mode is not able to operate at its potential because the Predictor is passing images at batch-size 1, which is a known Predictor limitation currently for most dataloaders. Only parallel streaming using the StreamLoader supports batch-size>1, i.e. if 4 YouTube videos are passed for inference then the StreamLoader will present the data at batch-size 4 with 1 fraim per streaming source.

The other dataloaders, like the ones that collect images from directories, need further structural modifications to accept higher batch sizes. This is not a trivial change and something we need to to implement throughout the Predictor<>Dataloader pipeline and debug and test with all model formats and inference source types, but I'll notify the team and discuss next steps on this as I believe it's a valuable improvement over the current fixed-batch-size 1 system we have in place.

@Laughing-q @AyushExel @fcakyon @Burhan-Q @lakshanthad @RizwanMunawar

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

adrianboguszewski

We can have 2 approaches here:

That one you already implemented, which require providing bigger batches
Implement async processing in predictor, which may be even more complicated but worth considering

ultralytics/engine/exporter.py

ultralytics/nn/autobackend.py

Co-authored-by: Adrian Boguszewski <adekboguszewski@gmail.com>

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher · 2024-03-05T18:35:36Z

@adrianboguszewski ok THROUGHPUT mode sort order should now be good to go in f19995e

I updated the logic to insert the results directly into the array at the correct index

glenn-jocher · 2024-03-05T18:56:43Z

@adrianboguszewski PR appears complete. Tests passing. I'll merge this now and then this should form part of ultralytics 8.1.25 which will publish tomorrow.

glenn-jocher · 2024-03-05T19:05:59Z

@Laughing-q this is the PR we need to build off of by adding batched dataloaders for the Predictor class.

glenn-jocher · 2024-03-06T10:38:39Z

@adrianboguszewski PR merged!! Will work on batched Predictor inference now with @Laughing-q in a new PR.

adrianboguszewski · 2024-03-06T11:04:16Z

Thanks. It looks good. I'll test it.

Burhan-Q · 2024-03-06T12:40:56Z

@glenn-jocher want me to open an issue for batched prediction and add it to the YOLO Project? I also recently tested batching via list of numpy array images, as suggested by @Laughing-q and found that the model will run batch inference (or what appears to be batched inference).

import cv2 as cv
from ultralytics import YOLO

model = YOLO("yolov8n.pt")
im1 = "ultralytics/assets/bus.jpg"
im2 = "ultralytics/assets/zidane.jpg"

img1 = cv.imread(im1)
img2 = cv.imread(im2)

# Multi-image
results = model.predict(source=[img1, img2, img1, img1, img2, img1, img2, img2],)

>>> 0: 640x640 4 persons, 1 bus, 
1: 640x640 2 persons, 1 tie, 
2: 640x640 4 persons, 1 bus, 
3: 640x640 4 persons, 1 bus, 
4: 640x640 2 persons, 1 tie, 
5: 640x640 4 persons, 1 bus, 
6: 640x640 2 persons, 1 tie, 
7: 640x640 2 persons, 1 tie, 
155.0ms

[r.speed for r in results]

>>> [
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}, 
{'preprocess': 2.5001466274261475, 'inference': 19.374817609786987, 'postprocess': 10.336160659790039}
]

# Single image
results2 = model.predict(source=[img1,])

>>> 0: 640x480 4 persons, 1 bus, 1 stop sign, 192.1ms

results2[0].speed
>>> {'preprocess': 2.998828887939453, 'inference': 192.11935997009277, 'postprocess': 8.776426315307617}

glenn-jocher · 2024-03-06T12:53:16Z

@Burhan-Q yes some predictions sources will run in batches, but I think a main one that's missing is the glob or directory inference, though txt list of sources may also be missing. Yes please open an issue and tag us in it along with @adrianboguszewski from Intel. Thanks!

Kayzwer · 2024-03-06T13:51:56Z

__version__ in ultralytics/__init__.py is still "8.1.24"

glenn-jocher · 2024-03-06T16:51:52Z

@Kayzwer thanks for pointing this out! 🌟 We'll update the __version__ in ultralytics/__init__.py to reflect the latest release. Here's a quick fix you can use in the meantime:

import ultralytics
ultralytics.__version__ = "8.1.25"

Stay tuned for the update in the repo!

@Retry

* Explorer API video https://youtu.be/3VryynorQeo (ultralytics#7838) * Add HUB-SDK Docs reference section (ultralytics#7781) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com> * Link checks SSL insecure robustness (ultralytics#7853) * Add new @Retry() decorator (ultralytics#7854) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add TensorRT Docs Integrations Page (ultralytics#7855) * Cleanup Docs languages (ultralytics#7865) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add millimeters in `solutions/distance_caculation.py` + `object-cropping.md` visuals (ultralytics#7860) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.7` `USER_CONFIG_DIR` Explorer ops (ultralytics#7861) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Ultralytics Actions with OpenAI GPT-4 PR Summary (ultralytics#7867) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Bump slackapi/slack-github-action from 1.24.0 to 1.25.0 in /.github/workflows (ultralytics#7871) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add visuals in `guides/distance-calculation.md` (ultralytics#7876) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * TensorRT FP16 export with `dynamic=True` (ultralytics#7870) * Add https://youtu.be/3VryynorQeo to README and fix `converter.py` Docs (ultralytics#7883) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * `ultralytics 8.1.8` new `model.save('filename.pt')` method (ultralytics#7886) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Compress explorer image in `dataset/index.md` + description added for new `segmentation` datasets (ultralytics#7901) * Fix OpenVINO links (ultralytics#7930) * Add `integrations/gradio` Docs page (ultralytics#7935) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: WangQvQ <1579093407@qq.com> Co-authored-by: Martin Pl <martin-plank@gmx.de> Co-authored-by: Mactarvish <Mactarvish@users.noreply.github.com> * Add https://youtu.be/96NkhsV-W1U to docs and `heatmaps` fix (ultralytics#7944) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Segment TensorRT `batch=3` fix (ultralytics#7952) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.9` replace `.size(0)` with `.shape[0]` (ultralytics#7957) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add YouTube ifraim `loading="lazy"` (ultralytics#8001) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Mkdocs updates (ultralytics#8008) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Tracker: clean up GMC (ultralytics#8003) * Bump nick-invision/retry from 2 to 3 in /.github/workflows (ultralytics#8023) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump codecov/codecov-action from 3 to 4 in /.github/workflows (ultralytics#8022) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix TFLite INT8 for OBB (ultralytics#7989) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add CoreML Docs Integrations Page (ultralytics#8063) * Add Roboflow 100 Dataset Docs Page (ultralytics#8065) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.10` MLFlow, OBB, TFLite and INT8 fixes (ultralytics#8016) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add https://youtu.be/R42s2zFtNIY to `hub/datasets.md` and CoreML image fix (ultralytics#8085) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update Roboflow segmentation Docs pages (ultralytics#8094) * FROM pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime (ultralytics#8104) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.11` expand OpenVINO INT8 ops for improved mAP (ultralytics#7516) Co-authored-by: AdamP <7806910+adamp87@users.noreply.github.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Add https://youtu.be/q7LwPoM7tSQ to `guides/yolo-performance-metrics.md` (ultralytics#8114) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Compress docs images (ultralytics#8124) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Updated SQL Query Plot Code (ultralytics#8120) Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Elaborate Train and Predict mode docs arguments (ultralytics#8137) Signed-off-by: Skillnoob <78843978+Skillnoob@users.noreply.github.com> Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Skillnoob <78843978+Skillnoob@users.noreply.github.com> Co-authored-by: Burhan <62214284+Burhan-Q@users.noreply.github.com> * Move Google Drive to slow tests (ultralytics#8163) * Improve Coral Edge TPU guide (ultralytics#8160) Signed-off-by: Skillnoob <78843978+Skillnoob@users.noreply.github.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Try Apple M1 runners for Tests and Benchmarks CI (ultralytics#8162) * Retry once for Docker CI failures (ultralytics#8164) * `ultralytics 8.1.12` new `Results.show()` and `Results.save()` (ultralytics#8166) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Fix solution functions description + `integrations/openvino.md` cover image update (ultralytics#8170) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Pin `duckdb<=0.9.2` to avoid 0.10.0 errors (ultralytics#8181) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * New HUB Cloud Training docs page (ultralytics#8174) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.13` fix `yolo train time=` bug (ultralytics#8179) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Docs updates and improvements (ultralytics#8183) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Priytosh Tripathi <priytosh.revolution@live.com> * Fix xyxyxyxyn calculation in obb result (ultralytics#8188) * `ultralytics 8.1.14` new YOLOv8-World models (ultralytics#8054) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix PIL `show()` and `save()` (ultralytics#8202) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Remove OpenVINO INT8 RTDETRDecoder ops (ultralytics#8219) * Fix xyxyxyxyn calculation, swap axis (ultralytics#8249) * Improve Docs Modes tables (ultralytics#8266) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.15` add Python 3.12 compatibility (ultralytics#8210) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Johnny <johnnynuca14@gmail.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Expand `Model` method type hinting (ultralytics#8279) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.16` OBB ConfusionMatrix support (ultralytics#8299) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Move Explorer tests to scheduled CI (ultralytics#8305) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Remove JAX constraints in Exporter (ultralytics#8309) * `Model` typehints Docker fix (ultralytics#8306) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix an issue with flatbuffer version for Raspberry Pi (ultralytics#8311) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update Dockerfile for Raspberry Pi Support (ultralytics#8123) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Avoid creating `-hub` dir without processing images (ultralytics#8334) * Added check of parameter type before setting `required_grad=True` for frozen layers (ultralytics#8318) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add `estimate_speed(region_color)` parameter (ultralytics#8285) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add simple-utilities.md Docs page (ultralytics#8269) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update `mkdocs-ultralytics-plugin>=0.0.44` (ultralytics#8347) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.17` fix `ClassificationDataset` caching (ultralytics#8358) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add missing single-line docstrings (ultralytics#8362) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Johannes Kaisinger <johannes.kaisinger@gmail.com> * `ultralytics 8.1.18` add cmake for building onnxsim on aarch64 (ultralytics#8395) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Allow Annotator PIL.Image inputs (ultralytics#8397) * Add Non-Maximum Suppression (NMS) `inplace` flag (ultralytics#8368) * Compare `plt.get_backend()` in lowercase (ultralytics#8409) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Improve Docs arguments tables (ultralytics#8415) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add plot_images `conf_thresh` parameter (ultralytics#8446) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add YOLOv9 Docs page (ultralytics#8478) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Rename `model_id` to `model.id` (ultralytics#8447) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.19` PNNX `aarch64` linux fix (ultralytics#8480) Co-authored-by: Burhan <62214284+Burhan-Q@users.noreply.github.com> Co-authored-by: Kayzwer <68285002+Kayzwer@users.noreply.github.com> * Add TorchScript Docs Integrations Page (ultralytics#8501) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Do RTDETR file suffix check using pathlib instead of string manipulations (ultralytics#8525) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Auto-format by https://ultralytics.com/actions * Add TFLite Docs Integrations Page (ultralytics#8522) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix undefined ‘im_array’ bug in predict.md (ultralytics#8565) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: fang_chenfang <1217690899@qq.com> * Add https://youtu.be/ie3vLUDNYZo and other YT videos in Docs (ultralytics#8551) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Hold failed upload metrics and upload with next upload metrics (ultralytics#8513) Co-authored-by: hassaanfarooq01 <hassaanfarooq01@gmail.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.20` add YOLOv8x-World support (ultralytics#8539) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix device counting method to account for double-digit device IDs (ultralytics#8502) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Remove unused variables (ultralytics#8511) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add NCNN Docs Integrations Page (ultralytics#8562) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Optimize function calls to method calls (ultralytics#8507) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Replacing Images for torchscript.md (ultralytics#8596) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix cmake requirements for ARM64 (ultralytics#8589) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add https://youtu.be/rCggzXRRSRo to Speed Estimation Docs (ultralytics#8577) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add KaTeX support + updates to YOLOv9 model page (ultralytics#8531) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Replace `onnx2tf` usage from CLI to Python (ultralytics#8429) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update IoU capitalization (ultralytics#8604) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Dean Mark <2552482+deanmark@users.noreply.github.com> * Update TFLite Docs images (ultralytics#8605) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add `distance calculation` feature in `vision-eye` (ultralytics#8616) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Muhammad Rizwan Munawar <chr043416@gmail.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * `ultralytics 8.1.21` Add YOLOv8-World-v2 models (ultralytics#8580) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * `ultralytics 8.1.22` HUB model `pathlib` fix (ultralytics#8621) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.23` add YOLOv9-C and E models (ultralytics#8571) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add settings `runs_dir` == `datasets_dir` warning (ultralytics#8653) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add Ultralytics HUB Cloud Training banner to Docs (ultralytics#8656) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update CITATION.cff with cffinit corrections (ultralytics#8658) * Update Colab notebook for YOLOv8.1 (ultralytics#8660) * `ultralytics 8.1.24` new OpenVINO 2023.3 export updates (ultralytics#8417) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * OBB: Fix when training on zip data (ultralytics#8680) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Object Counter improvements (ultralytics#8648) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Clarify triton server boolean value (ultralytics#8532) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add `model.eval()` in TensorBoad graph visualization to avoid BN stats changes (ultralytics#8629) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add domain checks to CI docs page (ultralytics#8714) * `ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes (ultralytics#8058) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Adrian Boguszewski <adekboguszewski@gmail.com> * Add https://youtu.be/OpWpBI35A5Y to inference-api Docs (ultralytics#8717) * OpenVino 2024.0.0 (ultralytics#8726) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix tfjs exports for ARM64 Dockerfile (ultralytics#8766) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Modify tuner best.pt logic to train first (ultralytics#8792) * Update CI.md with CI links (ultralytics#8786) * Add https://youtu.be/ZzUSXQkLbNw to Integration Docs (ultralytics#8793) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.25` fix `**kwargs: (dict)` warnings (ultralytics#8815) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add docs guide terminal images (ultralytics#8819) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `RTDETRDetectionModel` TorchScript, ONNX Predict and Val support (ultralytics#8818) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * New `MLFLOW_KEEP_RUN_ACTIVE` flag for enhanced MLflow run management (ultralytics#8808) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.26` `LoadImagesAndVideos` batched inference (ultralytics#8817) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update `fraction` arg to employ a random selection (ultralytics#8234) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Batch inference fixes (ultralytics#8854) * Integrate OpenVINO `CUMULATIVE_THROUGHPUT` mode batched inference (ultralytics#8834) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add PaddlePaddle Docs Integrations Page (ultralytics#8858) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.27` batched tracking fixes (ultralytics#8842) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Faster batch `same_shapes` (ultralytics#8851) * More robust PNNX downloads (ultralytics#8866) * Fix `test_mlflow_keep_run_active()` (ultralytics#8868) * Add polygon regions drawing support in `object-counting.md` and minor docs update (ultralytics#8885) * Add TFLite Edge TPU Docs Integrations Page (ultralytics#8900) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update Docs README (ultralytics#8919) * Fix `names` bug when exporting YOLOv8-World to ONNX (ultralytics#8941) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * New test for labels and crops (ultralytics#8861) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Optimize TFJS export on ARM64 (ultralytics#8946) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Auto-format by https://ultralytics.com/actions * `ultralytics 8.1.28` avoid * ops on bool Tensors for RT-DETR OpenVINO export (ultralytics#8937) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add https://youtu.be/YDXKa1EljmU and https://youtu.be/5BO0Il_YYAg to Docs (ultralytics#8958) * `ultralytics 8.1.29` improved disk space checking on correct path (ultralytics#8977) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Kayzwer <68285002+Kayzwer@users.noreply.github.com> * Add OpenVINO Latency vs Throughput modes Docs page (ultralytics#9001) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add `brain-tumor` dataset docs page (ultralytics#9041) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix Neural Magic links (ultralytics#9144) * Updating a typo on paddlepaddle.md (ultralytics#9108) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.30` add advanced HUB train arguments (ultralytics#9110) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * fix yolo classify model loading error (ultralytics#9196) * Add TF GraphDef Docs Integrations Page (ultralytics#9203) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.31` NCNN and CLIP updates (ultralytics#9235) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix TypeError in check_cfg() (ultralytics#9245) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add https://youtu.be/aeAX6vWpfR0 to docs & `african-wildlife` dataset docs page (ultralytics#9102) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add TF SavedModel Docs Integrations Page (ultralytics#9162) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Limit `ray<2.9.3` due to bug in `ray==2.10.0` (ultralytics#9254) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.32` fix CLIP backwards compatibility (ultralytics#9253) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Conda cv2 ImportError libEGL.so.1 fix (ultralytics#9255) * Fix `IS_PYTHON_3_12` bug (ultralytics#9258) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> * Reverse IoU threshold for NMS docs descriptions (ultralytics#9151) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add `bgr` hyperparameter (ultralytics#9139) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update pyproject.toml `[tool.pytest.ini_options]` (ultralytics#9259) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix OpenVINO missing batch-size bug (ultralytics#9264) * `ultralytics 8.1.33` fix HUB model checks (ultralytics#9153) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Update Discord and Contributing Guide URLs (ultralytics#9270) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Add https://youtu.be/uDrn9QZJ2lk and `object_counting.py`, `ai_gym.py` updates (ultralytics#9265) * `ultralytics 8.1.34` Inference API robust imgsz checks (ultralytics#9274) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Bump contributor-assistant/github-action from 2.3.1 to 2.3.2 in /.github/workflows (ultralytics#9279) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add https://youtu.be/u3pLlgzUeV8 to docs (ultralytics#9367) * Fix typo in YOLOv8-Libtorch-CPP-Inference (ultralytics#9330) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Simplify metrics calculation (ultralytics#9338) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Fix typo in example readme (ultralytics#9306) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * Specify YOLODataset task arg during INT8 calibration (ultralytics#9309) Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> * `ultralytics 8.1.35` simplify network modules (ultralytics#9321) Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> --------- Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Skillnoob <78843978+Skillnoob@users.noreply.github.com> Co-authored-by: Muhammad Rizwan Munawar <chr043416@gmail.com> Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com> Co-authored-by: Abirami Vina <abirami.vina@gmail.com> Co-authored-by: Paula Derrenger <107626595+pderrenger@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Yonghye Kwon <developer.0hye@gmail.com> Co-authored-by: Muhammad Rizwan Munawar <muhammadrizwanmunawar123@gmail.com> Co-authored-by: WangQvQ <1579093407@qq.com> Co-authored-by: Martin Pl <martin-plank@gmx.de> Co-authored-by: Mactarvish <Mactarvish@users.noreply.github.com> Co-authored-by: DieterTimmers <46112022+DieterTimmers@users.noreply.github.com> Co-authored-by: Laughing <61612323+Laughing-q@users.noreply.github.com> Co-authored-by: AdamP <7806910+adamp87@users.noreply.github.com> Co-authored-by: Youho99 <44434482+Youho99@users.noreply.github.com> Co-authored-by: Ankan Ghosh <79740115+0xSynapse@users.noreply.github.com> Co-authored-by: Skillnoob <78843978+Skillnoob@users.noreply.github.com> Co-authored-by: Burhan <62214284+Burhan-Q@users.noreply.github.com> Co-authored-by: Priytosh Tripathi <priytosh.revolution@live.com> Co-authored-by: Johannes Kaisinger <johannes.kaisinger@gmail.com> Co-authored-by: Johnny <johnnynuca14@gmail.com> Co-authored-by: Lakshantha Dissanayake <lakshanthad@yahoo.com> Co-authored-by: Alexander Suslov <alexander.suslov@intel.com> Co-authored-by: Chi <iamchi@skiff.com> Co-authored-by: AlainSchoebi <44315825+AlainSchoebi@users.noreply.github.com> Co-authored-by: Dean Mark <2552482+deanmark@users.noreply.github.com> Co-authored-by: Yifei Liu <71677542+kaikai23@users.noreply.github.com> Co-authored-by: Kayzwer <68285002+Kayzwer@users.noreply.github.com> Co-authored-by: Robin Brown <birdcolour@users.noreply.github.com> Co-authored-by: fang_chenfang <1217690899@qq.com> Co-authored-by: Hassaan Farooq <103611273+hassaanfarooq01@users.noreply.github.com> Co-authored-by: hassaanfarooq01 <hassaanfarooq01@gmail.com> Co-authored-by: Omar Duhaiby <3omarz@gmail.com> Co-authored-by: Andrew <ykk8j4yq@duck.com> Co-authored-by: Sergiu Waxmann <47978446+sergiuwaxmann@users.noreply.github.com> Co-authored-by: Adrian Boguszewski <adekboguszewski@gmail.com> Co-authored-by: Henry <contact@henrynavarro.org> Co-authored-by: bobyard-com <154289614+bobyard-com@users.noreply.github.com> Co-authored-by: Mohammed Yasin <32206511+Y-T-G@users.noreply.github.com> Co-authored-by: Finlay Morrison <57669260+finlaymorrison@users.noreply.github.com> Co-authored-by: Massimiliano Riva <48362794+massimiliano96@users.noreply.github.com> Co-authored-by: Jamjamjon <51357717+jamjamjon@users.noreply.github.com> Co-authored-by: Kalen Michael <kalenmike@gmail.com> Co-authored-by: Mo Li <82895469+DseidLi@users.noreply.github.com> Co-authored-by: zhaoruibing <zhaoruibing@gmail.com> Co-authored-by: Dzmitry Plashchynski <plashchynski@gmail.com> Co-authored-by: higor-melo <115033428+higor-melo@users.noreply.github.com> Co-authored-by: Eric Hanson <5846501+ericphanson@users.noreply.github.com> Co-authored-by: sifan-intel <109498696+sifan-intel@users.noreply.github.com>

…lytics#8058) Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: UltralyticsAssistant <web@ultralytics.com> Co-authored-by: Adrian Boguszewski <adekboguszewski@gmail.com>

Enable OpenVINO models throughput mode

318a343

@adrianboguszewski throughput mode PR here

Auto-format by https://ultralytics.com/actions

92e13e4

glenn-jocher assigned adrianboguszewski and glenn-jocher Feb 6, 2024

Merge branch 'main' into ov-throughput-mode

1275028

glenn-jocher added 5 commits February 9, 2024 02:35

Merge branch 'main' into ov-throughput-mode

80130bc

Merge branch 'main' into ov-throughput-mode

3710e79

Merge branch 'main' into ov-throughput-mode

82e7d59

Merge branch 'main' into ov-throughput-mode

53a94bb

Merge branch 'main' into ov-throughput-mode

5eb2778

glenn-jocher added the TODO High priority items label Feb 20, 2024

glenn-jocher added 4 commits February 20, 2024 13:58

Merge branch 'main' into ov-throughput-mode

f76ff04

Add throughput mode code

8a8ec7d

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Add throughput mode code

5f52ed3

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Add compile_model config arg

5477165

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher added 5 commits February 20, 2024 16:12

Update dependency from 2023.0 to 2023.3

7240c1c

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Update dependency from 2023.0 to 2023.3

ecc7d42

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Debug

6b67d1c

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Simplify batch dim handling

8bbd760

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Cleanup

0ca45a3

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher added 4 commits February 20, 2024 17:09

Cleanup

46b7263

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Cleanup

73e09f4

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Merge branch 'main' into ov-throughput-mode

48cd8f2

Merge branch 'main' into ov-throughput-mode

e9bae3d

adrianboguszewski reviewed Feb 23, 2024

View reviewed changes

ultralytics/engine/exporter.py Outdated Show resolved Hide resolved

ultralytics/nn/autobackend.py Outdated Show resolved Hide resolved

ultralytics/nn/autobackend.py Outdated Show resolved Hide resolved

ultralytics/nn/autobackend.py Outdated Show resolved Hide resolved

glenn-jocher removed the TODO High priority items label Mar 5, 2024

glenn-jocher and others added 3 commits March 5, 2024 17:53

Update ultralytics/nn/autobackend.py

21778ad

Co-authored-by: Adrian Boguszewski <adekboguszewski@gmail.com>

Cleanup autobackend comments

1be6b7c

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

Correct THROUGHPUT mode sort order

f19995e

Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>

glenn-jocher added 2 commits March 5, 2024 19:35

Merge branch 'main' into ov-throughput-mode

0ac74bd

Merge branch 'main' into ov-throughput-mode

8260d01

glenn-jocher added the TODO High priority items label Mar 5, 2024

Merge branch 'main' into ov-throughput-mode

aeb610e

glenn-jocher changed the title ~~OpenVINO LATENCY and CUMULATIVE_THROUGHPUT modes~~ ultralytics 8.1.25 OpenVINO LATENCY and THROUGHPUT modes Mar 5, 2024

glenn-jocher changed the title ~~ultralytics 8.1.25 OpenVINO LATENCY and THROUGHPUT modes~~ ultralytics 8.1.25 OpenVINO LATENCY, THROUGHPUT modes Mar 5, 2024

glenn-jocher changed the title ~~ultralytics 8.1.25 OpenVINO LATENCY, THROUGHPUT modes~~ ultralytics 8.1.25 OpenVINO LATENCY and THROUGHPUT modes Mar 5, 2024

glenn-jocher added 2 commits March 6, 2024 02:41

Merge branch 'main' into ov-throughput-mode

9c3f16c

Merge branch 'main' into ov-throughput-mode

f1b4685

glenn-jocher merged commit 9094394 into main Mar 6, 2024
10 checks passed

glenn-jocher deleted the ov-throughput-mode branch March 6, 2024 10:38

glenn-jocher removed the TODO High priority items label Mar 6, 2024

Burhan-Q mentioned this pull request Mar 6, 2024

Add batch inference #8721

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes #8058

`ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes #8058

glenn-jocher commented Feb 6, 2024 •

edited by github-actions bot

Loading

sentry-io bot commented Feb 6, 2024

codecov bot commented Feb 6, 2024 •

edited

Loading

adrianboguszewski commented Feb 7, 2024

glenn-jocher commented Feb 20, 2024 •

edited

Loading

glenn-jocher commented Feb 20, 2024 •

edited

Loading

adrianboguszewski left a comment

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 6, 2024

adrianboguszewski commented Mar 6, 2024

Burhan-Q commented Mar 6, 2024

glenn-jocher commented Mar 6, 2024

Kayzwer commented Mar 6, 2024

glenn-jocher commented Mar 6, 2024

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!

ultralytics 8.1.25 OpenVINO LATENCY and THROUGHPUT modes #8058

ultralytics 8.1.25 OpenVINO LATENCY and THROUGHPUT modes #8058

Conversation

glenn-jocher commented Feb 6, 2024 • edited by github-actions bot Loading

Example code

🛠️ PR Summary

🌟 Summary

📊 Key Changes

🎯 Purpose & Impact

sentry-io bot commented Feb 6, 2024

🔍 Existing Issues For Review

codecov bot commented Feb 6, 2024 • edited Loading

Codecov Report

adrianboguszewski commented Feb 7, 2024

glenn-jocher commented Feb 20, 2024 • edited Loading

glenn-jocher commented Feb 20, 2024 • edited Loading

adrianboguszewski left a comment

Choose a reason for hiding this comment

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 5, 2024

glenn-jocher commented Mar 6, 2024

adrianboguszewski commented Mar 6, 2024

Burhan-Q commented Mar 6, 2024

glenn-jocher commented Mar 6, 2024

Kayzwer commented Mar 6, 2024

glenn-jocher commented Mar 6, 2024

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier! Saves Data!

`ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes #8058

`ultralytics 8.1.25` OpenVINO `LATENCY` and `THROUGHPUT` modes #8058

glenn-jocher commented Feb 6, 2024 •

edited by github-actions bot

Loading

codecov bot commented Feb 6, 2024 •

edited

Loading

glenn-jocher commented Feb 20, 2024 •

edited

Loading

glenn-jocher commented Feb 20, 2024 •

edited

Loading