Repositories list

• .github

  Public
  0•0•0•0•Updated Jan 31, 2025Jan 31, 2025

• nndeploy

  Public
  nndeploy is an end-to-end model deployment framework. Based on multi-terminal inference and directed acyclic graph model deployment, it is committed to providing users with a cross-platform, easy-to-use, and high-performance model deployment experience.

  hpcparallelyolo+ 9easy-to-usetensorrtmnnmodel-deploymentopenvinoascendrknn+ 2

  C++

  •

  Apache License 2.0

  •101•682•7•0•Updated Jan 29, 2025Jan 29, 2025

• safetensors-cpp

  Public
  Header-only safetensors loader and saver in C++

  C++

  •

  MIT License

  •11•0•0•0•Updated Nov 19, 2024Nov 19, 2024

• onnx-llm

  Public
  llm deploy project based onnx.

  C++

  •

  Apache License 2.0

  •7•0•0•0•Updated Oct 9, 2024Oct 9, 2024

• tokenizers-cpp

  Public
  Universal cross-platform tokenizers binding to HF and sentencepiece

  C++

  •

  Apache License 2.0

  •69•1•0•0•Updated Jun 3, 2024Jun 3, 2024

• Awesome-LLM-Inference

  Public
  💻A small Collection for Awesome LLM Inference [Papers|Blogs|Docs] with codes, contains TensorRT-LLM, streaming-llm, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

  GNU General Public License v3.0

  •226•2•0•0•Updated Dec 3, 2023Dec 3, 2023

• onnx-simplifier

  Public
  Simplify your onnx model

  Python

  •

  Apache License 2.0

  •388•1•0•0•Updated Apr 27, 2022Apr 27, 2022