Practical course about Large Language Models.

[SIGIR'24] The official implementation code of MOELoRA.

Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"

An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT

Official repository of my book "A Hands-On Guide to Fine-Tuning LLMs with PyTorch and Hugging Face"

Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"

[ICML'24 Oral] APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference

CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model

Official code implemtation of paper AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?

memory-efficient fine-tuning; support 24G GPU memory fine-tuning 7B

AI Community Tutorial, including: LoRA/Qlora LLM fine-tuning, Training GPT-2 from scratch, Generative Model Architecture, Content safety and control implementation, Model distillation techniques, Dreambooth techniques, Transfer learning, etc for practice with real project!

High Quality Image Generation Model - Powered with NVIDIA A100

A Python library for efficient and flexible cycle-consistency training of transformer models via iteratie back-translation. Memory and compute efficient techniques such as PEFT adapter switching allow for 7.5x larger models to be trained on the same hardware.

[ICML 2025] Fast and Low-Cost Genomic Foundation Models via Outlier Removal.

【AIGC 实战入门笔记 —— AIGC 摩天大楼】分享 大语言模型（LLMs），大模型高效微调（SFT）,检索增强生成（RAG），智能体（Agent），PPT自动生成, 角色扮演，文生图（Stable Diffusion） ，图像文字识别（OCR），语音识别（ASR），语音合成（TTS），人像分割（SA），多模态（VLM），Ai 换脸(Face Swapping), 文生视频(VD)，图生视频（SVD），Ai 动作迁移，Ai 虚拟试衣，数字人，全模态理解（Omni），Ai音乐生成 干货学习 等 实战与经验。

Mistral and Mixtral (MoE) from scratch

EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition

Fine-tune StarCoder2-3b for SQL tasks on limited resources with LORA. LORA reduces model size for faster training on smaller datasets. StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The Stack v2 and some natural language text such as Wikipedia, Arxiv, and GitHub issues.

Fine-tuning Llama3 8b to generate JSON formats for arithmetic questions and process the output to perform calculations.

This repo is dedicated to an analysis of the LLMs performance as text classifiers on five German datasets from social media covering 13 tasks. We experiment with zero- and few-shot classification approaches as well as with fine-tuning using various LLMs