Skip to content

vllm cannot load model after megatron training #1757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
KAKSIS opened this issue May 29, 2025 · 0 comments
Open

vllm cannot load model after megatron training #1757

KAKSIS opened this issue May 29, 2025 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@KAKSIS
Copy link

KAKSIS commented May 29, 2025

After megatron training and convert to hf model, i want to infer using vllm, which meet problem when loading.
File "/python3.11/site-packages/vllm/model_executor/models/utils.py", line 250, in _load_module
raise ValueError(msg)
ValueError: There is no module or parameter named 'decoder' in Qwen3ForCausalLM
After model_merger.py process, the name of parameters changed.
@ccclyu ccclyu added the help wanted Extra attention is needed label May 29, 2025
@ShareLer ShareLer mentioned this issue May 30, 2025
Open
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ccclyu @KAKSIS
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy