@ShareLer Thanks a lot for helping us to fix this, it helps a lot~

Could you briefly point out what causes vllm inference failure? Seems also involved a lot of refactorization. Is it caused by the missing parameters transferring?

@ShareLer Thanks a lot for helping us to fix this, it helps a lot~

Could you briefly point out what causes vllm inference failure? Seems also involved a lot of refactorization. Is it caused by the missing parameters transferring?

Three main reasons:

1. The name of the converted layer has not been modified in _merge_state_dicts().
   model.decoder.layers.xxx in converted ckpt, but the actual is supposed to be model.layers.xxx.
   This is also the reason for the failure in the mentioned issue.

2. The name of qkv in the attention layer is error after converted.
   linear_qkv is converted to linear_q/linear_k/linear_v, but in reality it should be q_proj/k_proj/v_proj

3. The weight of the final output layer is processed incorrectly.
   When is_value_model=False, the output_layer is ColumnParallelLinear, but the weights of different TP were not merged in _merge_state_dicts().
   (When submitting for the first time, my method ignored the value_model, resulting in the failure of CI. It has just been fixed.)

is it possible to add a test that reproduces the issue?

You can reproduce this problem very simply by using the CI script (like job 'e2e_ppo_trainer_megatron-qwen3' in e2e_ppo_trainer_megatron.yml) just change command option in merger：
You need change the test operation in python scripts/model_merger.py test --backend megatron to merge.

There were no problems in the previous CI test because different logics were used in the test and merge options:
First of all, they all obtained the merged weights through the _merge_state_dicts() method. However, it should be noted that there are some problems with state_dicts at this time (the three problems in the previous reply).
Next, in the merge option, this problematic state_dicts was directly saved as the final ckpt. But these problematic layer names were corrected in the test option (remove the decoder and correct the name of the qkv) which used in CI.