We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
在npu上训练deepseek系列模型,需要flash attn库但是因为冲突 npu无法使用该库导致无法训练 请问是否考虑支持呢 可能需要把向前推理换成flashattn算子:https://www.hiascend.com/document/detail/zh/Pytorch/60RC1/ptmoddevg/trainingmigrguide/performance_tuning_0027.html
llamafactory cli train
希望可以支持npu训练deepseek
No response
The text was updated successfully, but these errors were encountered:
研究了一下代码主要涉及的部分是longlora.py中的LlamaFlashAttention2和LlamaSdpaAttention 可能需要按照https://www.hiascend.com/document/detail/zh/Pytorch/60RC1/ptmoddevg/trainingmigrguide/performance_tuning_0027.html 将transformer中的LlamaFlashAttention2文件第516和531行做替换为文档中的torch_npu.npu_fusion_attention Sdpa同理 可能可以支持一下
Sorry, something went wrong.
同时需要更改 模型中的modeling_deepseek.py
主要需要gpu2npu模型迁移 https://www.hiascend.com/document/detail/zh/Pytorch/60RC1/ptmoddevg/trainingmigrguide/PT_LMTMOG_0016.html
No branches or pull requests
Reminder
System Info
在npu上训练deepseek系列模型,需要flash attn库但是因为冲突 npu无法使用该库导致无法训练
请问是否考虑支持呢 可能需要把向前推理换成flashattn算子:https://www.hiascend.com/document/detail/zh/Pytorch/60RC1/ptmoddevg/trainingmigrguide/performance_tuning_0027.html
Reproduction
llamafactory cli train
Expected behavior
希望可以支持npu训练deepseek
Others
No response
The text was updated successfully, but these errors were encountered: