9 Pull requests merged by 7 people

• Fix bug when using pissa method with deepspeed

  #4580 merged Jun 27, 2024

• Exit the process with the subprocess's return code when utilizing the CLI

  #4590 merged Jun 27, 2024

• support flash-attn in Dockerfile

  #4461 merged Jun 27, 2024

• fix torch-npu dependency

  #4561 merged Jun 27, 2024

• fix docker-compose path

  #4544 merged Jun 26, 2024

• [Enhancement] Support ZeRO-3 when using BAdam

  #4352 merged Jun 24, 2024

• Add docker-npu

  #4355 merged Jun 24, 2024

• Print help if no arguments given

  #4409 merged Jun 24, 2024

• Add tool_format parameter to rewrite templates for different function call formats.

  #4417 merged Jun 24, 2024

141 Issues closed by 4 people

• 预测推理特别慢，跑完GPU利用率为0了一直卡在那里好像是构建generation

  #4638 closed Jul 1, 2024

• Qwen2 lora微调后用llamafactory-cli export命令合并模型 推理结果有"assstant: "前缀

  #4639 closed Jul 1, 2024

• 能不能把eval loss曲线加到训练过程中？

  #4637 closed Jul 1, 2024

• PiSSA训练和推理的疑问？

  #4634 closed Jul 1, 2024

• Qwen2 debug 发现 labels全为-100

  #4635 closed Jul 1, 2024

• 如何在yaml中配置环境变量中tensorboard的路径呢

  #4633 closed Jul 1, 2024

• 请问可以微调没有lm head模型吗

  #4630 closed Jul 1, 2024

• Qwen dpo训练卡住

  #4631 closed Jul 1, 2024

• 910b qwen2 lora生成的模型如何合并权重

  #4632 closed Jul 1, 2024

• deepspeed ds_z3_offload_config单卡全量微调训练glm4出现exits with return code = -9。出现该问题时，CPU内存（252G）占满，想问一下这个问题该如何解决？

  #4629 closed Jul 1, 2024

• 大模型微调分类任务，但是预测结果是不固定的

  #4627 closed Jul 1, 2024

• 4张M40 配置，使用accelerate启动训练，出现TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

  #4626 closed Jun 30, 2024

• windows上start直接Fail并出现llamafactory-cli乱码

  #4625 closed Jun 30, 2024

• DPO 训练时，prompt 与 answer 拼接问题，导致cutoff_length这一超参数无法对数据进行有效截断。

  #4617 closed Jun 30, 2024

• Cutoff Length only followed for chosen response in Pairwise Data for DPO

  #4402 closed Jun 30, 2024

• 训练34B-reward，Assertion `srcIndex < srcSelectDimSize` failed

  #4624 closed Jun 30, 2024

• 华为NPU训练不了，用的例子里的训练脚本，镜像也是官方镜像

  #4610 closed Jun 30, 2024

• ModuleNotFoundError: No module named 'vllm.lora'

  #4623 closed Jun 30, 2024

• 使用A10对qwen-14b-chat进行Lora微调，2机2卡训练比1机2卡慢了10倍

  #4620 closed Jun 29, 2024

• 什么时候支持基于Ray的分布式Lora微调呢？

  #4619 closed Jun 29, 2024

• qwen2-72b DPO 训练爆显存，OOM 问题；

  #4616 closed Jun 29, 2024

• 偶尔输出第一个符号是冒号

  #3883 closed Jun 28, 2024

• 希望支持InternVL视觉模型的微调

  #3802 closed Jun 28, 2024

• 关于Qwen2-72B 全量参数微调所需的显卡下限

  #4141 closed Jun 28, 2024

• Do you have plans to add fine-tuning scripts for other multimodal large models? For example, Qwen_VL, LLaVA1.6, MiniGPT4, etc.

  #4174 closed Jun 28, 2024

• [NPU]目前华为昇腾+llamaFactory多卡训练和推理

  #3959 closed Jun 28, 2024

• 顺利在 Apple silicon M3 上运行 README 中 Llama3-8B 相关示例工作流的小波折

  #4341 closed Jun 28, 2024

• 全参pt微调Qwen2-7B-Instruct模型，中断后继续训练，修改了lr但没生效

  #4595 closed Jun 28, 2024

• How to use a fine-tuned model to evaluate on a testset and save the output of the model?

  #4611 closed Jun 28, 2024

• ValueError: Output directory already exists and is not empty. Please set overwrite_output_dir.

  #4612 closed Jun 28, 2024

• 是否支持01-ai/Yi-VL-6B

  #4602 closed Jun 28, 2024

• 怎样使用accelerate库进行微调呢？

  #4601 closed Jun 28, 2024

• 你好，请问 KTO是否支持history？

  #4603 closed Jun 28, 2024

• 能加入matmulfreellm吗？

  #4604 closed Jun 28, 2024

• 大佬，fp8会考虑支持吗？

  #4607 closed Jun 28, 2024

• Gemma2

  #4605 closed Jun 28, 2024

• kto训练完如何预测

  #4600 closed Jun 28, 2024

• glm系列模型做eval时应该将template参数设为什么

  #4598 closed Jun 28, 2024

• 更新了最新代码，为什么webui里的模型选择看不到Qwen2?

  #4599 closed Jun 28, 2024

• npu支持GPTQ量化导出吗

  #4594 closed Jun 28, 2024

• 请问支持模型并行吗？如果我想要在48GB*8显卡上全量微调llama3-8b，怎么设置呢？

  #4596 closed Jun 28, 2024

• [bug] unsloth坏了

  #4593 closed Jun 27, 2024

• Raise error when using pissa method with deepspeed

  #4579 closed Jun 27, 2024

• 执行命令 报错 flash_attn未安装, 安装后报错ImportError. 使用docker compose ,docker同样的问题

  #4592 closed Jun 27, 2024

• PISSA模式下进行qLORA训练，指定了lora_rank=8，但是训练出来的adaptor_config中是lora_rank=16

  #4586 closed Jun 27, 2024

• 有离线安装方案嘛

  #4588 closed Jun 27, 2024

• 模型没有加载完，gpu利用率已经是100%了

  #4591 closed Jun 27, 2024

• llamafactory 支持 deepspeed zero3么？

  #4585 closed Jun 27, 2024

• 72b模型单机多卡训练，llamafctory对单卡的显存有要求么？

  #4584 closed Jun 27, 2024

• predict 能否自定义添加output字段

  #4583 closed Jun 27, 2024

• 训练数据未被shuffle？

  #4582 closed Jun 27, 2024

• 运行webui.py训练出错

  #4577 closed Jun 27, 2024

• 为什么predict阶段没法支持数据&模型并行呢？

  #4576 closed Jun 27, 2024

• 如何指定某个数据集作为eval数据集？

  #4581 closed Jun 27, 2024

• 请问是否支持模型的awq量化？

  #4575 closed Jun 27, 2024

• 请问training eval loss能否指定一个单独的数据集？

  #4574 closed Jun 27, 2024

• WARNING - llamafactory.model.model_utils.quantization - quantization_bit will not affect on the PTQ-quantized models

  #4573 closed Jun 27, 2024

• 910b 280t 8机64卡 跑qwen2 7b cutoff_len: 8192 一直OOM

  #4572 closed Jun 27, 2024

• Does LLaMA-Factory support AMD graphic cards?

  #3252 closed Jun 26, 2024

• oom when sft Qwen1.5-32B-Chat using deepspeed z3 offload + bs1

  #3259 closed Jun 26, 2024

• 使用KTO进行多机训练过程中再进行验证，报错RuntimeError: still have inglight params [{id：388, "status":"AVALIBLE"}]

  #3852 closed Jun 26, 2024

• [Feature Request] Support for LangBridge Integration for Multilingual Reasoning

  #3203 closed Jun 26, 2024

• On merging the PT Lora Adaptor to base model it increased the size to almost double to that of orig base model.

  #3137 closed Jun 26, 2024

• Qwen: Deepspeed(Zero3) + DPO error

  #2774 closed Jun 26, 2024

• 新人请教

  #4283 closed Jun 26, 2024

• deepseek-moe16B全量参数微调断点续训失败

  #4368 closed Jun 26, 2024

• A100上训练qwen2模型异常

  #4407 closed Jun 26, 2024

• 8*A800 80G lora训练qwen2-72B模型 内存占用异常

  #4453 closed Jun 26, 2024

• Out of Memory Error on Sagemaker while training LLava on 93000 images

  #4562 closed Jun 26, 2024

• Will you support HQQ quantization in the future?

  #4113 closed Jun 26, 2024

• Question about data preprocess

  #4570 closed Jun 26, 2024

• 关于基座模型和对话模型的疑问

  #4457 closed Jun 26, 2024

• lora微调后的glm4模型不生成回答

  #4454 closed Jun 26, 2024

• 请教多轮工具调用的数据格式

  #4548 closed Jun 26, 2024

• 微调llama3-8b的时候，eval_loss不断上升，考虑到了使用多个数据集混合，但还是没有效果，应该怎么解决？

  #4554 closed Jun 26, 2024

• [Help] Yi-34B + Simpo + Full + Novel writing task, fine-tuning results are unsatisfactory!

  #4563 closed Jun 26, 2024

• kto训练要求response大于1(feedback_dataset)函数

  #4564 closed Jun 26, 2024

• 【问题】为什么要把可训练参数精度强行转换为全精度？

  #4549 closed Jun 26, 2024

• PPO 跑example例子报错：value should be one of int, float, str, bool, or torch.Tensor

  #4458 closed Jun 26, 2024

• API_PORT=8000 llamafactory-cli api examples/inference/qwen2_vllm.yaml报错

  #4555 closed Jun 26, 2024

• llama board测评模型效果差

  #4558 closed Jun 26, 2024

• unsloth只支持lora吗。不支持pretrain？

  #4560 closed Jun 26, 2024

• CUDA out of memory | QLORA | Llama 3 70B | 4 * NVIDIA A10G 24 Gb

  #4559 closed Jun 26, 2024

• [bug] pissa init脚本报错

  #4556 closed Jun 26, 2024

• Get "RuntimeError: 'weight' must be 2-D" Error when finetuning llama3-8b using ZeRO3 and customised dataset

  #4557 closed Jun 26, 2024

• Llama3 微调文本分类数据集准备的格式，以及微调后模型回答输出非标签内容

  #4550 closed Jun 26, 2024

• llamafactory-cli支持输出向量吗？

  #4553 closed Jun 26, 2024

• vllm部署freeze微调后的qwen2-57b-instruct报错

  #4545 closed Jun 26, 2024

• llamafactory-cli api examples/inference/qwen2_vllm.yaml报错

  #4551 closed Jun 26, 2024

• Help！ 训练结束加载大模型，使用训练数据集中的问题向大模型提问，答案不是训练数据集中的答案。训练耗时不足2分钟，怀疑训练失败，请大佬们帮忙诊断一下！！

  #4552 closed Jun 26, 2024

• lora微调Qwen-14b-chat后导入模型报错

  #4547 closed Jun 26, 2024

• 出一个中文版的操作手册

  #4546 closed Jun 26, 2024

• how to authenticate to access huggingface

  #4543 closed Jun 26, 2024

• 什么时候可以支持glm4v等多模态模型微调呀

  #4542 closed Jun 26, 2024

• scripts/pissa_init.py to initialize PiSSA for a quantized model.

  #4463 closed Jun 25, 2024

• 对sft阶段的数据进行packing之后，同一条训练数据内的指令之间是否会相互影响？

  #4462 closed Jun 25, 2024

• deepspeed zero3 出现 training_eval_loss 图为空白

  #4459 closed Jun 25, 2024

• ## feature request ## 支持 ZeRO3 infinity

  #4460 closed Jun 25, 2024

• LoRA微调和全参微调的时候总是会出现过拟合，在无法提高数据集大小的情况下，应该如何解决这个问题呢

  #4452 closed Jun 25, 2024

• docker容器内没有example和data文件

  #4456 closed Jun 25, 2024

• 最新代码中没有llamafactory-cli ，怎么合并权重

  #4455 closed Jun 25, 2024

• 请问训练框架能否支持Megatron-DeepSpeed来进行多机多卡训练？

  #2956 closed Jun 24, 2024

• LoftQ + Unsloth ?

  #3255 closed Jun 24, 2024

• 0.6.2版本进行mmlu榜单测试，出现问题

  #3614 closed Jun 24, 2024

• 我应该用什么脚本训练LoftQ？

  #3685 closed Jun 24, 2024

• 预训练codeqwen1.5-7b时显存分布异常，训练一段时间后爆OOM

  #3908 closed Jun 24, 2024

• 使用多GPU单节点的example进行尝试的时候失败

  #3953 closed Jun 24, 2024

• 超过了设置的最大token数，模型还是有返回

  #3969 closed Jun 24, 2024

• glm4的template似乎和官方未对齐

  #4289 closed Jun 24, 2024

• qwen1.5_7B使用Zero-2方式在8张A100(40G)和64张910A(32G)上SFT，报OOM

  #4133 closed Jun 24, 2024

• 请问有大佬全量sft过qwen2-72B或者qwen1.5-72B的模型吗？有参数推荐吗

  #4255 closed Jun 24, 2024

• llama_factory安装后执行校验llamafactory-cli train -h时报错

  #4322 closed Jun 24, 2024

• 量化训练、推理问题

  #4333 closed Jun 24, 2024

• 在奖励模型训练完成之后，model.safetensor不能保存到本地，提示权限不足

  #4418 closed Jun 24, 2024

• Error during inference LLaMA2 + LoRA: RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::Half != float

  #4432 closed Jun 24, 2024

• 保存后的Reward Model用来inference

  #4379 closed Jun 24, 2024

• PPO微调没有保存出模型：Trainer.model is not a `PreTrainedModel`, only saving its state dict.

  #4416 closed Jun 24, 2024

• 前端页面无法预览训练集

  #4367 closed Jun 24, 2024

• 在llama3-8B-Instruct上进行全参数ICL微调后，模型的ICL效果变的很差，求助

  #4259 closed Jun 24, 2024

• 数据集./data/identity.json 格式转换报错

  #4394 closed Jun 24, 2024

• 用docker compose启动llama-factory页面hold住了

  #4404 closed Jun 24, 2024

• 并行工具响应返回，formmatter tool extract实现效果不太好

  #4405 closed Jun 24, 2024

• TrainerState.__init__() got an unexpected keyword argument 'stateful_callbacks'

  #4406 closed Jun 24, 2024

• 单机多卡全参数微调glm4报错

  #4426 closed Jun 24, 2024

• 为什么webui服务没法启动呢

  #4419 closed Jun 24, 2024

• 如何指定已划分好的训练集和验证集？

  #4451 closed Jun 24, 2024

• Memory Error during tokenization while fine tuning LLava1.5-7B-Chat more than 8000 images

  #4450 closed Jun 24, 2024

• 在 webui chat 上自定义基模型（和 LoRA adapter）的浮点数表示方式

  #4410 closed Jun 24, 2024

• DPO训练导致模型能力遗忘的一些疑问

  #4413 closed Jun 24, 2024

• api访问流式输出报错

  #4415 closed Jun 24, 2024

• 训练多个lora合并后乱码

  #4421 closed Jun 24, 2024

• RuntimeError: disagreement between rank0 and rank6 while fully supervised finetuning deepseek v2 lite during the evaluation phase.

  #4428 closed Jun 24, 2024

• LLaMA to LLaVA

  #4427 closed Jun 24, 2024

• glm-4-9b-chat在使用web_demo进行推理时报错

  #4429 closed Jun 24, 2024

• 沒有過擬和的狀況，但是loss到一個點後就會難以下降，並且推理對話會有重覆內容

  #4434 closed Jun 24, 2024

• 1*8 H20微调qwen2-72b-instruct，保存模型失败

  #4435 closed Jun 24, 2024

• Low MMLU of llama2

  #4436 closed Jun 24, 2024

• 单机多卡微调glm4-9B设置max_grad_norm=1,但是仍然出现了梯度爆炸的问题

  #4438 closed Jun 24, 2024

• 预训练方式lora微调Qwen2 base模型，是否需要添加template

  #4439 closed Jun 24, 2024

• How to pre-train Llava1.5 from vicuna1.5?

  #4440 closed Jun 24, 2024

• 训练glm4报错：RuntimeError when using flash attention with 8-bit quantization，同样的参数训llama3则没问题

  #4441 closed Jun 24, 2024

8 Issues opened by 8 people

• Yi-1.5-9B推理gpu利用率为0

  #4628 opened Jul 1, 2024

• RuntimeError: "addmm_impl_cpu_" not implemented for 'Half' 华为910 命令行推理报错

  #4622 opened Jun 30, 2024

• 关于对话模板作用以及其在lm-evaluation-harness仓库下对评测效果影响的问题

  #4618 opened Jun 29, 2024

• 🚨FAQs | 常见问题🚨

  #4614 opened Jun 28, 2024

• ppo合并失败

  #4609 opened Jun 28, 2024

• fsdp + DPO + fullyfintune会报错

  #4608 opened Jun 28, 2024

• [PPU]大佬有对ppu环境进行过测试么

  #4606 opened Jun 28, 2024

• 8卡A800全参数预训练GLM4-9B-base,使用bf16,loss在暴涨后突然消失

  #4597 opened Jun 28, 2024

6 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Feature/support qwenvl glm4-v phi3-v(tested)

  #4377 commented on Jul 1, 2024 • 25 new comments

• Phi-3-small exploding gradient issue.

  #3881 commented on Jun 27, 2024 • 1 new comment

• Add dataset % sample num equally distribute

  #3976 commented on Jun 28, 2024 • 1 new comment

• 可以支持英伟达的新模型Nemotron 340B吗?

  #4313 commented on Jun 24, 2024 • 0 new comments

• Support Several MLLM Models

  #4136 commented on Jul 1, 2024 • 0 new comments

• Implement efficient packing without cross-contamination attention

  #4224 commented on Jul 1, 2024 • 0 new comments