Above PR should resolve this issue by raise error. For the example you specified in the issue description, following is the error message that will be raised

ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Please check the target modules and 
try again.

Above PR should resolve this issue by raise error. For the example you specified in the issue description, following is the error message that will be raised

ValueError: Target modules ['q_proj', 'v_proj'] not found in the base model. Please check the target modules and 
try again.

I know maybe it's a different question but how do I solve this if I get this error? How to know what target module name I need?

hi @nameless0704, did you get any solution for the error?
My error was solved by changing target module to ["query_key_value"] but I don't know how to find for different models?

Hello, for models that aren't in the https://github.com/huggingface/peft/blob/main/src/peft/utils/other.py#L202, you need to look into the modeling_*.py file in transformers https://github.com/huggingface/transformers/tree/main/src/transformers/models

for example, you can vim to view your model layout file like pytorch_model.bin.index.json
you can see something like
model.layers.3.self_attn.q_proj.weight

or
transformer.encoder.layers.5.self_attention.query_key_value.weight

Can you help me understand what parameter will it be in this model, I can see there is nothing like q_proj like in the original phi model. The repo id is following: TheBloke/phi-2-GPTQ/modeling_phi.py

Can you help me understand what parameter will it be in this model, I can see there is nothing like q_proj like in the original phi model. The repo id is following: TheBloke/phi-2-GPTQ/modeling_phi.py

We have documented how to apply adapters to models that are not pre-configured here. Moreover, you can pass config = LoraConfig(..., target_modules="all-linear") to apply LoRA to all linear layers, which often works well.