I am currently fine-tuning llama-2 models (7b, 13b, and 70b) with prompt tuning methodology using the peft library.

My peft_config is defined as.

    peft_config = PromptTuningConfig(
        task_type=TaskType.CAUSAL_LM,
        prompt_tuning_init=PromptTuningInit.TEXT,
        num_virtual_tokens=20,
        prompt_tuning_init_text="my init text.",
        tokenizer_name_or_path=model_id, 
    )

After training, I was able to combine the base model and the generated adapter as show below;

model = PeftModel.from_pretrained(base_model, peft_model)

but I could only save the adapter to disk and not the combined model.

It seems to me that the "merge_and_unload" function in peft is only applicable to LoRA or am I missing something

Is there a way to save the combined model to disk as a standalone model?