pip install git+https://github.com/mlflow/mlflow.git@refs/pull/11014/merge

gh pr checkout 11014

What changes are proposed in this pull request?

In predict() method of _TransformersWrapper, self.model_config is modified in-place for a few times, e.g. (1) params overrides model config values (2) some pop() operations to remove our custom keys like include_prompt before actual pipeline inference. These updates directly change the original model config, so as a result, the next prediction uses the different model config tweaked by the first call.

E.g.

model_config = {
    "top_k": 2,
    "num_beams": 5,
    "max_length": 30,
}
    
pyfunc_model = _TransformersWrapper(text2text_generation_pipeline, model_config=model_config)
assert pyfunc_model.model_config["top_k"] == 2

# Params will be used to override the values of model_config
params = {
    "top_k": 3,
    "max_length": 50,
}
pyfunc_model.predict("How to learn Python in 3 weeks?", params=params)
    
>       assert pyfunc_model.model_config["top_k"] == 2
E       assert 3 == 2

This also cause another issue in model logging, where incorrect model config is saved. This is because we run sometimes multiple prediction during model saving e.g. when user specified input_example we generate model output to infer signature from it. Then the saved model config is the one tweaked by these predictions.

Note: I hit this issue while development for another issue and running test_instructional_pipeline_no_prompt_in_output test, which is skipped in CI (due to large model size). Other CI tests didn't catch this, because most of them specify signature manually + not giving input example so only single prediction call only in the test case.

How is this PR tested?

• Existing unit/integration tests
• New unit/integration tests
• Manual tests

Does this PR require documentation update?

• No. You can skip the rest of this section.
• Yes. I've updated:
  • Examples
  • API references
  • Instructions

We probably need to update some document to warn users for this model config issue, as we suggest users to use the option (example). I can do that as a follow-up.

Release Notes

Is this a user-facing change?

• No. You can skip the rest of this section.
• Yes. Give a description of this change to be included in the release notes for MLflow users.

Fix idempotent prediction issue due to in-place update to model_config, which can also cause incorrect metadata to be saved.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

• area/artifacts: Artifact stores and artifact logging
• area/build: Build and test infrastructure for MLflow
• area/deployments: MLflow Deployments client APIs, server, and third-party Deployments integrations
• area/docs: MLflow documentation pages
• area/examples: Example code
• area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
• area/models: MLmodel format, model serialization/deserialization, flavors
• area/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templates
• area/projects: MLproject format, project running backends
• area/scoring: MLflow Model server, model deployment tools, Spark UDFs
• area/server-infra: MLflow Tracking server backend
• area/tracking: Tracking Service, tracking client APIs, autologging

Interface

• area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev server
• area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
• area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
• area/windows: Windows support

Language

• language/r: R APIs and clients
• language/java: Java APIs and clients
• language/new: Proposals for new client languages

Integrations

• integrations/azure: Azure and Azure ML integrations
• integrations/sagemaker: SageMaker integrations
• integrations/databricks: Databricks integrations

How should the PR be classified in the release notes? Choose one:

• rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
• rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
• rn/feature - A new user-facing feature worth mentioning in the release notes
• rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
• rn/documentation - A user-facing documentation change worth mentioning in the release notes