TensorFlow renames inputs when restoring/resaving SavedModel #50130

dwyatte · 2021-06-07T20:22:42Z

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Any
TensorFlow installed from (source or binary): Binary
TensorFlow version (use command below): tf-nightly
Python version: 3.7
CUDA/cuDNN version: CPU
GPU model and memory: CPU

Describe the current behavior
When a user restores a SavedModel and resaves it, TensorFlow renames the model inputs in the signature, prefixing them with "inputs/". I'm not sure when this behavior was introduced (it happened non-deterministically in tensorflow==2.4.1, but is deterministic in tensorflow==2.5.0 and tf-nightly) or expected. I have a use case where I am constrained in my input names in the signature, although I can work around it if renaming is expected (and deterministic) so this issue is mainly to raise awareness if the renaming is unexpected

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.

import subprocess
import tensorflow as tf

inputs = {
    "first_feature": tf.constant([1, 2, 3]),
    "second_feature": tf.constant([4, 5, 6]),
}

class MyModel(tf.keras.Model):
    def call(self, inputs, training=None):
        return tf.random.uniform([3, 1])

model = MyModel()
model.compile(optimizer="sgd", loss="mse")
model.fit(inputs, tf.constant([7, 8, 9]))
model.save("model")
dict_model_def = subprocess.check_output("saved_model_cli show --dir model --tag_set serve --signature_def serving_default".split())
print("MODEL")
print(dict_model_def.decode("utf-8"))

print("RESTORED MODEL")
restored_model = tf.keras.models.load_model("model")
restored_model.save("restored_model")
restored_model_def = subprocess.check_output("saved_model_cli show --dir restored_model --tag_set serve --signature_def serving_default".split())
print(restored_model_def.decode("utf-8"))

Output:

1/1 [==============================] - 0s 117ms/step - loss: 56.5648
MODEL
The given SavedModel SignatureDef contains the following input(s):
  inputs['first_feature'] tensor_info:
      dtype: DT_INT32
      shape: (-1, 1)
      name: serving_default_first_feature:0
  inputs['second_feature'] tensor_info:
      dtype: DT_INT32
      shape: (-1, 1)
      name: serving_default_second_feature:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['output_1'] tensor_info:
      dtype: DT_FLOAT
      shape: (3, 1)
      name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict

RESTORED MODEL
The given SavedModel SignatureDef contains the following input(s):
  inputs['inputs/first_feature'] tensor_info:
      dtype: DT_INT32
      shape: (-1, 1)
      name: serving_default_inputs/first_feature:0
  inputs['inputs/second_feature'] tensor_info:
      dtype: DT_INT32
      shape: (-1, 1)
      name: serving_default_inputs/second_feature:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['output_1'] tensor_info:
      dtype: DT_FLOAT
      shape: (3, 1)
      name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict

The text was updated successfully, but these errors were encountered:

saikumarchalla · 2021-06-10T15:45:29Z

Was able to reproduce the issue in TF 2.5 but in TF 2.4 model inputs are same. Please find the attached gists. Thanks!

jesford · 2021-06-10T20:12:33Z

@saikumarchalla if I re-run your TF 2.4 gist a few times I do get the inputs modified intermittently about half the time.

dwyatte · 2021-07-21T14:32:41Z

Bump, as I dug further into this. If I change the code above to simply decorate MyModel.call with @tf.function and do model.call.get_concrete_function(inputs).structured_input_signature, I get

(({'first_feature': TensorSpec(shape=(3,), dtype=tf.int32, name='inputs/first_feature'),
   'second_feature': TensorSpec(shape=(3,), dtype=tf.int32, name='inputs/second_feature')},
  None),
 {})

This information is used when restoring the model in infer_inputs_from_restored_function_call

tensorflow/tensorflow/python/keras/saving/saved_model/load.py

Line 1133 in 0733c41

def infer_inputs_from_restored_call_function(fn):

where the docstring notes:

Restored layer call function. It is assumed that `fn` has at least
        one concrete function and that the inputs are in the first argument.

When restoring the model above, infer_inputs_from_restored_function_call does have multiple concrete functions, but the input names differ across them (in tensorflow<2.5 perhaps there was some randomness in the ordering of these concrete functions).

Given that get_concrete_function joins the Python variable name (inputs in this case) with the name of the input (first_feature or second_feature), I wonder if there is a way to mark the correct concrete function from which to infer the inputs when saving the Keras model instead of just assuming the first one or otherwise remove the Python variable name.

dwyatte added the type:bug Bug label Jun 7, 2021

google-ml-butler bot assigned saikumarchalla Jun 7, 2021

saikumarchalla added TF 2.5 Issues related to TF 2.5 comp:apis Highlevel API related issues labels Jun 10, 2021

saikumarchalla added the regression issue To spot regression issues in latest version label Jun 10, 2021

saikumarchalla assigned ymodak and unassigned saikumarchalla Jun 10, 2021

ymodak added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jun 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TensorFlow renames inputs when restoring/resaving SavedModel #50130

TensorFlow renames inputs when restoring/resaving SavedModel #50130

TensorFlow renames inputs when restoring/resaving SavedModel #50130

TensorFlow renames inputs when restoring/resaving SavedModel #50130

Comments