[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TFLite Converter segfaults when trying to convert per-channel quantized transposed convolutions #53767

Open
lgeiger opened this issue Jan 14, 2022 · 5 comments
Assignees
Labels
stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.7 Issues related to TF 2.7.0 TFLiteConverter For issues related to TFLite converter type:bug Bug

Comments

@lgeiger
Copy link
Contributor
lgeiger commented Jan 14, 2022

When converting transposed convolutions using per-channel weight quantization the converter segfaults and crashes the Python process. Per-channel quantization is supported by TFLite Transposed convolutions:

TF_LITE_ENSURE_EQ(context, weights->quantization.type,
kTfLiteAffineQuantization);
const auto* affine_quantization =
reinterpret_cast<TfLiteAffineQuantization*>(
weights->quantization.params);
const int channels_out = weights->dims->data[0];
TF_LITE_ENSURE(context, affine_quantization);
TF_LITE_ENSURE(context, affine_quantization->scale);
TF_LITE_ENSURE(context, (affine_quantization->scale->size == 1 ||
affine_quantization->scale->size == channels_out));

so the converter shouldn't segfault when trying to convert such a model.

It looks like this issue has been introduced in TensorFlow 2.6 since the same model code produced a valid TFLite file in TensorFlow 2.5. This issue might also be related to #53766, but in any case the converter should never segfault.

1. System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS / Ubuntu
  • TensorFlow installation (pip package or built from source): pip package
  • TensorFlow library (version, if pip package or github SHA, if built from source): 2.6, 2.7, 2.8rc0 and 2.9.0-dev20220114

2. Code

A minimal reproduction of the issue and a workaround is available in this notebook.

import tensorflow as tf


class QuantConv2DTransposed(tf.keras.layers.Layer):
    def build(self, input_shape):
        self.kernel = self.add_weight("kernel", [3, 3, input_shape[-1], 24])

    def call(self, inputs):
        filters = tf.quantization.fake_quant_with_min_max_vars_per_channel(
            self.kernel, -3.0 * tf.ones([24]), 3.0 * tf.ones([24]), narrow_range=True
        )
        filters = tf.transpose(filters, (0, 1, 3, 2))
        return tf.nn.conv2d_transpose(inputs, filters, [*inputs.shape[:-1], 24], 1)


inp = tf.keras.Input(shape=(6, 8, 48), batch_size=1)
x = tf.quantization.fake_quant_with_min_max_vars(inp, -3.0, 3.0, narrow_range=True)
x = QuantConv2DTransposed()(x)
x = tf.quantization.fake_quant_with_min_max_vars(x, -3.0, 3.0, narrow_range=True)

model = tf.keras.Model(inp, x)

model.save("/tmp/testing")
converter = tf.lite.TFLiteConverter.from_saved_model("/tmp/testing")
converter.optimizations = [tf.lite.Optimize.DEFAULT]

# terminated by signal SIGSEGV (Address boundary error)
tflite_model = converter.convert()
@sushreebarsa
Copy link
Contributor

@Saduf2019 Was able to replicate the issue on colab using TF v2.6.0, 2.7.0 and tf-nightly(2.9.0-dev20220114),please find the attached gists.Thank you!

@lgeiger
Copy link
Contributor Author
lgeiger commented Jan 17, 2022

Was able to replicate the issue on colab using TF v2.6.0, 2.7.0 and tf-nightly(2.9.0-dev20220114),please find the attached gists

@sushreebarsa Thanks for confirming. Just for reference, your reproduction on TF 2.6.0 actually now fails due to an unrelated Keras version conflict. Changing the dependency from v2.6.0 to v2.6.2 will fix this and allow you to correctly reproduce the segfault mentioned in this issue.

@jvishnuvardhan jvishnuvardhan added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jan 20, 2022
@lgeiger
Copy link
Contributor Author
lgeiger commented Mar 18, 2022

This is still an issue in 2.9.0-dev20220318. Are there any updates on this?
Being able to trigger a converter segfault seems to be quite problematic.

@lgeiger
Copy link
Contributor Author
lgeiger commented May 11, 2022

@JunyoungLim I retested the above example 2.10.0-dev20220427 and the converter still segfaults.

@lgeiger
Copy link
Contributor Author
lgeiger commented Sep 13, 2022

I retested the above example with 2.10.0 and the segfault seems to be fixed now, however conversion still fails with:

'tfl.transpose' op has mismatched quantized axes of input and output

See here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.7 Issues related to TF 2.7.0 TFLiteConverter For issues related to TFLite converter type:bug Bug
Projects
None yet
Development

No branches or pull requests

5 participants