[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ComplexOp to TFLite #61290

Open
drubinstein opened this issue Jul 16, 2023 · 6 comments
Open

Add ComplexOp to TFLite #61290

drubinstein opened this issue Jul 16, 2023 · 6 comments
Assignees
@zichuan-wei @pkgoogle
Labels
awaiting review Pull request awaiting review comp:lite TF Lite related issues comp:lite-kernels TensorFlow Lite kernel issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.13 For issues related to Tensorflow 2.13 type:feature Feature requests

Comments

@drubinstein
Copy link
Contributor

System information

  • MacOS 13.4.1
  • binary (via pip)
  • 2.13.0

Provide the text output from tflite_convert

loc(callsite(callsite(fused["Complex:", callsite("sequential/lambda/Complex@__inference__wrapped_model_30"("/opt/homebrew/lib/python3.11/site-packages/tensorflow/python/util/dispatch.py":1176:0) at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/python/util/traceback_utils.py":150:0 at callsite("/Users/drubinstein/test/complex_test/complex.py":4:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/layers/core/lambda_layer.py":212:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py":96:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/base_layer.py":1150:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py":65:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/sequential.py":420:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py":96:0 at "/opt/homebrew/lib/python3.11/site-packages/keras/src/engine/base_layer.py":1150:0)))))))))] at fused["PartitionedCall:", callsite("PartitionedCall@__inference_signature_wrapper_126"("/opt/homebrew/lib/python3.11/site-packages/tensorflow/python/saved_model/save.py":1313:0) at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/python/saved_model/save.py":1280:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py":1427:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/convert_phase.py":205:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py":1504:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py":1526:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py":1042:0 at callsite("/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py":1065:0 at "/Users/drubinstein/test/complex_test/complex.py":11:0))))))))]) at fused["PartitionedCall:", "PartitionedCall"])): error: 'tf.Complex' op is neither a custom op nor a flex op
error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Complex
Details:
	tf.Complex(tensor<3xf32>, tensor<3xf32>) -> (tensor<3xcomplex<f32>>) : {device = ""}

Traceback (most recent call last):
  File "/Users/drubinstein/test/complex_test/complex.py", line 11, in <module>
    tflite_model = converter.convert()
                   ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py", line 1065, in wrapper
    return self._convert_and_export_metrics(convert_func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py", line 1042, in _convert_and_export_metrics
    result = convert_func(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py", line 1526, in convert
    saved_model_convert_result = self._convert_as_saved_model()
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py", line 1507, in _convert_as_saved_model
    return super(TFLiteKerasModelConverterV2, self).convert(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/lite.py", line 1296, in convert
    result = _convert_graphdef(
             ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/convert_phase.py", line 212, in wrapper
    raise converter_error from None  # Re-throws the exception.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/convert_phase.py", line 205, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/convert.py", line 918, in convert_graphdef
    data = convert(
           ^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/tensorflow/lite/python/convert.py", line 367, in convert
    raise converter_error
tensorflow.lite.python.convert_phase.ConverterError: /opt/homebrew/lib/python3.11/site-packages/tensorflow/python/util/dispatch.py:1176:0: error: 'tf.Complex' op is neither a custom op nor a flex op
<unknown>:0: note: loc(fused["PartitionedCall:", "PartitionedCall"]): called from
/opt/homebrew/lib/python3.11/site-packages/tensorflow/python/util/dispatch.py:1176:0: note: Error code: ERROR_NEEDS_FLEX_OPS
<unknown>:0: error: failed while converting: 'main':
Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select
TF Select ops: Complex
Details:
	tf.Complex(tensor<3xf32>, tensor<3xf32>) -> (tensor<3xcomplex<f32>>) : {device = ""}

Standalone code to reproduce the issue

import tensorflow as tf

model = tf.keras.models.Sequential(
    [tf.keras.layers.Lambda(lambda x: tf.dtypes.complex(x[0], x[1]))]
)

print(model([[1.0, 2.0, 3.0], [2.0, 3.0, 4.0]]))

# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

# Save the model.
with open('model.tflite', 'wb') as f:
    f.write(tflite_model)

Any other info / logs

I have begun implementing ComplexOp in TFLite, but I'm not sure if what I'm doing is correct as I can't find a guide on how to add operators. I have so far done everything I think I needed to to add Complex as a builtin in the lite directory, but am a little stuck on what to do with MLIR. I believe I need to:

  • add ComplexOp as a tfl op
  • legalize the ComplexOp pattern so TF and TFL can both lower to TOSA IR

Is this correct? Is there anything I'm missing? How do I test the converter?

Thanks!
@drubinstein drubinstein added the comp:lite TF Lite related issues label Jul 16, 2023
@sushreebarsa sushreebarsa added TF 2.13 For issues related to Tensorflow 2.13 type:feature Feature requests labels Jul 18, 2023
@sushreebarsa
Copy link
Contributor

@pjpratik I was able to replicate the issue, please find the gist here. Thank you!

@pjpratik
Copy link
Contributor

Hi @drubinstein

You might need to add in

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/ir/tfl_ops.cc
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/ir/tfl_ops.td
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/tests/legalize-tf.mlir

Can you please refer to sample commit ace4433 which adds bitcast operator into builtin ops.

Thanks.

@pjpratik pjpratik added the stat:awaiting response Status - Awaiting response from author label Jul 19, 2023
@drubinstein drubinstein mentioned this issue Jul 24, 2023
Open
@github-actions
Copy link

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

@github-actions github-actions bot added the stale This label marks the issue/pr stale - to be closed automatically if no activity label Jul 27, 2023
@drubinstein
Copy link
Contributor Author
drubinstein commented Jul 27, 2023 •
edited
Loading

To keep the issue open, I'm working on PR #61359.

@google-ml-butler google-ml-butler bot removed stale This label marks the issue/pr stale - to be closed automatically if no activity stat:awaiting response Status - Awaiting response from author labels Jul 27, 2023
@pjpratik pjpratik added the awaiting review Pull request awaiting review label Jul 27, 2023
@pkgoogle pkgoogle added the comp:lite-kernels TensorFlow Lite kernel issues label Jul 27, 2023
@hamlatzis
Copy link

in the given link there is no support for models with kTfLiteFloat16 eg `real_input->type == kTfLiteFloat16'

maybe using Eigen::half?

@pkgoogle pkgoogle assigned pkgoogle and zichuan-wei and unassigned pjpratik Jan 30, 2024
@pkgoogle pkgoogle added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Jan 31, 2024
@pkgoogle
Copy link

Hi @zichuan-wei, it seems like the PR needs a review so I'm assigning to you for now, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@zichuan-wei zichuan-wei

@pkgoogle pkgoogle

Labels
awaiting review Pull request awaiting review comp:lite TF Lite related issues comp:lite-kernels TensorFlow Lite kernel issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.13 For issues related to Tensorflow 2.13 type:feature Feature requests
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@drubinstein @hamlatzis @zichuan-wei @tilakrayal @sushreebarsa @pjpratik @pkgoogle