[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AutoGraph cannot handle python 3.10's structural pattern matching #57166

Open
burnpanck opened this issue Aug 16, 2022 · 6 comments
Open

AutoGraph cannot handle python 3.10's structural pattern matching #57166

burnpanck opened this issue Aug 16, 2022 · 6 comments
Labels
comp:autograph Autograph related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.9 Issues found in the TF 2.9 release (or RCs) type:bug Bug

Comments

@burnpanck
Copy link
burnpanck commented Aug 16, 2022 •
edited
Loading
Click to expand!

Issue Type

Bug

Source

binary

Tensorflow Version

2.9.2

Custom Code

No

OS Platform and Distribution

macOS 12.5

Mobile device

none

Python version

3.10

Bazel version

No response

GCC/Compiler version

No response

CUDA/cuDNN version

none

GPU model and memory

Apple's METAL (though unlikely to be relevant here)

Current Behaviour?

Trying to AutoGraph a function containing a PEP 634/635/636 structural pattern matching statement from python 3.10 (i.e. match/case) results in the WARNING:tensorflow:AutoGraph could not transform <function test at 0x2c4c99120> and will run it as-is.

Standalone code to reproduce the issue

import tensorflow as tf

@tf.function
def test(x, selector):
    match selector:
        case "square":
            return x**2
        case "double":
            return x*2.
        case _:
            raise ValueError
            
test(tf.linspace(-1,1,10),"double")

Relevant log output

INFO:tensorflow:Error transforming entity <function test at 0x2c4c99120>
Traceback (most recent call last):
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 427, in converted_call
    converted_f = _convert_actual(target_entity, program_ctx)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 269, in _convert_actual
    transformed, module, source_map = _TRANSPILER.transform(entity, program_ctx)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 282, in transform
    return self.transform_function(obj, user_context)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 466, in transform_function
    nodes, ctx = super(PyToPy, self).transform_function(fn, user_context)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 359, in transform_function
    result = self.transform_ast(node, context)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 237, in transform_ast
    node = self.initial_analysis(node, ctx)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/impl/api.py", line 223, in initial_analysis
    graphs = cfg.build(node)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/cfg.py", line 970, in build
    visitor.visit(node)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/cfg.py", line 766, in visit_FunctionDef
    self._process_function_def(node, is_lambda=False)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tensorflow/python/autograph/pyct/cfg.py", line 757, in _process_function_def
    self.visit(stmt)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 414, in generic_visit
    for field, value in iter_fields(node):
  File "/Users/yves/.pyenv/versions/3.10.4/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 252, in iter_fields
    for field in node._fields:
AttributeError: 'NoneType' object has no attribute '_fields'
@google-ml-butler google-ml-butler bot added the type:bug Bug label Aug 16, 2022
@sushreebarsa sushreebarsa added comp:autograph Autograph related issues TF 2.9 Issues found in the TF 2.9 release (or RCs) labels Aug 16, 2022
@gadagashwini
Copy link
Contributor

Hi @burnpanck, When I executed given code, it worked as expected. Tested with Python3.10 and Tensorflow 2.9.1

(tfpy310) gadag@ashwini-gpu:~$ python3.10
Python 3.10.4 (main, Mar 31 2022, 08:41:55) [GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
2022-08-29 11:04:48.007921: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-08-29 11:04:48.008070: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>> @tf.function
... def test(x, selector):
...    match selector:
...      case "square":
...         return x**2
...      case "double":
...         return x*2.
...      case _:
...         raise ValueError
... 
>>> test(tf.linspace(-1,1,10),"double")
2022-08-29 11:07:05.071427: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-08-29 11:07:05.072416: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:975] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-08-29 11:07:05.073324: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.073473: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.073592: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.073703: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.073821: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcurand.so.10'; dlerror: libcurand.so.10: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.073939: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.074064: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.074181: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory
2022-08-29 11:07:05.074241: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1850] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2022-08-29 11:07:05.074775: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:tensorflow:AutoGraph could not transform <function test at 0x7f703b8085e0> and will run it as-is.
Cause: Unable to locate the source code of <function test at 0x7f703b8085e0>. Note that functions defined in certain environments, like the interactive Python shell, do not expose their source code. If that is the case, you should define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.experimental.do_not_convert. Original error: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function test at 0x7f703b8085e0> and will run it as-is.
Cause: Unable to locate the source code of <function test at 0x7f703b8085e0>. Note that functions defined in certain environments, like the interactive Python shell, do not expose their source code. If that is the case, you should define them in a .py source file. If you are certain the code is graph-compatible, wrap the call using @tf.autograph.experimental.do_not_convert. Original error: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
<tf.Tensor: shape=(10,), dtype=float64, numpy=
array([-2.        , -1.55555556, -1.11111111, -0.66666667, -0.22222222,
        0.22222222,  0.66666667,  1.11111111,  1.55555556,  2.        ])>

@gadagashwini gadagashwini added the stat:awaiting response Status - Awaiting response from author label Aug 29, 2022
@burnpanck
Copy link
Author

Hey @gadagashwini, thanks for looking into this. Please note that in my case, the code does execute as well. The traceback I posted under "relevant log output" is really a logged message (right after the warning that I mention), not from an exception that reaches my code.

Your code logs the same warning - although the reason it gives is different: Original error: could not get source code. It appears that you did run the example code from an interactive base interpreter, which apparently prevents AutoGraph from grabbing the source code. Please re-run the code from within a .py file or a jupyter notebook, as that seems to provide the source code via a temp-file (that is what I did).

Either way, do not expect the code to throw an exception. The issue that I'm reporting here is the warning message both our runs are logging, and the implications: The example code is not being handled by AutoGraph, even though it still can execute.

@google-ml-butler google-ml-butler bot removed the stat:awaiting response Status - Awaiting response from author label Aug 30, 2022
@gowthamkpr gowthamkpr added the stat:awaiting tensorflower Status - Awaiting response from tensorflower label Sep 8, 2022
@gowthamkpr gowthamkpr removed their assignment Sep 8, 2022
@mdanatg mdanatg removed their assignment Feb 7, 2023
@burnpanck
Copy link
Author

Any updates here? Still present in TF 2.15.0.

Structural pattern matching is part of standard Python and by now several years and python releases old. If AutoGraph cannot handle structural pattern matching, it basically means it is unable to read the python syntax. This is a pity, because pattern matching is very useful, even if it does not involve any tensors. That is, supporting the syntax is important even if the syntax does not unlock any tensor operations. As the feature becomes more well-known, this will only grow in importance.

@burnpanck burnpanck mentioned this issue Jan 7, 2024
Open
@Datamagician10
Copy link

@burnpanck I think TensorFlow's AutoGraph may not yet have implemented support for converting this syntax into graph operations.Did u try another approach ?

@burnpanck
Copy link
Author

@Datamagician10 You think right. AutoGraph does not support structural pattern matching, despite this being a standard python feature since almost three years now (python 3.10 was released in 2021) - that is what I am reporting here.

@mdanatg @gowthamkpr @sushreebarsa @gadagashwini Each of you were assigned to this at some point, but it seems the issue has now fallen between the cracks. Implementing this shouldn't be too difficult. Judging from previous release schedules, the next TF release may well drop support for python 3.9, at which point basically doesn't support any version of python correctly at all anymore!

@burnpanck
Copy link
Author
burnpanck commented Apr 24, 2024 •
edited
Loading

Implementing this shouldn't be too difficult.

I do see that AutoGraph in fact offers to convert some python control statements into TF graphs. To support that use-case, AutoGraph needs to "read and understand" the python AST at some point before running the TF graph (i.e. during the first phase that includes tracing). For example, the documentation states that if statements are transformed when the condition is a tensor. The same could be done for match statements, e.g. by transforming it into an equivalent set of if statements. Given the breath of possible matching patterns, this may still be a non-negligible effort. Instead, I suggest not to support any transformations for match at all. IMHO, the majority of use-cases for match are only relevant during the tracing phase. Therefore, AutoGraph can be fairly oblivious of the existence of match at all, just letting it execute normally during tracing. The only reason it causes issues in the first place is likely because AutoGraph needs to inspect the AST to support the transformation of if, while and for, causing it to choke on an unknown AST node. So I'm confident that 99 % of the use-cases for match could be solved simply by teaching AutoGraph to recognise and (mostly) ignore match statements. As it stands today however, all code containing match is being rejected entirely by AutoGraph, even if the match statement has zero effect on the graph execution phase (apparently even when it appears in the surrounding scope of a definition - see #60458).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
comp:autograph Autograph related issues stat:awaiting tensorflower Status - Awaiting response from tensorflower TF 2.9 Issues found in the TF 2.9 release (or RCs) type:bug Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@burnpanck @mdanatg @gowthamkpr @Datamagician10 @sushreebarsa @gadagashwini