Coverage not working for TensorFlow Model call function #856
Comments
|
I've never used TensorFlow before. I'm going to guess that the execution of Python code within call(...) doesn't trigger the Python trace function, but I don't know. Can you provide very very explicit reproduction steps? Include the version of Python, the commands to run to install everything, and the commands to run the program. Thanks. |
|
Python 3.7.4 on macOS 10.14.6. The steps so far are: git clone https://github.com/Data-Science-Projects/demo-routenet.git Since I am looking the results in PyCharm, I am not quite sure how to recreate what I see in PyCharm at the CLI. What seems to be missing, in PyCharm, is any indication that the call function in the src/routenet/model/routenet_model.py code is covered. It is clear that the code is called, as one can see the output from the print statement. |
|
Thanks, this lets me reproduce the problem. I can see that line 65 in routenet_model.py is reported as uncovered. I'll dig into it. |
|
Thanks, apologies for the delay in following up before. This is a weekend project ...
… On 26 Oct 2019, at 13:18, Ned Batchelder ***@***.***> wrote:
Thanks, this lets me reproduce the problem. I can see that line 65 in routenet_model.py is reported as uncovered. I'll dig into it.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub <#856?email_source=notifications&email_token=ADGFXFVWAHUQTMPGANGH4ULQQQRPPA5CNFSM4I7ZSAJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECKFXVI#issuecomment-546593749>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGFXFSA3SMVWUJYOWW7CJLQQQRPPANCNFSM4I7ZSAJA>.
|
|
@NathanDotTo this was an interesting one! It turns out that TensorFlow is making a transformed copy of your code, and then running it. So your original file isn't actually executed. In a way, coverage.py was right! I've opened tensorflow/tensorflow#33759 with TensorFlow to see what we can do about it. |
|
Thank you. This may also change with TF2.0, which is changing from a graph, run in a session, programming model, to something more like traditional Python with “eager” processing. I am still coming to grips with TF2.0.
… On 27 Oct 2019, at 13:28, Ned Batchelder ***@***.***> wrote:
@NathanDotTo <https://github.com/NathanDotTo> this was an interesting one! It turns out that TensorFlow is making a transformed copy of your code, and then running it. So your original file isn't actually executed. In a way, coverage.py was right! I've opened tensorflow/tensorflow#33759 <tensorflow/tensorflow#33759> with TensorFlow to see what we can do about it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#856?email_source=notifications&email_token=ADGFXFUSA56JATJSNDS6JDTQQWCOBA5CNFSM4I7ZSAJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECK5GIQ#issuecomment-546689826>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGFXFWGV5ATOW6KVSFNUSLQQWCOBANCNFSM4I7ZSAJA>.
|
|
I have a similar question, when I debug tensor2tensor and set a breakpoint in t2t_model.py at line 316, the debugger doesn't break as if there is no breakpoint. Then I run the code step by step, in the end, I also found a temporary Python file in /tmp/.
|
|
@Zminghua debugging is not a coverage.py concern, but sharing my new-found expertise, it should work if you put a "pdb.set_trace()" line in your t2t_model.py file. That line will then be copied to the temporary file, and the debugger will break. You will be in a slightly strange world, since your file has been changed, but you can at least be in the debugger. |
|
@nedbat Thanks for your reminding. I have learned that it is AutoGraph conversion in Tensorflow keras module. After I added a "pdb.set_trace()" line, I was really in a slightly strange world. However, I have modified tensorflow source code to prevent the conversion. Thank you very very very much. |
* Factor out the common bits of PyTorch & Tensorflow batch classes to BaseDenseBatch & BaseSparseBatch * Inline _ensure_attr * Test both the public TensorflowTileDBDataset and the private _generator to work around nedbat/coveragepy#856 * Factor out the PyTorch & Tensorflow generator logic to _batch_utils.tensor_generator * Change Tuple to Sequence or Iterator in signatures * Rename idx parameter to perm_idxs * Project-wide rename of *attribute_names to *attrs * Optimization/fix: ensure batch_offsets doesn't extend beyond the size of the current buffer
This problem was first reported at https://youtrack.jetbrains.com/oauth?state=%2Fissue%2FPY-38366
1 - I am running a unit test that is invoking the call(...) function of a TensorFlow model. I can see the trace message, that I have placed in the function, printed so I know the function is being called.
2 - The coverage analysis shows that code in red, as though it was not invoked.
3 - It should show green as it was invoked.
See test_model.py in https://github.com/Data-Science-Projects/demo-routenet.
See also the requirements.txt in that project.
The text was updated successfully, but these errors were encountered: