[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build configuration is missing definitions #12

Closed
superbobry opened this issue Aug 16, 2018 · 11 comments
Closed

Build configuration is missing definitions #12

superbobry opened this issue Aug 16, 2018 · 11 comments

Comments

@superbobry
Copy link
Member

System information

  • Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow Model Analysis)    : N/A
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): N/A
  • TensorFlow Model Analysis installed from (source or binary): N/A
  • TensorFlow Model Analysis version (use command below): N/A
  • Python version: N/A
  • Jupyter Notebook version: N/A
  • Exact command to reproduce: N/A

Describe the problem

The HEAD version of TFMA seems to be missing some definitions for third-party dependencies. The easy-to-fix one is ProtoBuf:

ERROR: error loading package 'tensorflow_model_analysis/proto': Extension file not found. Unable to load package for '@protobuf_bzl//:protobuf.bzl': The repository could not be resolved

which can be fixed by changing the load call slightly

diff --git a/tensorflow_model_analysis/proto/BUILD b/tensorflow_model_analysis/proto/BUILD
index af3386c..af787da 100644
--- a/tensorflow_model_analysis/proto/BUILD
+++ b/tensorflow_model_analysis/proto/BUILD
@@ -2,7 +2,7 @@ licenses(["notice"])  # Apache 2.0

 package(default_visibility = ["//visibility:public"])

-load("@protobuf_bzl//:protobuf.bzl", "py_proto_library")
+load("@com_google_protobuf//:protobuf.bzl", "py_proto_library")

The other one is a bit more difficult since the BUILD file for third_party/py/typing is truly missing from the repo

ERROR: [...]/model-analysis/tensorflow_model_analysis/slicer/BUILD:3:1: no such package 'third_party/py/typing': BUILD file not found on package path and referenced by '//tensorflow_model_analysis/slicer:slicer'

Finally, some of the TFMA BUILD files reference third_party/py/numpy which is missing as well.
@superbobry superbobry mentioned this issue Aug 17, 2018
Closed
@xinzha623
Copy link
Contributor

Hi Sergei,

What command did you run? If you would like to generate whl file from source codes, you could simply run python bdist setup.py.

@superbobry
Copy link
Member Author

I wanted to run tests with Bazel. Would you recommend running them via e.g. pytest instead?

@chuanyu
Copy link
Contributor
chuanyu commented Aug 21, 2018

It turns out that our BUILD files need to be fixed quite a bit to get them to work. We'll work on fixing the BUILD files and pushing a new version with the fixed BUILD files soon. In the meantime, you can try running the tests directly with Python. Sorry for the inconvenience, and thanks for your patience!

@chuanyu
Copy link
Contributor
chuanyu commented Aug 21, 2018

Okay, after some discussion, we'll be removing the BUILD files for the Python targets for now, since they don't really add much. Could you try running the tests with Python directly, and let us know if you run into any issues?

@superbobry
Copy link
Member Author

I have tried running the tests locally:

============== 23 failed, 85 passed, 1 warnings in 247.46 seconds ==============

most failures involve the repr of dilll internals and are tricky to diagnose. Full output of pytest: failures.txt

@chuanyu
Copy link
Contributor
chuanyu commented Aug 22, 2018

My guess is that they have something to do with Beam. Which version of Beam do you have installed?

@superbobry
Copy link
Member Author

2.6.0 which matches the apache-beam[gcp]>=2.5,<3 constraint in setup.py.

I have the same number of failures with 2.5.0 and 2.4.0, and the error messages look similar. Could it be the dill version then?

@superbobry
Copy link
Member Author
superbobry commented Aug 23, 2018 •
edited
Loading

The failures are caused by pytest. I think it instruments unittest classes (or possibly instances) in a way which makes them non-pickleable. Therefore, the error: "cannot pickle generators".

Interestingly, the error goes away if pytest is executed with the--pdb flag :)

Would you accept a PR adding Travis CI support?

@chuanyu
Copy link
Contributor
chuanyu commented Aug 23, 2018

I see, thanks for looking into this!

It would be nice to have CI support! Let me check with some folks to see whether there's a preferred solution or whether Travis CI would be fine.

@chuanyu
Copy link
Contributor
chuanyu commented Aug 31, 2018

Okay, so I checked with some other folks, and it looks like the plan is to set up a CI solution similar to what we have with the TensorFlow repository. Thanks for offering to add Travis CI support though!

@superbobry
Copy link
Member Author

Thanks for checking, closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@superbobry @chuanyu @xinzha623