Fix configure.py to more directly specify the find_cuda_config.py instead of using glob()
#6427
Conversation
…tead of using `glob()` We know the exact location and don't need to glob() here. The issue with glob is that it can be slow, in particular when re-running the configure script after building with Bazel: the glob will process tens of GB of build directories!
|
@angerson, tensorflow/tensorflow@b1f946a added the uncodntional glob. Before that commit, there was specific logic to try to find |
|
This is the PR that suggested a glob of some kind would be useful: tensorflow/tensorflow#46075 I've never seen it brought up again, though. |
|
Ok great, I think I'm going to merge this then. Thanks!! |
|
Even if TensorFlow or XLA is included in another project, the fact this PR uses |
…fig.py instead of using `glob()` Imported from GitHub PR openxla/xla#6427 We know the exact location and don't need to glob() here. The issue with glob is that it can be slow, in particular when re-running the configure script after building with Bazel: the glob will process tens of GB of build directories! Copybara import of the project: -- d0ccab91c465871c6c87f9bcf1746cbd5da47967 by Mehdi Amini <mamini@nvidia.com>: Fix configure.py to more directly specify the find_cuda_config.py instead of using `glob()` We know the exact location and don't need to glob() here. The issue with glob is that it can be slow, in particular when re-running the configure script after building with Bazel: the glob will process tens of GB of build directories! Merging this change closes #6427 PiperOrigin-RevId: 575020900
|
Shouldn't the path be: 'third_party/tsl/third_party/gpus/find_cuda_config.py' instead of: 'third_party/gpus/find_cuda_config.py'? Currently it is throwing "FileNotFoundError: Can't find 'find_cuda_config.py' script inside working directory, expected in /xla/third_party/gpus/find_cuda_config.py" |
You're right, seems like edd97f346c08c69616719d72606ec6e97d15aced broke it yesterday by removing the path used here. I'll fix it. |
|
The double nesting of |
|
#6610 for the fix. |
|
Intentional, but controversial. We have workspace macros that should automatically load TSL's/XLA's dependencies. Just like TF has workspace{0,1,2,3}.bzl, XLA/TSL also have workspace macros that do the same thing. e.g. XLA calls TSL here: https://github.com/openxla/xla/blob/main/workspace2.bzl#L90 |
We know the exact location and don't need to glob() here. The issue with glob is that it can be slow, in particular when re-running the configure script after building with Bazel: the glob will process tens of GB of build directories!