-
Notifications
You must be signed in to change notification settings - Fork 26.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Better check for packages availability #23163
Conversation
The documentation is not available anymore as the PR was closed or merged. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the refactor, this is super nice!
There are a couple of packages that do not properly implement metadata (I kno for sure opencv
does not since we are adding it in another PR), could you quickly check that all the packages for which you did this PR do implement the metadata? if they don't we need to rely on the old way, which is fine as it should be an exceptional case.
I have checked the packages and found issues only on sklearn (the import is sklearn and package scikit-learn) and decord. Could you please double check: smdistributed, tensorflow_text, torchdistx? |
I have validated that Checking internally for |
Not hearing anything back on Can you just fix the conflict? |
don't think failures are due to this PR. Also main is failing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
* Better check for packages availability * amend _optimumneuron_available * amend torch_version * amend PIL detection and lint * lint * amend _faiss_available * remove overloaded signatures of _is_package_available * fix sklearn and decord detection * remove unused checks * revert
* fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Better check for packages availability * amend _optimumneuron_available * amend torch_version * amend PIL detection and lint * lint * amend _faiss_available * remove overloaded signatures of _is_package_available * fix sklearn and decord detection * remove unused checks * revert
* fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
* Better check for packages availability * amend _optimumneuron_available * amend torch_version * amend PIL detection and lint * lint * amend _faiss_available * remove overloaded signatures of _is_package_available * fix sklearn and decord detection * remove unused checks * revert
* fix * fix --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Following up huggingface/accelerate#1356
Refactor checks to avoid boilerplate and to ensure we are not picking up a folder that happens to be called as the package.
I assume all _*_available are for caching purposes. But is that really needed?