[go: nahoru, domu]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dependency issues of libdali.so and backend_impl.so #1024

Open
ruiyuanlu opened this issue Jun 28, 2019 · 3 comments
Open

Dependency issues of libdali.so and backend_impl.so #1024

ruiyuanlu opened this issue Jun 28, 2019 · 3 comments
Labels
enhancement New feature or request external contribution welcome A good place to start contributing to the project
Projects
Users requests

Comments

@ruiyuanlu
Copy link
ruiyuanlu commented Jun 28, 2019 •
edited
Loading

Hi, I met a strange issue when after compiling DALI from scratch on Ubuntu 18.04, Anaconda python 3.6.

I created a {build_root} for compile process, and no error occurred during compiling. But as long as I renamed or deleted the "{build_root}/dali/python/nvidia/dali/libdali.so" file, I would get the following error when excuting import:

python -c "import nvidia.dali"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/room/anaconda3/lib/python3.6/site-packages/nvidia/dali/__init__.py", line 17, in <module>
    from . import ops
  File "/home/room/anaconda3/lib/python3.6/site-packages/nvidia/dali/ops.py", line 19, in <module>
    from nvidia.dali import backend as b
  File "/home/room/anaconda3/lib/python3.6/site-packages/nvidia/dali/backend.py", line 15, in <module>
    from nvidia.dali.backend_impl import *
ImportError: libdali.so: cannot open shared object file: No such file or directory

The python site-packages dir has the correct libdali.so file, but it seems that "backend_impl.cpython-36m-x86_64-linux-gnu.so" can only recognize "libdali.so" in the {build_root}, not site-packages dir in python. I guess there might be some dependency issues here ?

Any tips?
@ruiyuanlu ruiyuanlu changed the title build directory cannot be removed after "make install" Dependency issues of libdali.so and backend_impl.so Jun 28, 2019
@JanuszL JanuszL added the question Further information is requested label Jun 28, 2019
@JanuszL
Copy link
Contributor
JanuszL commented Jun 28, 2019

Hi,
Yes, the backend will have a path set to libdali.so located in your build directory. When we create a standalone wheel package we use bundle-wheel.sh, and it patches library RPATH. I think you should go through this script and check how you need to adjust the build process in your case (we do a couple of additional things there - like adding a hash value to library name to avoid collisions and adding dependent libraries to the actual wheel).

@ruiyuanlu
Copy link
Author
ruiyuanlu commented Jun 28, 2019 •
edited
Loading

@JanuszL Thanks for you reply. May be this can be added to the readme or installation guide? I think install from a standalone wheel might be a prefer way for many users, but it might make install process more complicated.

More specifically, 3 extra steps might be taken:

  1. The prefix might need to be set manually ("/usr", "/usr/local" etc.).

  2. patchelf need to be pre-installed which is required by bundle-wheel.sh.

  3. The wheel required bundle-wheel.sh should be built manually before calling bundle-wheel.sh.

@JanuszL
Copy link
Contributor
JanuszL commented Jun 28, 2019

@ruiyuanlu it is a good idea. I just hesitate to provide that many details in the documentation as the recommended way is to use docker image to build DALI.
Basically, bundle-wheel.sh assumes that libraries are at a certain location so they can be bundled in the wheel later, Ubuntu patchelf doesn't work properly - one form manylinux is fine and probably there are other caveeats that I have missed. Also, most of the time libraries on the system are dynamic while DALI relays on them to be static (if cmake find_pacakge functions provide dynamic one then whl will not bundle it and wheel won't work).
So I would say that the docker way, is tested and recommended by us. Still, I would love to see any external contribution improving the build process for other environments that docker we have prepared (a lot of different libraries and tools configurations and dependencies that are hard to support).

@JanuszL JanuszL added the external contribution welcome A good place to start contributing to the project label Jun 28, 2019
@JanuszL JanuszL added this to ToDo in Users requests Jul 10, 2019
@JanuszL JanuszL moved this from ToDo to Done in Users requests Jul 10, 2019
@JanuszL JanuszL moved this from Done to ToDo in Users requests Jul 10, 2019
@ccoulombe ccoulombe mentioned this issue Sep 23, 2019
Closed
@JanuszL JanuszL added enhancement New feature or request and removed question Further information is requested labels Jan 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request external contribution welcome A good place to start contributing to the project
Projects
Users requests
  
ToDo
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ruiyuanlu @JanuszL