Wrong numpy range in setup.py #456

TobyRoseman · 2019-09-20T00:58:52Z

🐞Describe the bug

The setup.py for coremltools specifies the numpy dependency range as 'numpy >= 1.14.5'. However using any version before then 1.16.0 causes a segmentation fault. Any version >= to 1.16.0 works.

It would be nice to support versions before 1.16.0. That version is less than a year old. However at the very least the specified dependency ranges should be accurate.

Trace

No trace displayed. Only output is Segmentation fault: 11

To Reproduce

In a fresh/new Conda environment, run:
pip install Pillow coremltools==3.0b6 numpy==1.15.4

Download the MNIST Model.

Execute the following code:

import coremltools
from PIL import Image
import numpy as np

data = np.empty((28,28), dtype=np.uint8)
input_image = Image.fromarray(data)

model = coremltools.models.MLModel('./Downloads/MNISTClassifier.mlmodel')
print(model.predict({'image': input_image}))

This give a segmentation fault.

If you then run pip install numpy==1.16.0, then the above code works.

System environment (please complete the following information):

coremltools version (e.g., 3.0b5): 3.0b6
OS (e.g., MacOS, Linux): MacOS
macOS version (if applicable): 10.14.6
XCode version (if applicable): Not using
How you install python (anaconda, virtualenv, system): anaconda
python version (e.g. 3.7): 3.6

The text was updated successfully, but these errors were encountered:

Shekharrajak · 2019-09-20T15:31:20Z

Can I work on this issue?

1duo · 2019-09-20T16:27:36Z

Can I work on this issue?

Hi @Shekharrajak, you're more than welcome to contribute here. Thanks!

Shekharrajak · 2019-09-20T16:37:19Z

Thanks @1duo !

I just need a clarification :

Do we want to make it compatible for version < 1.16.0 or should we constraint version >= 1.16.0 in setup.py ?

1duo · 2019-09-20T17:05:18Z

Thanks @1duo !

I just need a clarification :

Do we want to make it compatible for version < 1.16.0 or should we constraint version >= 1.16.0 in setup.py ?

Hi @TobyRoseman, it'd be great if you can comment on / clarify the expected version we want to cover here. Thanks!

TobyRoseman · 2019-09-20T19:24:53Z

The most important thing is that the version range listed in setup.py is accurate.

That being said, coremltools should really support numpy versions older than 9 months ago. I believe numpy is very good about backwards compatibility. It would be great if we could actually support numpy >= 1.14.5 or even earlier.

coremltools==2.1.0 works just fine with numpy==1.14.5. What changed? What version of numpy was installed in the environments that built the most recent artifacts for PyPI? The fix could be as simple as just installing an old version of numpy in those environments. I think that is worth a try.

Shekharrajak · 2019-09-22T04:12:28Z

I followed the steps given :

mkdir build
cd build
cmake ../

make

but I got the error :

 error: unknown type name 'MLSequence'
            py::object convertSequenceValueToPython(MLSequence *seq) API_AVAILABLE(macos(10.14));
                                                    ^
/Users/coremltools/coremlpython/CoreMLPython.mm:58:13: error: unknown type name 'MLModelConfiguration'
            MLModelConfiguration *configuration = [MLModelConfiguration new];
            ^
/User/coremltools/coremlpython/CoreMLPython.mm:58:52: error: use of undeclared identifier 'MLModelConfiguration'
            MLModelConfiguration *configuration = [MLModelConfiguration new];
                                                   ^
/Users/coremltools/coremlpython/CoreMLPython.mm:60:46: error: use of undeclared identifier 'MLComputeUnitsCPUOnly'
                configuration.computeUnits = MLComputeUnitsCPUOnly;
                                             ^
8 warnings and 4 errors generated.
make[2]: *** [CMakeFiles/coremlpython.dir/coremlpython/CoreMLPython.mm.o] Error 1
make[1]: *** [CMakeFiles/coremlpython.dir/all] Error 2
make: *** [all] Error 2

Can you please help me to get the setup done ?

1duo · 2019-09-26T04:22:29Z

I followed the steps given :

mkdir build
cd build
cmake ../

make

but I got the error :

 error: unknown type name 'MLSequence'
            py::object convertSequenceValueToPython(MLSequence *seq) API_AVAILABLE(macos(10.14));
                                                    ^
/Users/coremltools/coremlpython/CoreMLPython.mm:58:13: error: unknown type name 'MLModelConfiguration'
            MLModelConfiguration *configuration = [MLModelConfiguration new];
            ^
/User/coremltools/coremlpython/CoreMLPython.mm:58:52: error: use of undeclared identifier 'MLModelConfiguration'
            MLModelConfiguration *configuration = [MLModelConfiguration new];
                                                   ^
/Users/coremltools/coremlpython/CoreMLPython.mm:60:46: error: use of undeclared identifier 'MLComputeUnitsCPUOnly'
                configuration.computeUnits = MLComputeUnitsCPUOnly;
                                             ^
8 warnings and 4 errors generated.
make[2]: *** [CMakeFiles/coremlpython.dir/coremlpython/CoreMLPython.mm.o] Error 1
make[1]: *** [CMakeFiles/coremlpython.dir/all] Error 2
make: *** [all] Error 2

Can you please help me to get the setup done ?

Can you share your system environment info?

Shekharrajak · 2019-09-26T15:02:59Z

Here is the system I am using

coremltools version (e.g., 3.0b5): Didn't install prevously. trying to instal from source
OS (e.g., MacOS, Linux): MacOS
macOS version (if applicable): 10.13
XCode version (if applicable): Not using
How you install python (anaconda, virtualenv, system): conda
python version (e.g. 3.7): 3.7.3

Shekharrajak · 2019-09-26T15:52:26Z

I installed the codemltool using pip not through source but I came across different error messege :

[ 92%] Building CXX object CMakeFiles/caffeconverter.dir/deps/caffe/src/caffe/proto/caffe.pb.cc.o
[ 92%] Building CXX object CMakeFiles/caffeconverter.dir/deps/caffe/src/caffe/util/upgrade_proto.cpp.o
[ 93%] Linking CXX shared library libcaffeconverter.dylib
ld: file not found: /Users/miniconda3/envs/apple-dev/lib/arc/libarclite_macosx.a
clang-8: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [libcaffeconverter.dylib] Error 1
make[1]: *** [CMakeFiles/caffeconverter.dir/all] Error 2
make: *** [all] Error 2

Can you please help me to resolve this? Thanks!

1duo · 2019-10-08T17:33:15Z

Here is the system I am using

coremltools version (e.g., 3.0b5): Didn't install prevously. trying to instal from source
OS (e.g., MacOS, Linux): MacOS
macOS version (if applicable): 10.13
XCode version (if applicable): Not using
How you install python (anaconda, virtualenv, system): conda
python version (e.g. 3.7): 3.7.3

Hi @Shekharrajak, the build issue you're hitting is because of the out-dated macOS version (10.13). In the error message, you can see it requires a minimum of macOS 10.14.

py::object convertSequenceValueToPython(MLSequence *seq) API_AVAILABLE(macos(10.14));

Shekharrajak · 2019-10-09T14:36:30Z

Oh! Thanks for letting me know this. I have upgraded the system and it is completed now.

Shekharrajak · 2019-10-09T17:25:00Z

Can you please let me know how can I use the cloned repos and generated build to test the code. I tried running

import coremltools
from PIL import Image
import numpy as np

inside the cloned repo.

1duo · 2019-10-09T17:33:44Z

@Shekharrajak You can now try to build from source again: https://github.com/apple/coremltools#building-from-source

Shekharrajak · 2019-10-11T14:53:01Z

Yes, I tried the same and after that I am running the code snippet (of the issue description):

>>> import coremltools
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/Users/miniconda3/envs/apple-dev/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING:root:Keras version 2.2.5 detected. Last version known to be fully compatible of Keras is 2.2.4 .
>>> from PIL import Image
>>> import numpy as np
>>> data = np.empty((28,28), dtype=np.uint8)
>>> input_image = Image.fromarray(data)
>>> model = coremltools.models.MLModel('../models/MNISTClassifier.mlmodel')
>>> print(model.predict({'image': input_image}))
exception loading model proxy: No module named 'coremltools.libcoremlpython'

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/apple/coremltools/coremltools/models/model.py", line 350, in predict
    raise Exception('Unable to load CoreML.framework. Cannot make predictions.')
Exception: Unable to load CoreML.framework. Cannot make predictions.

1duo · 2019-10-14T17:36:42Z

Hi @Shekharrajak, we'll investigate this build issue, in the meantime, this is due to missing libcoremlpython .so file, you can copy it from <your_build_dir>/coremltools/*.so to <coremltools_repo_root_dir>/coremltools/*.so and run pip install -e . so you have a working developer enviroment. You can verify using python -m pytest coremltools/converters/tensorflow/test/test_convnets.py -v

TobyRoseman · 2019-12-13T00:01:11Z

Installing the oldest version of numpy we want to support, before running cmake and make, solves this issue.

TobyRoseman · 2020-01-28T20:27:41Z

This is not fixed in 3.2. We still get a segfault using the original instructions. Reopening.

* adding more typehints * resolving mypy issues * resolving formatting issue * fixing isort issue Co-authored-by: V Vishnu Anirudh <git.vva@gmail.com> Co-authored-by: V Vishnu Anirudh <vvani@kth.se>

Revert "adding more typehints to DDIM scheduler (apple#456)" This reverts commit a0558b1.

TobyRoseman added the bug Unexpected behaviour that should be corrected (type) label Sep 20, 2019

TobyRoseman mentioned this issue Sep 20, 2019

conflicting numpy dependencies in v5.7 with Python 3.6 apple/turicreate#2249

Closed

1duo added good first issue Issue for a contributor to self-assign and complete help wanted labels Sep 20, 2019

1duo removed the good first issue Issue for a contributor to self-assign and complete label Sep 20, 2019

TobyRoseman mentioned this issue Nov 18, 2019

Unfix numpy dependency apple/turicreate#2664

Closed

TobyRoseman self-assigned this Dec 20, 2019

1duo added the 3.2 label Jan 15, 2020

1duo closed this as completed Jan 15, 2020

TobyRoseman reopened this Jan 28, 2020

srikris mentioned this issue Jan 29, 2020

Add back support for older versions of numpy #602

Merged

srikris closed this as completed in #602 Feb 1, 2020

TomAugspurger mentioned this issue Oct 16, 2020

Update meta.yaml conda-forge/coremltools-feedstock#1

Merged

5 tasks

Birch-san pushed a commit to Birch-san/coremltools that referenced this issue Nov 27, 2022

Revert "adding more typehints to DDIM scheduler" (apple#533)

8897217

Revert "adding more typehints to DDIM scheduler (apple#456)" This reverts commit a0558b1.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrong numpy range in setup.py #456

Wrong numpy range in setup.py #456

Wrong numpy range in setup.py #456

Wrong numpy range in setup.py #456

Comments

🐞Describe the bug

Trace

To Reproduce

System environment (please complete the following information):