FIX array_api support for non-integer n_components in PCA #27431
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ogrisel
commented
Sep 21, 2023
|
I converting this PR back to draft because ============================================================================ FAILURES ============================================================================
_______________ test_pca_array_api_compliance[PCA(n_components=0.1,svd_solver='full',whiten=True)-check_array_api_input_and_values-cupy-None-None] _______________
estimator = PCA(n_components=0.1, svd_solver='full', whiten=True), check = <function check_array_api_input_and_values at 0x7f1dca8adda0>, array_namespace = 'cupy'
device = None, dtype = None
@pytest.mark.parametrize(
"array_namespace, device, dtype", yield_namespace_device_dtype_combinations()
)
@pytest.mark.parametrize(
"check",
[check_array_api_input_and_values, check_array_api_get_precision],
ids=_get_check_estimator_ids,
)
@pytest.mark.parametrize(
"estimator",
[
PCA(n_components=2, svd_solver="full"),
PCA(n_components=0.1, svd_solver="full", whiten=True),
PCA(
n_components=2,
svd_solver="randomized",
power_iteration_normalizer="QR",
random_state=0, # how to use global_random_seed here?
),
],
ids=_get_check_estimator_ids,
)
def test_pca_array_api_compliance(estimator, check, array_namespace, device, dtype):
name = estimator.__class__.__name__
xp, device, dtype = _array_api_for_tests(array_namespace, device, dtype)
if not isinstance(estimator.n_components, int) and not hasattr(xp, "cumsum"):
# Our code anticipates the implementation of xp.cumsum that should be
# standardized at some point, see:
# https://github.com/data-apis/array-api/issues/597
pytest.xfail(
f"Array API namespace {array_namespace} does not support cumsum yet."
)
if not isinstance(estimator.n_components, int) and not hasattr(xp, "searchsorted"):
# Our code anticipates the implementation of xp.searchsorted that
# should be standardized at some point, see:
# https://github.com/data-apis/array-api/issues/688
pytest.xfail(
f"Array API namespace {array_namespace} does not support searchsorted yet."
)
> check(name, estimator, array_namespace, device=device, dtype=dtype)
array_namespace = 'cupy'
check = <function check_array_api_input_and_values at 0x7f1dca8adda0>
device = None
dtype = None
estimator = PCA(n_components=0.1, svd_solver='full', whiten=True)
name = 'PCA'
xp = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
sklearn/decomposition/tests/test_pca.py:773:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
sklearn/utils/estimator_checks.py:1002: in check_array_api_input_and_values
return check_array_api_input(
array_namespace = 'cupy'
device = None
dtype = None
estimator_orig = PCA(n_components=0.1, svd_solver='full', whiten=True)
name = 'PCA'
sklearn/utils/estimator_checks.py:889: in check_array_api_input
est_xp.fit(X_xp, y_xp)
X = array([[-2.02514259, 0.0291022 , -0.47494531, ..., -0.33450124,
0.86575519, -1.20029641],
[ 1.6137112...623, -1.06511366],
[-0.00607091, 1.30857636, -0.17495976, ..., 0.99204235,
0.32169781, -0.66809045]])
X_xp = array([[-2.02514259, 0.0291022 , -0.47494531, ..., -0.33450124,
0.86575519, -1.20029641],
[ 1.6137112...623, -1.06511366],
[-0.00607091, 1.30857636, -0.17495976, ..., 0.99204235,
0.32169781, -0.66809045]])
array_attributes = {'components_': array([[-0.03484641, -0.60455227, 0.09228081, 0.09317968, -0.02118701,
0.46225127, -0.03672..., -0.03647842, 0.00852413, 0.13682879,
-0.03120609, 0.05840072, 0.05582498, 0.12556161, -0.03976961]), ...}
array_namespace = 'cupy'
check_values = True
device = None
dtype = None
est = PCA(n_components=0.1, svd_solver='full', whiten=True)
est_xp = PCA(n_components=0.1, svd_solver='full', whiten=True)
estimator_orig = PCA(n_components=0.1, svd_solver='full', whiten=True)
name = 'PCA'
xp = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
y = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
y_xp = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
sklearn/base.py:1215: in wrapper
return fit_method(estimator, *args, **kwargs)
args = (array([[-2.02514259, 0.0291022 , -0.47494531, ..., -0.33450124,
0.86575519, -1.20029641],
[ 1.613711...,
0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0]))
estimator = PCA(n_components=0.1, svd_solver='full', whiten=True)
fit_method = <function PCA.fit at 0x7f1dcac16c00>
global_skip_validation = False
kwargs = {}
partial_fit_and_fitted = False
prefer_skip_nested_validation = True
sklearn/decomposition/_pca.py:438: in fit
self._fit(X)
X = array([[-2.02514259, 0.0291022 , -0.47494531, ..., -0.33450124,
0.86575519, -1.20029641],
[ 1.6137112...623, -1.06511366],
[-0.00607091, 1.30857636, -0.17495976, ..., 0.99204235,
0.32169781, -0.66809045]])
self = PCA(n_components=0.1, svd_solver='full', whiten=True)
y = array([0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,
0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0,...1,
0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1,
1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0])
sklearn/decomposition/_pca.py:520: in _fit
return self._fit_full(X, n_components)
X = array([[-2.21502293, -0.00923 , -0.55143331, ..., -0.39032622,
0.74019358, -1.1605268 ],
[ 1.4238309...785, -1.02534406],
[-0.19595126, 1.27024416, -0.25144776, ..., 0.93621737,
0.19613619, -0.62832085]])
is_array_api_compliant = True
n_components = 0.1
self = PCA(n_components=0.1, svd_solver='full', whiten=True)
xp = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
sklearn/decomposition/_pca.py:585: in _fit_full
n_components = xp.searchsorted(ratio_cumsum, n_components, side="right") + 1
S = array([2.02981634e+01, 1.42856603e+01, 1.28046793e+01, 1.21903333e+01,
1.18505054e+01, 1.16648157e+01, 1.109279...8.04206402e+00, 7.80207531e+00, 7.50569464e+00,
7.01282418e+00, 6.55356018e+00, 2.57592758e-15, 8.36741292e-16])
U = array([[ 0.00893647, 0.05215268, -0.09820667, ..., -0.0008873 ,
0.12132781, 0.01039382],
[-0.0242668...911, 0.03320314],
[-0.05989941, -0.06672174, -0.08371833, ..., 0.04763651,
-0.03077404, 0.01670712]])
Vt = array([[-3.48464090e-02, -6.04552271e-01, 9.22808056e-02,
9.31796838e-02, -2.11870097e-02, 4.62251267e-01,
...-4.62021233e-17,
6.93044994e-01, 5.18349048e-17, 1.48737107e-17,
2.97854026e-17, -2.14399161e-17]])
X = array([[-2.21502293, -0.00923 , -0.55143331, ..., -0.39032622,
0.74019358, -1.1605268 ],
[ 1.4238309...785, -1.02534406],
[-0.19595126, 1.27024416, -0.25144776, ..., 0.93621737,
0.19613619, -0.62832085]])
components_ = array([[-3.48464090e-02, -6.04552271e-01, 9.22808056e-02,
9.31796838e-02, -2.11870097e-02, 4.62251267e-01,
...-4.62021233e-17,
6.93044994e-01, 5.18349048e-17, 1.48737107e-17,
2.97854026e-17, -2.14399161e-17]])
explained_variance_ = array([4.16177210e+00, 2.06141506e+00, 1.65615972e+00, 1.50105278e+00,
1.41853008e+00, 1.37442348e+00, 1.242930...6.53280745e-01, 6.14872516e-01, 5.69044969e-01,
4.96764677e-01, 4.33829808e-01, 6.70242715e-32, 7.07208071e-33])
explained_variance_ratio_ = array([1.92326985e-01, 9.52636840e-02, 7.65357155e-02, 6.93677955e-02,
6.55541933e-02, 6.35159055e-02, 5.743924...3.01899078e-02, 2.84149574e-02, 2.62971399e-02,
2.29568679e-02, 2.00484738e-02, 3.09737673e-33, 3.26820385e-34])
is_array_api_compliant = True
n_components = 0.1
n_features = 20
n_samples = 100
ratio_cumsum = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
0.95699466, 0.97995153, 1. , 1. , 1. ])
self = PCA(n_components=0.1, svd_solver='full', whiten=True)
singular_values_ = array([2.02981634e+01, 1.42856603e+01, 1.28046793e+01, 1.21903333e+01,
1.18505054e+01, 1.16648157e+01, 1.109279...8.04206402e+00, 7.80207531e+00, 7.50569464e+00,
7.01282418e+00, 6.55356018e+00, 2.57592758e-15, 8.36741292e-16])
total_var = array(21.63904407)
xp = <module 'array_api_compat.cupy' from '/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/array_api_compat/cupy/__init__.py'>
/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/cupy/_sorting/search.py:422: in searchsorted
return _searchsorted(a, v, side, sorter, True)
a = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
0.95699466, 0.97995153, 1. , 1. , 1. ])
side = 'right'
sorter = None
v = 0.1
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
0.95699466, 0.97995153, 1. , 1. , 1. ])
v = 0.1, side = 'right', sorter = None, assume_increasing = True
def _searchsorted(a, v, side, sorter, assume_increasing):
"""`assume_increasing` is used in the kernel to
skip monotonically increasing or decreasing verification
inside the cuda kernel.
"""
if not isinstance(a, cupy.ndarray):
raise NotImplementedError('Only int or ndarray are supported for a')
if not isinstance(v, cupy.ndarray):
> raise NotImplementedError('Only int or ndarray are supported for v')
E NotImplementedError: Only int or ndarray are supported for v
a = array([0.19232698, 0.28759067, 0.36412638, 0.43349418, 0.49904837,
0.56256428, 0.62000352, 0.67092467, 0.715985...26, 0.83675251, 0.87209265, 0.90228256, 0.93069752,
0.95699466, 0.97995153, 1. , 1. , 1. ])
assume_increasing = True
side = 'right'
sorter = None
v = 0.1
/data/parietal/store3/work/ogrisel/mambaforge/envs/pytorch/lib/python3.11/site-packages/cupy/_sorting/search.py:434: NotImplementedError |
|
Ok now it seems to work on all supported hardware/library combinations:
|
|
/cc @betatim @thomasjpfan |
betatim
reviewed
Sep 29, 2023
betatim
approved these changes
Oct 13, 2023
|
LGTM |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #27427.
Our array api tests did not check for cases such as
PCA(n_components=.2)andPCA(n_components="mle")and therefore did not check some helper functions such asstable_cumsumand_assess_dimension.Here is a start to do so but:
stable_cumsumtries to force the use ofxp.float64which is not available on MPS devices for PyTorch.searchsortedto the specification data-apis/array-api#688In the long term, I think we might want to deprecate
stable_cumsumand try to use a regular cumsum with the input data dtype instead.EDIT: In the mean time, we can just do the cumsum/searchsorted calls on a local numpy copy, this is not an expensive operation in the case of the
PCA(only depends onn_features).