These kinds of cross-platform inconsistencies often come from things like uninitialised memory... Not sure where to look.

At #11925 I was trying to identify whether the discrepancy came from the initial fits for each fold, or from the refit process in LogisticRegressionCV (as this would make sense for why it's broken only for the CV variant).

From https://travis-ci.org/scikit-learn/scikit-learn/jobs/421403473#L375 we see that while the binary logistic fits produce an identical model each time, the mean coefficients of the best model from CV are slightly different across repeated runs (and I'm not sure we should be using the mean here):

(multi_class='auto')

[[ -0.59306404   3.09940484  -6.811572    -2.96272071  26.26385125]
 [ -1.55696142  -2.36361588   0.09320993  -5.3674105   29.95119552]
 [  2.15002545  -0.73578896   6.71836207   8.33013121 -56.21504677]]

vs
(multi_class='multinomial')

[[ -0.59604468   3.09992681  -6.8005941   -2.98726904  26.26175622]
 [ -1.56230187  -2.34716593   0.08463475  -5.34018104  29.92984087]
 [  2.15834654  -0.75276089   6.71595935   8.32745008 -56.19159708]]

Using these as the initial coefficients results in a vastly discrepant refit:

[[ -1.57069836   2.66750768  -9.14886852  -4.43393543  47.95429526]
 [ -2.90164875  -2.27389574  -1.65205449  -6.94935689  50.24802659]
 [  4.47234712  -0.39361193  10.80092301  11.38329232 -98.20232185]]

vs

[[-2.01577764e+00  2.68288189e+00 -9.15560217e+00 -4.01869064e+00
   5.02771291e+01]
 [-2.84247918e+00 -2.76571395e+00 -1.49917595e+00 -7.49195111e+00
   5.15594962e+01]
 [ 4.85825682e+00  8.28320544e-02  1.06547781e+01  1.15106417e+01
  -1.01836625e+02]]

I think this can't be just about LogisticRegressionCV. At https://travis-ci.org/scikit-learn/scikit-learn/jobs/421410785, for two of the three KFold splits, we get identical coefficients. For the third split, one fit gets:

 [[-2.15440499e+00  2.59582912e+00 -9.40201575e+00 -3.45705788e+00
    5.14770829e+01]
  [-2.72327380e+00 -2.58044861e+00 -1.36975377e+00 -7.73415753e+00
    5.00827769e+01]
  [ 4.87767879e+00 -1.53805063e-02  1.07717695e+01  1.11912154e+01
   -1.01559860e+02]]

The other gets:

 [[-1.24424826e+00  3.47553540e+00 -9.13287242e+00 -4.34631953e+00
    4.32699505e+01]
  [-3.14665603e+00 -2.61939440e+00 -1.22801012e+00 -7.34972009e+00
    5.14471399e+01]
  [ 4.39090429e+00 -8.56140993e-01  1.03608825e+01  1.16960396e+01
   -9.47170904e+01]]]

Which seems quite different.

The fact that this occurs when multi_class='multinomial' and not with ovr or binary problems sent me to investigate the gradient function _multinomial_loss_grad, but I can't see an issue there.

I'm wondering if we should just skip the test on mac: I presume that this same bug was present, but not tested for, in 0.19, so it's not really release sensitive!!

Thanks for investigating this! +1 to skip that assert for now.

I have the same problem on MacOS, that the test of LogisticRegressionCV with newton-cg fail nearly all the time and rarely the test LogisticRegression with newton-cg fails as well.

Not equal to tolerance rtol=1e-07, atol=0
Mismatched elements: 6 / 12 (50%)
Max absolute difference: 9.82197427e-08
Max relative difference: 4.4415004e-07

Sorry, the solver lbfgs doesn't actually fail, so I opened a new Issue #16279.

Can we close this issue?
We moved from travis to azure CI for MacOS. CI seems to pass.

BTW, this issue seems to focus on test_logistic_regression_multi_class_auto[lbfgs-est1] where est1 = LogisticRegressionCV. If there still is a problem, it would be nice to have a test that actually fails.

Okay, lets close this issue as the CI on macOS has been passing for a while, along with the wheel building on OSX.