Description

The QuantileLossFunction in sklearn/ensemble/gradient_boosting.py has a sign error, making it take negative values for certain inputs.

@pprett @amueller

Steps/Code to Reproduce

• You can consult formula (2) in http://avesbiodiv.mncn.csic.es/estadistica/curso2011/qr4.pdf for the correct implementation.

• The peculiar thing about the quantile loss function is that it is asymmetric, i.e $Loss(y, pred)$ is different from $Loss(pred, y)$, but of course it has to be positive for all inputs. See graph on top of page 1005 of the previous article for a graph of the loss function.

• The error has no dramatic consequence because the negative gradient is correctly implemented.

• This issue has been raised several times already, corrected, but never merged (FIX #6076. Bug in QuantileLossFunction #6133 and [MRG+1] Corrected sign error in QuantileLossFunction #6429).

• All what needs to be done is to replace + (1.0 - alpha) diff[~mask] by - (1.0 - alpha) diff[~mask] in lines 422 and 425.

Here is the code generating the wrong behavior

from sklearn.ensemble.gradient_boosting import QuantileLossFunction
import numpy as np

x = np.array([-1])
print(QuantileLossFunction(1, 0.5)(x, np.array([0])))

Expected Results

0.5

Actual Results

-0.5

Versions

Darwin-16.3.0-x86_64-i386-64bit
Python 3.5.2 |Anaconda custom (x86_64)| (default, Jul 2 2016, 17:52:12)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)]
NumPy 1.11.2
SciPy 0.18.1
Scikit-Learn 0.19.dev0