-
-
Notifications
You must be signed in to change notification settings - Fork 25.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH Added warning for ndcg_score
when used w/ negative y_true
values
#23461
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have not looked into the literature. Is there a valid use case for y_true < 0
?
There was a bit of discussion about this in the original issue #17639. I'm largely basing this off of Wang et. al. 2013 which has ~250 citations (details) and affirms the expectation that NDCG is contained within Indeed, with the idea of relevance in document retrieval where NDCG was first formulated, it makes more sense for an additional document to offer no information and have no relevance, than to remove information or relevance from the overall query. From what I've read so far, including this additional survey paper talking about IR methods overall, cumulative gain (CG) is assumed to be monotone non-decreasing, so I don't think that |
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nit, otherwise LGTM.
Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I just fixed the what's new entry that was not in the appropriate section.
Thanks @Micky774
…ues (scikit-learn#23461) Co-authored-by: trinhcon <conroy.trinh@mail.utoronto.ca> Co-authored-by: Victor Ko <vk07275@gmail.com> Co-authored-by: Thomas J. Fan <thomasjpfan@gmail.com> Co-authored-by: Jérémie du Boisberranger <34657725+jeremiedbb@users.noreply.github.com>
Reference Issues/PRs
Resolves #22710
Fixes #17639
What does this implement/fix? Explain your changes.
Adds warning for
ndcg_score
when used w/ negativey_true
valuesAny other comments?