weekly update:

• received a number of constructive feedbacks
• reviewed and discussed around guideline scopes

closed as duplicate of https://phabricator.wikimedia.org/T372707

It seems that we want to solve two problems - one being classifying good faith vs bad faith, another being the scoring itself which indicates the degree of it

• In this case, a simple algorithm like logreg would be a good staring point, as the algorithm will set weights to each variable; the algorithm will also output a interpreted probability as the likelihood of a given record is being either 1 (good faith) or 0 (bad faith)
• The scoring can then be taken from the likelihood directly

For "ground truth" data collection, one thing to be careful about is the nature of the good or bad faith data.

In T321224#10072541, @Lydia_Pintscher wrote:

In T321224#10054680, @XiaoXiao-WMF wrote:

from product side:

• product adoption part is not clear @Lydia_Pintscher please clarify

From my side this would still be very useful to have, especially for tracking how the content evolves over time at scale. It is however less important than the revert risk model for Wikidata.

no update this week, will pick up this task next week

@leila Yes, I have read it, the only question I have is do we (internally) have a criteria on bring a presentation forward as of the stage of the project? There seems to be an implicit understanding of it but not obvious to me...

weekly update:

• discussed Miriam's feedback and iterated on it
• obtained more examples from the team to make the guideline as specific and general as possible

Have spoken to @Samwalton9-WMF and @Trizek. Since this is a low-priority ask, and we expect that the level of vandalism in French wikionary will be fairy low compared to Wikipedias. This ask hasn't been looked into in details by the moderator tool team as per product development work in short term.

no capacity for rewriting the model at the moment - the model cannot be easily adopted by the current version without creating considerably amount of maintenance

summary from discussion with Isaac:

• we have a simple prototype done
• this prototype needs fairly large amount of pre-computed properties to work effectively in production
• without feature storage component readily being available today, there is no pathway for sustainable maintenance in production

the points above are mostly infrastructure constrains
@isarantopoulos fyi

Okay perfect. Thanks, I will mark this as resolve!

okay, so how can I renew this now? I have passed the 7/22 deadline

HI @SLyngshede-WMF, I have done these steps a while ago, here is what I see - it seems that it is only valid for a few days, does it look right to you?

I have followed the email instruction and I have done this step on May 23rd, and now I log into the stat machine I still see this line: "You do not have a valid Kerberos ticket in the credential cache, remember to kinit."

@fkaelin description changed as per discussion, please report your progress weekly on Fridays. Thanks!

weekly update:

In T357036#9976771, @Aitolkyn wrote:

Prepared the code here: https://gitlab.wikimedia.org/repos/research/reference-quality/-/tree/classifiers

ref. need - time performance tested with CPU setting (no batching) on 1000 revisions shows 50% of revisions take <0.70sec for prediction, and 95% of revisions take <6.06sec, refer to the cdf distribution below

ref.risk - annotating additional 1000 domains

a comment for language support

weekly update:

• thanks for Miriam's feedback, a few adjustments added

weekly update:

• deliverables for the remaining two essential work tasks are being finalized, though there are back-and-forth on details that have not been agreed up on. Will finalized on Monday.

weekly update:

initial document:
https://docs.google.com/document/d/1Va6owaYtpx9eNMB0ZkhHS_iTDQhzj2NHYnsdq7QVjZY/edit?pli=1#heading=h.63vmzm9jkjni

reopening this task as research engineering and deployment component is not done yet.

closing the ticket to mark the conclusion of the PoC on this.

most of the work is done, what's left is to resolve comments on the MR above. Will resolve this task mid-July ...

dependency on https://phabricator.wikimedia.org/T357036 will start once this task is concluded

@MunizaA any comments please put in the ticket and please close this out

will continue under our essential work for model quantization in q1

Muniza has completed the last bit of the task:
https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/blob/4062396de9545939e8e892003f14ecb8344f933f/research/dags/article_topics_dag.py

Do we have an estimate of when this access can be given to Muniza?
My understanding is that we are sort of blocked by this access, if it can be resolved sooner that would be appreciated. Thanks!

weekly update:

to review https://gitlab.wikimedia.org/repos/research/research-datasets/-/merge_requests/31

airflow done, will merge this week

still in research phase