I think what we do here depends on whether we see Newcomer Tasks as a new experiment or a continuation of the existing Homepage experiment. If we see it as a continuation, then I'm primarily concerned about the users who are currently in the Homepage treatment group, and I'd then turn on the Help Panel for all of those so they can get the new features.

If Newcomer Tasks is a new experiment, and perhaps one where 100% of users get the Homepage so we can A/B test specific features of Newcomer Tasks, then I see alignment of preferences for previous users as answering two questions: 1) what's needed to ensure the Homepage works for the existing group?, and 2) should any of these features be activated for existing users? I'm not sure what the right answer to question 2 is, but as for 1 I think it's the same as earlier, we'll want to turn the Help Panel on so they can get Newcomer Tasks working correctly.

Do we have a separate phab tasks for discussing whether Newcomer Tasks is a new experiment or not?

@nettrom_WMF and I discussed this, and our current favorite idea is to split up the upcoming newcomer tasks experiment into three groups:

• A: no homepage
• B: homepage with one version of newcomer tasks
• C: homepage with another version of newcomer tasks

This would allow us to see the overall impact of our work, as well as allow us to iterate on newcomer tasks.

@nettrom_WMF is now thinking through that approach and whether it is the best way forward.

How are notices expected to work? E.g. if someone who got the help panel but not the homepage is now realigned to have both, do they have to see the homepage discovery tour?

@Tgr -- thanks for thinking about this. I don't think we would end up giving homepage to users that didn't start with it. It's more likely that we would do it the other way around: giving help panel to users that didn't start with it. We would not want users who are not brand new to start getting popups about their homepage. More to come on this soon as @nettrom_WMF and I finish drafting the measurement plan.

One aspect around this that has come up during the work on the Newcomer Tasks measurement plan is how we handle A/B testing variants of the intervention for existing users. We'll be discussing that in the team and see what we end up with.

@nettrom_WMF and I have decided what approach we want to take for aligning the treatment groups, and have updated the description in this task accordingly. This is ready for development. Because the help panel experiment is complete and the data is being pulled for the homepage experiment, it is okay to deploy this change when it is ready. It does not need to be coordinated with any other deployments of features.

We've decided that aligning the treatment groups is not required for v1.0 because the help panel will not be part of the experience yet.

Moving off sprint board in favor of Newcomer Tasks V1.0 tasks.

Change 553218 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] Allow auto-enable for help panel to be tied to homepage

https://gerrit.wikimedia.org/r/553218

Change 553222 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] GrowthExperiments: Align help panel new account enabling with homepage

https://gerrit.wikimedia.org/r/553222

Change 553218 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Allow auto-enable for help panel to be tied to homepage

https://gerrit.wikimedia.org/r/553218

Change 555639 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] [beta] Align GrowthExperiments treatment groups in beta labs

https://gerrit.wikimedia.org/r/555639

Change 555639 merged by jenkins-bot:
[operations/mediawiki-config@master] [beta] Align GrowthExperiments treatment groups in beta labs

https://gerrit.wikimedia.org/r/555639

Checked in betalabs - new users do get both of the features.
@MMiller_WMF - only two questions

• I did not see old users got Help panel enabled. Would it be re-enabled for users who disabled it manually?
• growthexperiments-tour-homepage-welcome has 0 for accounts that have growthexperiments-homepage-suggestededits-activated. I did not see any implications of it - welcome tour cannot be triggered by a user action.

@Catrope -- how will your implementation handle users who change their preferences manually? Does it, like, automatically turn off help panel if a user turns off homepage? Or is more like it sets them both on for a user, but then whatever happens after that is up to the user?

@Etonkovidova -- can you explain your second point? What are those phrases (growthexperiments-tour-homepage-welcome and growthexperiments-homepage-suggestededits-activated)?

In T232396#5730324, @MMiller_WMF wrote:

@Etonkovidova -- can you explain your second point? What are those phrases (growthexperiments-tour-homepage-welcome and growthexperiments-homepage-suggestededits-activated)?

These growthexperiments-tour-homepage-welcome and growthexperiments-homepage-suggestededits-activated refers to db user_properties table. I actually re-checked in betalabs if users have homepage welcome tour marked as zero but suggested edits were activated - all seems to be fine.

In T232396#5730324, @MMiller_WMF wrote:

@Catrope -- how will your implementation handle users who change their preferences manually? Does it, like, automatically turn off help panel if a user turns off homepage? Or is more like it sets them both on for a user, but then whatever happens after that is up to the user?

It's the latter: at signup time, we will either enable both or neither, but after that the user can do whatever they want in their preferences. There are still separate preferences for the homepage and the help panel, and the user can choose to turn on/off one but not the other.

Change 553222 merged by jenkins-bot:
[operations/mediawiki-config@master] GrowthExperiments: Align help panel new account enabling with homepage

https://gerrit.wikimedia.org/r/553222

Mentioned in SAL (#wikimedia-operations) [2019-12-13T00:10:58Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: GrowthExperiments: Align help panel new account enabling with homepage (T232396) (duration: 00m 56s)