Autonag: Bugzilla triaging bot and alerting system
This page lists all the actions done by the Autonag bot.
Contents
- 1 Introduction
- 2 Rules
- 2.1 With autofix
- 2.1.1 Bug assigned but marked as UNCONFIRMED
- 2.1.2 Regression keyword is missing
- 2.1.3 Remove leave-open keyword on closed bug
- 2.1.4 [meta] in title but not in keywords
- 2.1.5 Update Firefox Status flags for bugs reopened during Nightly cycle
- 2.1.6 Bug with no assignee but a patch landed
- 2.1.7 Close crash bugs with no crashes over 12 weeks
- 2.1.8 Remove stalled keyword on closed bugs
- 2.1.9 Has meta keyword but not [meta] in the bug title
- 2.1.10 Set a Firefox status flag for beta if there is one for nightly and release
- 2.1.11 Increase priority of bugs tracked y Release Managers
- 2.1.12 Add regression keywords to bugs (uses Machine Learning)
- 2.1.13 Move untriaged bug into the correct component (uses machine learning)
- 2.1.14 Copy crash signature from duplicate bugs to main bugs
- 2.1.15 Increase severity of bugs tracked by Release Managers
- 2.1.16 Nag triage owner to set priority on bugs
- 2.1.17 Nag triage owner on P1 bugs with no activity
- 2.1.18 Nag triage owner to find an assignee for P1 bugs with no assignee and no activity
- 2.1.19 Change the priority from P2 to P1 on assigned bug the merge day
- 2.1.20 Patch not landed
- 2.1.21 Regression without regressed_by and some bug dependencies
- 2.1.22 Product::Component changed after the priority has been set
- 2.1.23 Close intermittent test failure bugs
- 2.1.24 Ask for a beta uplift request
- 2.2 Without autofix
- 2.2.1 Feature vs regression
- 2.2.2 Inactive bugs with the leave-open keyword set
- 2.2.3 Inactive Meta bugs
- 2.2.4 Low-severity bug but tracked by a release manager
- 2.2.5 Alert for lack of feedback in a bug
- 2.2.6 Bugs with no priority and no activity
- 2.2.7 Bugs with only one or two words in the summary
- 2.2.8 Remind developers about tracked bugs
- 2.2.9 Reporter not answering to Needinfo request
- 2.2.10 Top crashers vs normal severity
- 2.2.11 Identify tracked bugs with a NeedInfo request
- 2.2.12 Tracked bugs untouched for a week
- 2.2.13 Regressed in upcoming release
- 2.2.14 Identify unlanded uplifts to Beta and ESR
- 2.2.15 Bugs with high severity in Firefox::Untriaged
- 2.2.16 Version set but not status_firefox
- 2.2.17 Check release dates on the Wiki
- 2.2.18 Information about bugs landed during Soft Code Freeze
- 2.2.19 Hightlight Fixed bugs in nightly we may want to uplift
- 2.2.20 New user with a Needinfo request
- 2.1 With autofix
Introduction
Every day hundreds of tickets are opened in Bugzilla which track the tasks, defects and enhancements needed for the development of Firefox and other Mozilla projects. Triaging and priotirizing these bugs are an essential part of our development process and we automate part of this process so as to increase our development turn around and improve the quality of our bugs metadata. The set of scripts we use to improve the quality of our bugs, decrease our triaging time and help Release Management ship better quality software is called Autonag.
Initially created as an alerting system (by email) to our triagers, Autonag rules now also make changes to our bugs metadata on Bugzilla so as to fix inconsistencies.
Rules that change automatically some data in Bugzilla (change a priority, needinfo somebody, close a bug…) are called "Rules with Autofix" and those rules are applied once per hour. For now, security bugs aren't touched.
Rules that do not change data but have other results such as generate a report or send an email are called "Rules without Autofix". Rules without autofix are processed once a day (at 2PM CET).
Rules
With autofix
UNCONFIRMED
Purpose | Mitigate an issue in Bugzilla (bugs reported by new users are not tagged as NEW |
---|---|
Action | Change the status from UNCONFIRMED to ASSIGNED if there is an assignee |
Example | Bug 1495908 |
Code | assignee_but_unconfirmed.py |
Purpose | Regression keyword is important to differentiate an actual defect |
---|---|
Action | Sets the regression keyword on bugs with the regression-range-wanted keyword is set |
Example | Bug 1461034 |
Code | has_regression_range_no_keyword.py |
leave-open
keyword on closed bug
Purpose | Clean up a mismatch in metadata |
---|---|
Action | If a bug is closed but the leave-open keyword is still set, remove it |
Example | Bug 1382185 |
Code | leave_open.py |
Purpose | Improve metedata quality |
---|---|
Action | Adds the meta keyword if the bug title starts by [META] |
Example | Bug 1435799 |
Code | meta_summary_missing.py |
Purpose | Avoids potential issues for sheriffs and release managers |
---|---|
Action | Set firefox-status flag back from fixed to affected |
Example | Bug 1495962 |
Code | nightly_reopened.py |
Purpose | Attribute unassigned bug to the developer that fixed it |
---|---|
Action | The ASSIGNEE field on the bug is changed from nobody@mozilla.org to the author of the patch that landed in mozilla-central |
Example | Bug 1514338 |
Code | no_assignee.py |
Purpose | Reduce the backlog of bugs to check for Release Managers |
---|---|
Action | Crash bugs without any crash reports for more than 12 weeks |
Example | Bug 1470864 |
Code | no_crashes.py |
stalled
keyword on closed bugs
Purpose | Fix inconsistency between metadata and bug status |
---|---|
Action | If a bug is marked as FIXED and also has a stalled keyword set, the keyword is removed |
Example | Bug 1491624 |
Code | stalled.py |
Purpose | Having [meta] in the bug title helps with quick search results |
---|---|
Action | If a bug has the meta keyword set, [meta] is added to the bug title |
Example | Bug 1257692 |
Code | summary_meta_missing.py |
Purpose | Fix inconsistency in Firefox status flags that can lead to a bug going undetected by Release Managers during the beta cycle |
---|---|
Action | If a status exists for Firefox N-1 and for Firefox N+1, guess a value for Firefox N |
Example | Bug 1500273 |
Code | missing_beta_status.py |
Purpose | Priotitizes bugs needing an action for shipping quality software |
---|---|
Action | If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest) |
Example | Bug 1515946,1508277,1512493 |
Code | mismatch_priority_tracking_release.py |
Note | There are multiple files mismatch-priority-tracking-*.py, one per channel |
regression
keywords to bugs (uses Machine Learning)
Purpose | Surface regressions not filed as such |
---|---|
Action | If a bug is tracked by a Release Manager, update the priority (P2 for nightly, P1 for the rest) |
Example | Bug 1529139 |
Code | regression.py |
Purpose | Decrease manual triage time |
---|---|
Action | Uses machine learning to mass move bugs in Firefox::Untriaged into a better component |
Example | Bug 1530316 |
Code | component.py |
Purpose | Crash bugs marked as duplicate of another one may have a different crash signature. We need to consolidate all signatures in the bug where a patch to fix them is being worked on, other wise we may not fix them all |
---|---|
Action | If a crash bug is marked as a duplicate, the signatures it referenced are added to the new bug |
Example | Bug 1517205 |
Code | copy_duplicate_info.py |
Purpose | Fix inconsistency in metadata |
---|---|
Action | If a bug is tracked for an upcoming release but it's severity field is low, the severity is increased |
Example | Bug 1538966 |
Code | tracked_bad_severity.py |
Purpose | Every defect should have a severity set. Also bugs with no severity set are harder to prioritize for Release Managers with regards to the release (see what-do-you-triage) |
---|---|
Action | The triage owner is needinfoed in Bugzilla or nagged via email to set a priority on bugs |
Example | Bug 1527818 |
Code | workflow/no_severity.py |
Purpose | Since P1 means that the bug should be fixed for the next release (see what-do-you-triage), the bug needs to show any progress in its resolution (or maybe the priority should be decreased) |
---|---|
Action | The triage owner is nagged via email to have a reminder |
Example | Bug 1507251 |
Code | workflow/p1_no_activity.py |
Purpose | Since P1 means that the bug should be fixed for the next release (see what-do-you-triage), the bug urgently needs to have an assignee (or the priority should be decreased) |
---|---|
Action | The triage owner is needinfoed in Bugzilla or nagged via email to find an assignee on bugs |
Example | Bug 1541291 |
Code | workflow/p1_no_assignee.py |
Purpose | Since P2 means that the bug should be fixed for the next next release (see what-do-you-triage), when we are the merge day then this bug automatically becomes a P1. |
---|---|
Action | Update the priority to P1 and add a comment to explain. |
Code | workflow/p2_merge_day.py |
Purpose | Identify bugs with an unlanded r+ patch. Maybe the patch is now obsolete or the bug needs to have a sheriff land the code with the checkin-needed keyword set. |
---|---|
Action | The bug assignee is needinfoed in Bugzilla to ask her/him to have look |
Example | Bug 1496844 |
Code | not_landed.py |
Purpose | Identify regressions which have dependencies (blocks or depends_on) and an empty regressed_by, probably people forgot to use the new field. |
---|---|
Action | The bug assignee or the person who added some dependencies or the reporter is needinfoed to ask to fill regressed_by when it's possible. |
Example | Bug 1545797 |
Code | regression_without_regressed_by.py |
Purpose | Identify bugs where Product::Component changed after the priority has been set. The priority is probably no more up-to-date and so triage owner needs to set it. |
---|---|
Action | Reset the priority to "--" with a comment explaining why the bot is doing that. |
Example | Bug 1546366 |
Code | prod_comp_changed_with_priority.py |
Purpose | Close intermittent test failure bugs which are P5s, not marked leave-open , and have not had a new failure in 21 days. |
---|---|
Action | RESOLVE bug as INCOMPLETE with a comment explaining why the bot is doing this. |
Example | Bug 1577895 |
Code | close_intermittents.py |
Purpose | Identify bugs where a patch fixed a bug in nightly and beta is marked as affected. |
---|---|
Action | Needinfo the assignee to ask it's worth making an uplift request. |
Example | Bug 1620434 |
Code | uplift_beta.py |
Without autofix
Purpose | Fix inconsistency in bugs with both the regression and feature keywords set |
---|---|
Action | No action yet |
Code | feature_regression.py |
leave-open
keyword set
Purpose | Help identify dead/inactive bugs |
---|---|
Action | Needinfo the triage owner if a bug is inactive but has the leave-open keyword set |
Example | Bug 1367072 |
Code | leave_open_no_activity.py |
Purpose | Help identify dead/inactive bugs |
---|---|
Action | Needinfo the triage owner if a meta bug has no activity and no dependencies set |
Code | meta_no_deps_no_activity.py |
Purpose | Idenitfy mismatch in metadata, a tracked bug should not have a priotity lower than normal |
---|---|
Action | No action yet |
Code | mismatch-priority-tracking.py |
Purpose | Increase reaction time on bugs prioritized by upper management or release managers |
---|---|
Action | If a bug as an unanswered NeedInfo request from a release manager or a director, send a warning email |
Code | ni_from_manager.py |
Purpose | Triage owners need to process the backlog |
---|---|
Action | Needinfo the triage owner (only :Overholt for now) |
Example | Bug 1503461 |
Code | ni_triage_owner.py |
Purpose | Make sure the bug is actually useful, a two-words summary often indicates a poor quality bug report |
---|---|
Action | No action yet |
Example | Bug 1512823 |
Code | one_two_word_summary.py |
Purpose | Make sure that bugs tracked for the next release are addressed |
---|---|
Action | No action yet |
Code | query_creator.py |
Purpose | identify bugs stalled because we need more information fron the reporter to reproduce it |
---|---|
Action | No action yet |
Code | reporter_with_ni.py |
Purpose | Consistency and help getting traction on bugs |
---|---|
Action | No action yet |
Example | Bug 1471692 |
Code | topcrash_bad_severity.py |
Purpose | Make sure that bugs tracked for a release or nominated for tracking which also have a NeedInfo request do not get stuck |
---|---|
Action | Send an email |
Code | tracked_needinfo.py |
Purpose | Identify important bugs for a release which are not being acted upon |
---|---|
Action | Send an email |
Code | tracking.py |
Purpose | Identify potential regression |
---|---|
Action | Send an email with the list of bugs marked as non affecting a release but affecting the next one |
Code | unaffected_affected_no_reg.py |
Purpose | Make sure that we ship with the bugs that we need |
---|---|
Action | Send an email wih the list of bugs with uplift requests not landed on the affected branch |
Example | Bug 1509394 |
Code | unlanded.py |
Purpose | Identify potentially important issues left untriaged |
---|---|
Action | No action yet |
Code | untriage_important_sev.py |
Version
set but not status_firefox
Purpose | The Version value is set automatically by Bugzilla on landing a patch but not the status_firefoxXX |
---|---|
Action | Send an email |
Code | version_affected.py |
Purpose | Verify the consistency of our release dates on wiki.mozilla.org as we use them in automation |
---|---|
Action | Send an email |
Code | ../next_release.py |
Purpose | List fixed bug with patches which landed in mozilla-central during the soft freeze week. The listing includes patch statistics. |
---|---|
Action | Send an email |
Code | code_freeze_week.py |
Purpose | Identify potentially upliftable bugs. |
---|---|
Action | Send an email with the recent fixes, the affected branches and metadata for prioritization |
Code | missed_uplifts.py |
Purpose | Help identify potentially unactionnable bugs per lack of information |
---|---|
Action | Send an email |
Code | newbie_with_ni.py |