[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clients(devtools): audits2 -> audits1 and defer reading resources #9344

Merged
merged 3 commits into from
Jul 11, 2019

Conversation

connorjclark
Copy link
Collaborator

Current canary errors when saving as HTML. Issue is with how DevTools is bundled. Read comments for more.

Associated chromium change: https://chromium-review.googlesource.com/c/chromium/src/+/1696348

Bug: https://bugs.chromium.org/p/chromium/issues/detail?id=979131

@@ -21,7 +21,7 @@ else
frontend_dir="$chromium_dir/third_party/blink/renderer/devtools/front_end"
fi

tests_dir="$frontend_dir/../../../web_tests/http/tests/devtools/audits2"
tests_dir="$frontend_dir/../../../web_tests/http/tests/devtools/audits"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

totally slipped on this

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this only affects the test fixture/expectation updates though right?

Copy link
Collaborator
@patrickhulce patrickhulce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -43,10 +43,10 @@ cp -pPR "$lh_bg_js" "$lh_worker_dir/lighthouse-dt-bundle.js"
echo -e "\033[96m ✓\033[39m (Potentially stale) lighthouse-dt-bundle copied."

# copy report generator + cached resources into $fe_lh_dir
cp -r dist/dt-report-resources/ $fe_lh_dir
cp -r dist/dt-report-resources/* $fe_lh_dir
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I faintly recall commenting that folder/ vs. folder/* vs folder seem scary to me.

Is it time to expand the comment? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you were referring to folder vs folder/. This glob syntax seems obvious to me.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess my point is, why the change now? Does this have any effect on our situation? If it doesn't, why are we changing it? If it does have an effect and was wrong before, then it probably deserves capturing whatever subtle mistake led to this.

AFAIK this should be the same if $fe_lh_dir is always going to exist, is that right?

Copy link
Collaborator Author
@connorjclark connorjclark Jul 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was always wrong and I didn't notice during any of the rolls to Chromium.

This landed 4 months ago, but for each roll I actually just ignored / deleted the dt-report-assets b/c the Chromium changes that necessitated it (saving as HTML) were still pending review. That just changed during this latest roll. So I've now noticed.

#7567 (comment) was wrong (see https://askubuntu.com/a/1151667/399299)

I'll try cp -r dist/dt-report-resources/. $fe_lh_dir (seems better?) and update with a comment.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm that's weird maybe it's OS/shell/cp-impl dependent?

On my machine with zsh, cp -r folder/ other and cp -r folder/ other/ both result in the files of folder being copied into other without the extra folder/ layer suggested by https://askubuntu.com/a/1151667/399299

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way, very clearly stating the desired effect of the cp command in a comment with the important variations that won't work seems like a great idea to me :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think my issue was that the folder already existed, perhaps from much earlier messing around, so in that case it does copy into that folder.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it would work in most cases, but not in the one small case where you happen to have a dt-report-resources folder :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh dt-report-resources already existed in other/? interesting, I guess cp be all kinds of 😑

@@ -16,9 +16,21 @@
// @ts-ignore: Runtime exists in Devtools.
const cachedResources = Runtime.cachedResources;

// Getters are necessary because the DevTools bundling processes
// resources after this module is resolved. These properties are not
// read from immeditely, so we can defer reading with getters and everything
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe worth clarifying when exactly in the lifecycle these are/can be accessed since it's burned us?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the time the Audits panel is visible, it's fine. Which is basically the beginning (so there's not really any restrictions to keep in mind for usage afaict). Just not at module-parsing time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is restricted to the tiniest bundle we have report-generator.js:

(function(f){if(typeof exports==="object"&&typeof module!=="undefined"){module.exports=f()}else if(typeof define==="function"&&define.amd){define([],f)}else{var g;if(typeof window!=="undefined"){g=window}else if(typeof global!=="undefined"){g=global}else if(typeof self!=="undefined"){g=self}else{g=this}(g.Lighthouse || (g.Lighthouse = {})).ReportGenerator = f()}})(function(){var define,module,exports;return (function(){function r(e,n,t){function o(i,f){if(!n[i]){if(!e[i]){var c="function"==typeof require&&require;if(!f&&c)return c(i,!0);if(u)return u(i,!0);var a=new Error("Cannot find module '"+i+"'");throw a.code="MODULE_NOT_FOUND",a}var p=n[i]={exports:{}};e[i][0].call(p.exports,function(r){var n=e[i][1][r];return o(n||r)},p,p.exports,r,e,n,t)}return n[i].exports}for(var u="function"==typeof require&&require,i=0;i<t.length;i++)o(t[i]);return o}return r})()({"./html/html-report-assets.js":[function(require,module,exports){
/**
 * @license Copyright 2019 Google Inc. All Rights Reserved.
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 */
'use strict';

/**
 * @fileoverview Instead of loading report assets form the filesystem, in Devtools we must load
 * them via Runtime.cachedResources. We use this module to shim
 * lighthouse-core/report/html/html-report-assets.js in Devtools.
 */

/* global Runtime */

// @ts-ignore: Runtime exists in Devtools.
const cachedResources = Runtime.cachedResources;

module.exports = {
  get REPORT_CSS() {
    return cachedResources['audits/lighthouse/report.css'];
  },
  get REPORT_JAVASCRIPT() {
    return cachedResources['audits/lighthouse/report.js'];
  },
  get REPORT_TEMPLATE() {
    return cachedResources['audits/lighthouse/template.html'];
  },
  get REPORT_TEMPLATES() {
    return cachedResources['audits/lighthouse/templates.html'];
  },
};

},{}],1:[function(require,module,exports){
/**
 * @license Copyright 2017 Google Inc. All Rights Reserved.
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 */
'use strict';

const htmlReportAssets = require('./html/html-report-assets.js');

class ReportGenerator {
  /**
   * Replaces all the specified strings in source without serial replacements.
   * @param {string} source
   * @param {!Array<{search: string, replacement: string}>} replacements
   * @return {string}
   */
  static replaceStrings(source, replacements) {
    if (replacements.length === 0) {
      return source;
    }

    const firstReplacement = replacements[0];
    const nextReplacements = replacements.slice(1);
    return source
        .split(firstReplacement.search)
        .map(part => ReportGenerator.replaceStrings(part, nextReplacements))
        .join(firstReplacement.replacement);
  }

  /**
   * Returns the report HTML as a string with the report JSON and renderer JS inlined.
   * @param {LH.Result} lhr
   * @return {string}
   */
  static generateReportHtml(lhr) {
    const sanitizedJson = JSON.stringify(lhr)
      .replace(/</g, '\\u003c') // replaces opening script tags
      .replace(/\u2028/g, '\\u2028') // replaces line separators ()
      .replace(/\u2029/g, '\\u2029'); // replaces paragraph separators
    const sanitizedJavascript = htmlReportAssets.REPORT_JAVASCRIPT.replace(/<\//g, '\\u003c/');

    return ReportGenerator.replaceStrings(htmlReportAssets.REPORT_TEMPLATE, [
      {search: '%%LIGHTHOUSE_JSON%%', replacement: sanitizedJson},
      {search: '%%LIGHTHOUSE_JAVASCRIPT%%', replacement: sanitizedJavascript},
      {search: '/*%%LIGHTHOUSE_CSS%%*/', replacement: htmlReportAssets.REPORT_CSS},
      {search: '%%LIGHTHOUSE_TEMPLATES%%', replacement: htmlReportAssets.REPORT_TEMPLATES},
    ]);
  }

  /**
   * Converts the results to a CSV formatted string
   * Each row describes the result of 1 audit with
   *  - the name of the category the audit belongs to
   *  - the name of the audit
   *  - a description of the audit
   *  - the score type that is used for the audit
   *  - the score value of the audit
   *
   * @param {LH.Result} lhr
   * @return {string}
   */
  static generateReportCSV(lhr) {
    // To keep things "official" we follow the CSV specification (RFC4180)
    // The document describes how to deal with escaping commas and quotes etc.
    const CRLF = '\r\n';
    const separator = ',';
    /** @param {string} value @return {string} */
    const escape = value => `"${value.replace(/"/g, '""')}"`;

    // Possible TODO: tightly couple headers and row values
    const header = ['category', 'name', 'title', 'type', 'score'];
    const table = Object.values(lhr.categories).map(category => {
      return category.auditRefs.map(auditRef => {
        const audit = lhr.audits[auditRef.id];
        // CSV validator wants all scores to be numeric, use -1 for now
        const numericScore = audit.score === null ? -1 : audit.score;
        return [category.title, audit.id, audit.title, audit.scoreDisplayMode, numericScore]
          .map(value => value.toString())
          .map(escape);
      });
    });

    return [header].concat(...table)
      .map(row => row.join(separator)).join(CRLF);
  }

  /**
   * Creates the results output in a format based on the `mode`.
   * @param {LH.Result} lhr
   * @param {LH.Config.Settings['output']} outputModes
   * @return {string|string[]}
   */
  static generateReport(lhr, outputModes) {
    const outputAsArray = Array.isArray(outputModes);
    if (typeof outputModes === 'string') outputModes = [outputModes];

    const output = outputModes.map(outputMode => {
      // HTML report.
      if (outputMode === 'html') {
        return ReportGenerator.generateReportHtml(lhr);
      }
      // CSV report.
      if (outputMode === 'csv') {
        return ReportGenerator.generateReportCSV(lhr);
      }
      // JSON report.
      if (outputMode === 'json') {
        return JSON.stringify(lhr, null, 2);
      }

      throw new Error('Invalid output mode: ' + outputMode);
    });

    return outputAsArray ? output : output[0];
  }
}

module.exports = ReportGenerator;

},{"./html/html-report-assets.js":"./html/html-report-assets.js"}]},{},[1])(1)
});

@@ -21,7 +21,7 @@ else
frontend_dir="$chromium_dir/third_party/blink/renderer/devtools/front_end"
fi

tests_dir="$frontend_dir/../../../web_tests/http/tests/devtools/audits2"
tests_dir="$frontend_dir/../../../web_tests/http/tests/devtools/audits"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this only affects the test fixture/expectation updates though right?

clients/devtools-report-assets.js Outdated Show resolved Hide resolved
Co-Authored-By: Paul Irish <paulirish@google.com>
@connorjclark
Copy link
Collaborator Author

Holding a bit - tomorrow me and Erik will explore if we can find an alternative solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants