[go: nahoru, domu]

Switch tools/cygprofile to PyLint 2.7

This change makes tools/cygprofile/PRESUBMIT.py run PyLint 2.7 instead
of the default, PyLint 1.5. It also fixes any issues found by PyLint
2.7.

This change also makes run_tests run on Python 3, and fixes any issues
resulting from the transition.

Most of the changes are straightforward fixes recommended by PyLint,
but there are a few that warrant explanation:
* orderfile_generator_backend.py:758: _CLOUD_STORAGE_BUCKET_FOR_DEBUG
  did not exist, and another class in the same file set its same-named
  member to None, so I couldn't find a reasonable value for it. I
  therefore removed it from the `upload_location` path altogether.
* orderfile_generator_backend.py:971: ClankCompiler had no
  `manual_libname` member. As far as I can tell, it's only a member of
  `options` (which is created by parsing the script's args). Because
  of that, I assumed this was meant to be retrieved from `options`.
* process_profiles.py:641: the function SortByTimestamp() did not
  exist, but __init() seems to sort by timestamp already, so I removed
  the line altogether.

The full list of errors found by PyLint 2.7 were as follows:

************* Module test_utils
test_utils.py:20:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
test_utils.py:26:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
************* Module cygprofile_utils
cygprofile_utils.py:11:0: R0205: Class 'WarningCollector' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
************* Module check_orderfile
check_orderfile.py:57:0: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module cluster
cluster.py:21:0: R0205: Class 'Clustering' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cluster.py:72:2: R0205: Class '_Cluster' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cluster.py:348:0: R1721: Unnecessary use of a comprehension (unnecessary-comprehension)
cluster.py:393:0: R1721: Unnecessary use of a comprehension (unnecessary-comprehension)
cluster.py:413:2: R1705: Unnecessary "else" after "return" (no-else-return)
************* Module compare_orderfiles
compare_orderfiles.py:45:13: R1714: Consider merging these comparisons with "in" to "symbol_name not in ('*', '.text')" (consider-using-in)
************* Module process_profiles
process_profiles.py:27:2: R1705: Unnecessary "else" after "return" (no-else-return)
process_profiles.py:34:0: R0205: Class 'SymbolOffsetProcessor' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:327:0: R0205: Class 'ProfileManager' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:366:2: R0205: Class 'AnnotatedOffset' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:393:31: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
process_profiles.py:396:31: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
process_profiles.py:404:2: R0205: Class '_RunGroup' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:641:2: E1101: Instance of 'ProfileManager' has no 'SortByTimestamp' member (no-member)
process_profiles.py:651:20: E1101: Instance of 'SymbolOffsetProcessor' has no 'GetReachedOffsetsFromDumps' member; maybe 'GetReachedOffsetsFromDump'? (no-member)
************* Module cyglog_to_orderfile_unittest
cyglog_to_orderfile_unittest.py:34:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
************* Module profile_android_startup
profile_android_startup.py:46:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
profile_android_startup.py:79:0: R0205: Class 'WprManager' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
profile_android_startup.py:177:0: R0205: Class 'AndroidProfileTool' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
************* Module cyglog_to_orderfile
cyglog_to_orderfile.py:27:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
cyglog_to_orderfile.py:45:0: R0205: Class 'ObjectFileProcessor' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cyglog_to_orderfile.py:137:0: R0205: Class 'OffsetOrderfileGenerator' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cyglog_to_orderfile.py:197:4: R1705: Unnecessary "elif" after "return" (no-else-return)
************* Module orderfile_generator_backend
orderfile_generator_backend.py:65:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
orderfile_generator_backend.py:99:2: R1705: Unnecessary "else" after "return" (no-else-return)
orderfile_generator_backend.py:158:0: R0205: Class 'StepRecorder' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:238:0: R0205: Class 'ClankCompiler' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:354:0: R0205: Class 'OrderfileUpdater' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:455:0: R0205: Class 'OrderfileGenerator' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:520:12: R1716: Simplify chained comparison between the operands (chained-comparison)
orderfile_generator_backend.py:759:10: E1101: Instance of 'OrderfileGenerator' has no '_CLOUD_STORAGE_BUCKET_FOR_DEBUG' member (no-member)
orderfile_generator_backend.py:971:10: E1101: Instance of 'ClankCompiler' has no 'manual_libname' member (no-member)

The errors printed by run_tests after moving to Python 3 were:

======================================================================
ERROR: testBasicParsing (symbol_extractor_unittest.TestLlvmBitcodeSymbolExtractor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor_unittest.py", line 25, in testBasicParsing
    symbol_names = symbol_extractor._SymbolInfosFromLlvmNm(lines)
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor.py", line 221, in _SymbolInfosFromLlvmNm
    line = line.decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'

======================================================================
ERROR: testSymbolInfosFromStream (symbol_extractor_unittest.TestSymbolInfosFromStream)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor_unittest.py", line 163, in testSymbolInfosFromStream
    symbol_infos = symbol_extractor._SymbolInfosFromStream(lines)
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor.py", line 153, in _SymbolInfosFromStream
    symbol_info = _FromObjdumpLine(line.decode('utf-8').rstrip('\n'))
AttributeError: 'str' object has no attribute 'decode'

----------------------------------------------------------------------

This change was tested by running
`git cl presubmit --force --files "tools/cygprofile/*"` on Linux,
which now completes without errors.

Bug: 1333589
Change-Id: I29764c8ed0d680795d0a557b2ba2ec09adf65cd8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3703235
Reviewed-by: Benoit Lize <lizeb@chromium.org>
Commit-Queue: Jesse McKenna <jessemckenna@google.com>
Cr-Commit-Position: refs/heads/main@{#1015417}
diff --git a/tools/cygprofile/orderfile_generator_backend.py b/tools/cygprofile/orderfile_generator_backend.py
index f03621e..8cd367d4 100755
--- a/tools/cygprofile/orderfile_generator_backend.py
+++ b/tools/cygprofile/orderfile_generator_backend.py
@@ -62,7 +62,7 @@
   """Indicates that a dispatched shell command exited with a non-zero status."""
 
   def __init__(self, value):
-    super(CommandError, self).__init__()
+    super().__init__()
     self.value = value
 
   def __str__(self):
@@ -98,8 +98,7 @@
   file_name_parts = os.path.basename(file_name).split('.')
   if len(file_name_parts) > 1:
     return file_name_parts[-1]
-  else:
-    return None
+  return None
 
 
 def _StashOutputDirectory(buildpath):
@@ -155,7 +154,7 @@
   shutil.move(stashpath, buildpath)
 
 
-class StepRecorder(object):
+class StepRecorder:
   """Records steps and timings."""
 
   def __init__(self, buildbot):
@@ -235,7 +234,7 @@
     return process.returncode
 
 
-class ClankCompiler(object):
+class ClankCompiler:
   """Handles compilation of clank."""
 
   def __init__(self, out_dir, step_recorder, arch, use_goma, goma_dir,
@@ -351,7 +350,7 @@
     self.Build(instrumented, use_call_graph, self._libchrome_target)
 
 
-class OrderfileUpdater(object):
+class OrderfileUpdater:
   """Handles uploading and committing a new orderfile in the repository.
 
   Only used for testing or on a bot.
@@ -452,7 +451,7 @@
     raise NotImplementedError
 
 
-class OrderfileGenerator(object):
+class OrderfileGenerator:
   """A utility for generating a new orderfile for Clank.
 
   Builds an instrumented binary, profiles a run of the application, and
@@ -466,6 +465,8 @@
   # Previous orderfile_generator debug files would be overwritten.
   _DIRECTORY_FOR_DEBUG_FILES = '/tmp/orderfile_generator_debug_files'
 
+  _CLOUD_STORAGE_BUCKET_FOR_DEBUG = None
+
   def _PrepareOrderfilePaths(self):
     if self._options.public:
       self._clank_dir = os.path.join(constants.DIR_SOURCE_ROOT,
@@ -517,8 +518,8 @@
       self._monochrome = False
       for device in devices:
         device_version = device.build_version_sdk
-        if (device_version >= version_codes.KITKAT
-            and device_version <= version_codes.LOLLIPOP_MR1):
+        if (version_codes.KITKAT <= device_version <=
+            version_codes.LOLLIPOP_MR1):
           return device
 
     assert not self._options.use_legacy_chrome_apk, \
@@ -968,7 +969,7 @@
       with open(self._options.manual_symbol_offsets) as f:
         symbol_offsets = [int(x) for x in f]
       processor = process_profiles.SymbolOffsetProcessor(
-          self._compiler.manual_libname)
+          self._options.manual_libname)
       generator = cyglog_to_orderfile.OffsetOrderfileGenerator(
           processor, cyglog_to_orderfile.ObjectFileProcessor(
               self._options.manual_objdir))