Switch tools/cygprofile to PyLint 2.7
This change makes tools/cygprofile/PRESUBMIT.py run PyLint 2.7 instead
of the default, PyLint 1.5. It also fixes any issues found by PyLint
2.7.
This change also makes run_tests run on Python 3, and fixes any issues
resulting from the transition.
Most of the changes are straightforward fixes recommended by PyLint,
but there are a few that warrant explanation:
* orderfile_generator_backend.py:758: _CLOUD_STORAGE_BUCKET_FOR_DEBUG
did not exist, and another class in the same file set its same-named
member to None, so I couldn't find a reasonable value for it. I
therefore removed it from the `upload_location` path altogether.
* orderfile_generator_backend.py:971: ClankCompiler had no
`manual_libname` member. As far as I can tell, it's only a member of
`options` (which is created by parsing the script's args). Because
of that, I assumed this was meant to be retrieved from `options`.
* process_profiles.py:641: the function SortByTimestamp() did not
exist, but __init() seems to sort by timestamp already, so I removed
the line altogether.
The full list of errors found by PyLint 2.7 were as follows:
************* Module test_utils
test_utils.py:20:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
test_utils.py:26:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
************* Module cygprofile_utils
cygprofile_utils.py:11:0: R0205: Class 'WarningCollector' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
************* Module check_orderfile
check_orderfile.py:57:0: R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
************* Module cluster
cluster.py:21:0: R0205: Class 'Clustering' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cluster.py:72:2: R0205: Class '_Cluster' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cluster.py:348:0: R1721: Unnecessary use of a comprehension (unnecessary-comprehension)
cluster.py:393:0: R1721: Unnecessary use of a comprehension (unnecessary-comprehension)
cluster.py:413:2: R1705: Unnecessary "else" after "return" (no-else-return)
************* Module compare_orderfiles
compare_orderfiles.py:45:13: R1714: Consider merging these comparisons with "in" to "symbol_name not in ('*', '.text')" (consider-using-in)
************* Module process_profiles
process_profiles.py:27:2: R1705: Unnecessary "else" after "return" (no-else-return)
process_profiles.py:34:0: R0205: Class 'SymbolOffsetProcessor' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:327:0: R0205: Class 'ProfileManager' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:366:2: R0205: Class 'AnnotatedOffset' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:393:31: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
process_profiles.py:396:31: C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
process_profiles.py:404:2: R0205: Class '_RunGroup' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
process_profiles.py:641:2: E1101: Instance of 'ProfileManager' has no 'SortByTimestamp' member (no-member)
process_profiles.py:651:20: E1101: Instance of 'SymbolOffsetProcessor' has no 'GetReachedOffsetsFromDumps' member; maybe 'GetReachedOffsetsFromDump'? (no-member)
************* Module cyglog_to_orderfile_unittest
cyglog_to_orderfile_unittest.py:34:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
************* Module profile_android_startup
profile_android_startup.py:46:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
profile_android_startup.py:79:0: R0205: Class 'WprManager' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
profile_android_startup.py:177:0: R0205: Class 'AndroidProfileTool' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
************* Module cyglog_to_orderfile
cyglog_to_orderfile.py:27:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
cyglog_to_orderfile.py:45:0: R0205: Class 'ObjectFileProcessor' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cyglog_to_orderfile.py:137:0: R0205: Class 'OffsetOrderfileGenerator' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
cyglog_to_orderfile.py:197:4: R1705: Unnecessary "elif" after "return" (no-else-return)
************* Module orderfile_generator_backend
orderfile_generator_backend.py:65:4: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)
orderfile_generator_backend.py:99:2: R1705: Unnecessary "else" after "return" (no-else-return)
orderfile_generator_backend.py:158:0: R0205: Class 'StepRecorder' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:238:0: R0205: Class 'ClankCompiler' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:354:0: R0205: Class 'OrderfileUpdater' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:455:0: R0205: Class 'OrderfileGenerator' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
orderfile_generator_backend.py:520:12: R1716: Simplify chained comparison between the operands (chained-comparison)
orderfile_generator_backend.py:759:10: E1101: Instance of 'OrderfileGenerator' has no '_CLOUD_STORAGE_BUCKET_FOR_DEBUG' member (no-member)
orderfile_generator_backend.py:971:10: E1101: Instance of 'ClankCompiler' has no 'manual_libname' member (no-member)
The errors printed by run_tests after moving to Python 3 were:
======================================================================
ERROR: testBasicParsing (symbol_extractor_unittest.TestLlvmBitcodeSymbolExtractor)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor_unittest.py", line 25, in testBasicParsing
symbol_names = symbol_extractor._SymbolInfosFromLlvmNm(lines)
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor.py", line 221, in _SymbolInfosFromLlvmNm
line = line.decode('utf-8')
AttributeError: 'str' object has no attribute 'decode'
======================================================================
ERROR: testSymbolInfosFromStream (symbol_extractor_unittest.TestSymbolInfosFromStream)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor_unittest.py", line 163, in testSymbolInfosFromStream
symbol_infos = symbol_extractor._SymbolInfosFromStream(lines)
File "/usr/local/google/home/jessemckenna/chromium/src/tools/cygprofile/symbol_extractor.py", line 153, in _SymbolInfosFromStream
symbol_info = _FromObjdumpLine(line.decode('utf-8').rstrip('\n'))
AttributeError: 'str' object has no attribute 'decode'
----------------------------------------------------------------------
This change was tested by running
`git cl presubmit --force --files "tools/cygprofile/*"` on Linux,
which now completes without errors.
Bug: 1333589
Change-Id: I29764c8ed0d680795d0a557b2ba2ec09adf65cd8
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3703235
Reviewed-by: Benoit Lize <lizeb@chromium.org>
Commit-Queue: Jesse McKenna <jessemckenna@google.com>
Cr-Commit-Position: refs/heads/main@{#1015417}
diff --git a/tools/cygprofile/orderfile_generator_backend.py b/tools/cygprofile/orderfile_generator_backend.py
index f03621e..8cd367d4 100755
--- a/tools/cygprofile/orderfile_generator_backend.py
+++ b/tools/cygprofile/orderfile_generator_backend.py
@@ -62,7 +62,7 @@
"""Indicates that a dispatched shell command exited with a non-zero status."""
def __init__(self, value):
- super(CommandError, self).__init__()
+ super().__init__()
self.value = value
def __str__(self):
@@ -98,8 +98,7 @@
file_name_parts = os.path.basename(file_name).split('.')
if len(file_name_parts) > 1:
return file_name_parts[-1]
- else:
- return None
+ return None
def _StashOutputDirectory(buildpath):
@@ -155,7 +154,7 @@
shutil.move(stashpath, buildpath)
-class StepRecorder(object):
+class StepRecorder:
"""Records steps and timings."""
def __init__(self, buildbot):
@@ -235,7 +234,7 @@
return process.returncode
-class ClankCompiler(object):
+class ClankCompiler:
"""Handles compilation of clank."""
def __init__(self, out_dir, step_recorder, arch, use_goma, goma_dir,
@@ -351,7 +350,7 @@
self.Build(instrumented, use_call_graph, self._libchrome_target)
-class OrderfileUpdater(object):
+class OrderfileUpdater:
"""Handles uploading and committing a new orderfile in the repository.
Only used for testing or on a bot.
@@ -452,7 +451,7 @@
raise NotImplementedError
-class OrderfileGenerator(object):
+class OrderfileGenerator:
"""A utility for generating a new orderfile for Clank.
Builds an instrumented binary, profiles a run of the application, and
@@ -466,6 +465,8 @@
# Previous orderfile_generator debug files would be overwritten.
_DIRECTORY_FOR_DEBUG_FILES = '/tmp/orderfile_generator_debug_files'
+ _CLOUD_STORAGE_BUCKET_FOR_DEBUG = None
+
def _PrepareOrderfilePaths(self):
if self._options.public:
self._clank_dir = os.path.join(constants.DIR_SOURCE_ROOT,
@@ -517,8 +518,8 @@
self._monochrome = False
for device in devices:
device_version = device.build_version_sdk
- if (device_version >= version_codes.KITKAT
- and device_version <= version_codes.LOLLIPOP_MR1):
+ if (version_codes.KITKAT <= device_version <=
+ version_codes.LOLLIPOP_MR1):
return device
assert not self._options.use_legacy_chrome_apk, \
@@ -968,7 +969,7 @@
with open(self._options.manual_symbol_offsets) as f:
symbol_offsets = [int(x) for x in f]
processor = process_profiles.SymbolOffsetProcessor(
- self._compiler.manual_libname)
+ self._options.manual_libname)
generator = cyglog_to_orderfile.OffsetOrderfileGenerator(
processor, cyglog_to_orderfile.ObjectFileProcessor(
self._options.manual_objdir))