[flutter_tools] support --split-debug-info option in android builds #49650

jonahwilliams · 2020-01-28T23:50:56Z

Work towards #48726

Allows further size reduction in AOT snapshots for android builds. Adds an additional argument '--save-debugging-info'. This is expected to point at a directory where the tool may write one or more debug files created during the build process. The act of specifying this argument enables dwarf stack traces.

I opted for this design over a general optimization level, because we need a user provided location for debug files. Placing them in build output folders isn't safe, since we have such a high usage of flutter clean.

This doesn't fully wire up support for iOS, since we still need to resolve why we're using xcode to strip symbols instead of gen_snapshot (see #49628 )

Example Workflow:

Developer is ready to release an APK for distribution on the Play Store.

They create a directory for the specific version of the app to be released. Example: symbols/v1.2.3-final-final
They run flutter build apk --release --split-debug-info=symbols/v1.2.3-final-final/
They otherwise upload the APK as normal. The symbol files are either stored locally, or commited to some internal symbol database.
They receive a crash report from version v1.2.3-final-final on an arm64 ABI.
They retrieve the corresponding symbols file generated in step two.
They run flutter symbolize --symbols=v1.2.3-final-final/app.android-arm64.symbols < stack.txt

What if the symbols file is lost or destroyed? In theory it could be regenerated by being on the exact same version of flutter with the exact same dart code

… tool

jonahwilliams · 2020-01-29T17:33:30Z

The actual "flutter symbolicate" command is a work in progress here: #49465

jonahwilliams · 2020-01-29T17:39:41Z

Adding @zanderso for initial thoughts

zanderso

Could you write here or in the symbolize PR a summary of what the commands in a typical workflow would look like for this?

zanderso · 2020-01-29T18:26:27Z

packages/flutter_tools/lib/src/base/build.dart

+    final String archName = darwinArch == null
+      ? getNameForTargetPlatform(platform)
+      : getNameForDarwinArch(darwinArch);
+    final String debugFilename = 'app.$archName.debug';


These files should be added to the .gitignore template

zanderso · 2020-01-29T18:28:08Z

packages/flutter_tools/lib/src/runner/flutter_command.dart

@@ -106,6 +106,7 @@ class FlutterOptions {
  static const String kEnableExperiment = 'enable-experiment';
  static const String kFileSystemRoot = 'filesystem-root';
  static const String kFileSystemScheme = 'filesystem-scheme';
+  static const String kDebuggingInfoOption = 'save-debugging-info';


This flag not only causes the debugging info to be saved off to the side, but also causes the VM to print debugging info in a different format, so I think we should consider a more descriptive flag name. Maybe --compress-debug-info, or --split-debug-info, or --separate-debug-info.

I like split because it is easier to spell than ~~separerate~~ separate

zanderso · 2020-01-29T18:36:55Z

packages/flutter_tools/lib/src/runner/flutter_command.dart

@@ -363,6 +364,12 @@ abstract class FlutterCommand extends Command<void> {
      negatable: false,
      hide: !verboseHelp,
      help: 'Build a JIT release version of your app${defaultToRelease ? ' (default mode)' : ''}.');
+    argParser.addOption(FlutterOptions.kDebuggingInfoOption,
+      help: 'Save a debug file used to symbolicate crashes from release mode applications. When provided '


Indicate somehow that this is a path to a directory.

jonahwilliams · 2020-01-29T18:40:22Z

Along with the workflow, we'll probably need some official docs on flutter.dev and possibly more

zanderso · 2020-01-31T15:38:34Z

packages/flutter_tools/templates/app/.gitignore.tmpl

@@ -33,5 +33,17 @@
 # Web related
 lib/generated_plugin_registrant.dart

+# Symbolication related
+app.android-arm.debug


Would "app.-.debug" work so that we don't have to update this?

zanderso · 2020-01-31T15:41:38Z

packages/flutter_tools/lib/src/base/build.dart

+    final String archName = darwinArch == null || platform == TargetPlatform.darwin_x64
+      ? getNameForTargetPlatform(platform)
+      : 'ios-${getNameForDarwinArch(darwinArch)}';
+    final String debugFilename = 'app.$archName.debug';


What if an app with different entrypoints is built out of the same project? Maybe these should be named like "$entrypoint-symbols.$archName.debug".

Also wdyt about an extension of "symbols" instead of "debug", so it would be entrypoint.arch.symbols instead.

I like symbols as an extension better, since it will match the symbolize command.

Beyond the entrypoint, there is the current dart+flutter version, pubspec version, in the future dart-defines. Rather than trying to capture all of it, I think we should document and explain that each file is 1-1 with a particular release.

zanderso · 2020-01-31T15:43:38Z

packages/flutter_tools/lib/src/runner/flutter_command.dart

@@ -363,6 +364,15 @@ abstract class FlutterCommand extends Command<void> {
      negatable: false,
      hide: !verboseHelp,
      help: 'Build a JIT release version of your app${defaultToRelease ? ' (default mode)' : ''}.');
+    argParser.addOption(FlutterOptions.kSplitDebugInfoOption,
+      help: 'A directory where a debug files for symbolication can be safely stored. '


Alternate wordsmithing:

In a release build, this flag reduces application size by storing Dart program symbols in a separate file on the host rather than in the application. The value of the flag should be a directory where program symbol files can be stored for later use. These symbol files contain the information needed to symbolize Dart stack traces. For an app built with this flag, the 'flutter symbolize' command with the right program symbol file are required to obtain a human readable stack trace.

The length of our explanations here suggests a website article is in order too

zanderso · 2020-01-31T16:10:36Z

packages/flutter_tools/templates/module/common/.gitignore.tmpl

+app.fuchsia-arm64.debug
+app.fuchsia-x64.debug
+app.ios-armv7.debug
+app.ios-arm64.debug


missing newline

…e_information_ios

jonahwilliams · 2020-02-03T23:32:04Z

This is ready for another review pass - TBD if it should actually be landed before the flutter symbolize command is ready though

zanderso · 2020-02-04T15:48:19Z

packages/flutter_tools/lib/src/base/build.dart

+    // the architecture, since a single build command may produce
+    // multiple debug files.
+    final String archName = darwinArch == null || platform == TargetPlatform.darwin_x64
+      ? getNameForTargetPlatform(platform)


Consider modifying getNameForTargetPlatform() to take an optional named DarwinArch parameter.

This logic could also use a test since this file will be the glue between this flag and flutter symbolize

Added here, plus a simple unit test.

The gen_snapshot tests also assert the addition of the correct string arg

zanderso · 2020-02-04T15:50:50Z

packages/flutter_tools/lib/src/build_info.dart

@@ -55,6 +56,11 @@ class BuildInfo {
  /// On Xcode builds it is used as CFBundleShortVersionString,
  final String buildName;

+  /// A optional directory path to save debugging information from dwarf stack


A optional -> An optional

zanderso · 2020-02-04T15:57:22Z

packages/flutter_tools/lib/src/runner/flutter_command.dart

@@ -363,6 +364,16 @@ abstract class FlutterCommand extends Command<void> {
      negatable: false,
      hide: !verboseHelp,
      help: 'Build a JIT release version of your app${defaultToRelease ? ' (default mode)' : ''}.');
+    argParser.addOption(FlutterOptions.kSplitDebugInfoOption,


We should probably reserve addBuildModeFlags() for the top-level build modes that map to the fields of BuildMode. This flag can go in its own add...Flag() method, or another existing one if there's one that's appropriate.

zanderso · 2020-02-04T16:01:02Z

packages/flutter_tools/lib/src/runner/flutter_command.dart

+        'symbol files can be stored for later use. These symbol files contain '
+        'the information needed to symbolize Dart stack traces. For an app built '
+        'with this flag, the \'flutter symbolize\' command with the right program '
+        'symbol file is required to obtain a human readable stack trace',


I think we can land this before flutter symbolize has landed if the github issue for flutter symbolize is noted here.

…e_information_ios

zanderso

lgtm w/ small suggestion

zanderso · 2020-02-05T21:00:36Z

packages/flutter_tools/lib/src/build_info.dart

@@ -390,7 +396,10 @@ DarwinArch getIOSArchForName(String arch) {
  return null;
 }

-String getNameForTargetPlatform(TargetPlatform platform) {
+String getNameForTargetPlatform(TargetPlatform platform, {DarwinArch darwinArch}) {
+  if (darwinArch != null) {


I'd move this to a nested switch under the TargetPlatform.ios case. I think that's the only case where it means anything.

[flutter_tools] initial support for dwarf stack traces in the flutter…

f649da9

… tool

fluttergithubbot added tool Affects the "flutter" command-line tool. See also t: labels. work in progress; do not review labels Jan 28, 2020

googlebot added the cla: yes label Jan 28, 2020

jonahwilliams added 2 commits January 29, 2020 09:23

plumb through define for android

ced6db1

update flutter.gradle

ee07c01

jonahwilliams changed the title ~~[flutter_tools] initial support for dwarf stack traces in the flutter…~~ [flutter_tools] support --save-debugging-info flag in android builds Jan 29, 2020

jonahwilliams requested a review from zanderso January 29, 2020 17:39

jonahwilliams added this to Awaiting triage in Tools - Dart and pub review via automation Jan 29, 2020

jonahwilliams moved this from Awaiting triage to Engineer reviewed in Tools - Dart and pub review Jan 29, 2020

zanderso reviewed Jan 29, 2020

View reviewed changes

update to split-debug-info, update documentation

b977d7d

zanderso reviewed Jan 31, 2020

View reviewed changes

jonahwilliams added 2 commits January 31, 2020 14:22

Merge branch 'master' of github.com:flutter/flutter into remove_sourc…

d55de32

…e_information_ios

address comments and update test cases

5a0493f

jonahwilliams changed the title ~~[flutter_tools] support --save-debugging-info flag in android builds~~ [flutter_tools] support --split-debug-info option in android builds Feb 1, 2020

jonahwilliams added 2 commits January 31, 2020 16:15

Update build_apk_test.dart

44667a4

Update build_apk_test.dart

8013f47

jonahwilliams marked this pull request as ready for review February 3, 2020 16:52

jonahwilliams removed the work in progress; do not review label Feb 3, 2020

jonahwilliams requested a review from zanderso February 3, 2020 23:31

zanderso reviewed Feb 4, 2020

View reviewed changes

jonahwilliams added 3 commits February 5, 2020 10:52

Merge branch 'master' of github.com:flutter/flutter into remove_sourc…

94d9bb6

…e_information_ios

Address comments

2e6c150

accept suggestion

93544ba

jonahwilliams requested a review from zanderso February 5, 2020 20:11

Update flutter.gradle

b85b144

zanderso approved these changes Feb 5, 2020

View reviewed changes

jonahwilliams added 3 commits February 5, 2020 13:03

Update macos_assemble.sh

8367f0f

Update build_info.dart

e11a8ce

Merge branch 'master' into remove_source_information_ios

c70088b

jonahwilliams merged commit da4b5d6 into flutter:master Feb 6, 2020

jonahwilliams deleted the remove_source_information_ios branch February 6, 2020 01:45

github-actions bot locked as resolved and limited conversation to collaborators Aug 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[flutter_tools] support --split-debug-info option in android builds #49650

[flutter_tools] support --split-debug-info option in android builds #49650

[flutter_tools] support --split-debug-info option in android builds #49650

[flutter_tools] support --split-debug-info option in android builds #49650

Conversation

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment