[go: nahoru, domu]

Revert "Reland "Reland "Reland "Add toolchains without PartitionAlloc-Everywhere for dump_syms et al""""

This reverts commit 4aceb04adee20a09c1e31760e8c6dafac2563d31.

Reason for revert: This appears to be causing build files to
fail to generate on "Centipede High End Upload Linux ASan"
giving the below error:

"""
ERROR Input to target not generated by a dependency.
The file:
  //out/Release/clang_x64_with_system_allocator/minidump_fuzzer
is listed as an input or source for the target:
  //third_party/breakpad:minidump_fuzzer
but no targets in the build generate that file.
"""

Original change's description:
> Reland "Reland "Reland "Add toolchains without PartitionAlloc-Everywhere for dump_syms et al"""
>
> This is a reland of commit 5e6def5bd2aa9cdbf69608a8edc32b5de696c091
> Which was a reland of commit 818e126f4350095dd0a54566ded107f0a5065f6f
> Which was a reland of commit 38c00784bc98a5dc885b06f5a0e738386c5f7df7
>
> When PartitionAlloc is linked into an executable, it takes over the
> system allocator (malloc, new, etc), which is called PartitionAlloc-
> Everywhere (or PA-E). When this occurs in dump_syms, we see that PA
> hits OOM and causes dump_syms to crash while generating the symbols
> for chrome. It's not at all clear why PA hits OOM but the system
> allocator does not, it occurs during construction of a std::string (on
> my machine anyway when I am running it in gdb, maybe elsewhere on bots).
> This happens on all platforms that we run dump_syms on as part of the
> official build: on linux and on mac, building for at least linux,
> android, chromeos, and mac. See also crbug.com/345514993.
>
> So we want to build dump_syms and other breakpad executables in a way
> that uses the system allocator. To do that we need to disable the
> use_partition_alloc_as_malloc GN variable. As this variable is global,
> we need a separate toolchain in order to disable it.
>
> We introduce a new toolchain with the suffix `_with_system_allocator`
> that can be used for this purpose. Initially we intended to use the
> Rust host build tools toolchain for this purpose, however we require
> careful naming to avoid toolchain collisions. For instance if building
> on a Linux x64 machine with an Other x64 target, we can have two
> toolchains:
> - default_toolchain: //build/toolchain/other:clang_x64
> - host_toolchain: //build/toolchain/other:clang_x64
>
> While these have different labels, it is the name at the end that is
> used as their output directory (this is hardcoded in GN). But they
> avoid colliding because the default toolchain is not placed in a
> subdirectory and uses the `root_build_dir`. However when we add another
> toolchain with them, they both get subdirs and collide:
> - for target: //build/toolchain/other:clang_x64_with_system_allocator
> - for host: //build/toolchain/linux:clang_x64_with_system_allocator
>
> Now both toolchains try to write to the clang_x64_with_system_allocator
> directory which causes errors. To avoid this, we actually make two
> toolchains per toolchain, one with a `host_` tag inside it.
> - target:  //build/toolchain/other:clang_x64_with_system_allocator
> - unused:  //build/toolchain/linux:clang_x64_with_system_allocator
> - unused:  //build/toolchain/other:clang_x64_host_with_system_allocator
> - host:    //build/toolchain/linux:clang_x64_host_with_system_allocator
>
> Then, when building for the host we choose the `host_` variety, which
> is specified in the `host_system_allocator_toolchain variable`. And
> when building for default target, we choose the non-`host_` one, which
> is specified in the `default_system_allocator_toolchain` variable.
>
> More clever strategies that try to avoid creating the unused toolchains
> above do not seem possible. Inside the toolchain-creating template,
> it is not clear how to determine which toolchain is being created, as
> the get_label_info() function on `target_name` does not produce
> anything that matches exactly with the string in `default_toolchain` or
> `host_toolchain`. We also tried using the current_cpu and current_os,
> however the `toolchain_args.current_os` is not actually set correctly in
> the default toolchain when targeting ChromeOS. The current_os variable
> is "chromeos" but inside the toolchain_args, it is "linux". So we just
> make extra toolchains (which can't be used or they'd make build errors)
> and we don't refer to them from the `host_system_allocator_toolchain`
> and `default_system_allocator_toolchain` variables, which makes them
> effectively inaccessible.
>
> In the process we learnt many things about how the breakpad executables
> are built. When you build them for the default toolchain, such as by
> building `//third_party/breakpad:dump_syms`, it redirects to the *host*
> toolchain on many platforms, but not on all platforms. This ends up
> putting a binary that may not work on the target machine in the
> `root_build_dir` which is highly unusual, but it is required by our
> testing scripts/infra.
>
> The key insight added here is that the toolchain that it should be
> built with is the platform from where the tests on the target will be
> *launched*. On Android, iOS, and ChromeOS, the tests are launched from
> the host machine and that's where the breakpad executables are run. We
> encode this explicitly in the breakpad GN file.
>
> One additional exception is that the breakpad tools do not build on
> Windows ARM, so when building on Windows x64 for Windows ARM, while
> the tests are launched from the ARM machine, we target the host x64
> machine still. This relies on the ARM machine being able to run the
> x64 binaries through emulation.
>
> There's no change here in how the breakpad binaries are built, but it
> is now more explicitly encoded and documented. What did change is that
> since we use a separate toolchain for building these tools, we also
> turn off component build in them. This allows us to replace the use
> of symlinks with copying (or hardlinking) the binaries from the
> toolchain's root directory up to the root_build_dir. This enables
> support for building these tools in the default_toolchain on Windows,
> something which was not possible before.
>
> Additional fixes from the original CL:
>
> MSAN is disabled in the toolchain with the system allocator as we only
> support MSAN in the default toolchain. If another toolchain has MSAN
> enabled it will try to also generate the MSAN instrumented libraries
> in the default toolchain's directory and they collide. This is similar
> to the rust host build tools toolchain, but there we disable all
> sanitizers. For the system allocator toolchain, we disable MSAN but
> retain the ability to build these tools with ASAN or UBSAN if needed.
>
> Angle's GN generation is fixed by not setting the PA variable directly
> from the toolchain. We add a variable in toolchain.gni that is always
> present, and set that. Then in the PA gni files, we check for that
> variable before enabling PA-Everywhere (and BRP, etc).
>
> Devtools standalone overrides BUILDCONFIG.gn but was not re-defining
> the TESTONLY_AND_VISIBILITY variable, so this is done in
> https://chrome-internal-review.googlesource.com/c/devtools/devtools-internal/+/7412037
>
> iOS official internal builders are now using the path to the
> root_build_dir for its dump_syms exe path from
> https://chrome-internal-review.googlesource.com/c/chrome/ios_internal/+/7411376
> and it expects the executable to be for the host. A TODO is added
> in the breakpad BUILD.gn file regarding cross-compiling for a
> different mac machine architecture that will upload/launch tests to
> the iOS device.
>
> Mac and Windows internal official builders are fixed by having the
> symupload tool depended on and built for the default toolchain so that
> it's present in the root_build_dir, but making this binary always
> redirect through the host_system_allocator_toolchain. The symupload
> binary is only run on official builders, it's not part of test
> failure reporting like dump_syms.
>
> Clank orderfile generator had a GN error due to PA-E being off but
> BRP being enabled. This is resolved by the fix for Angle, by turning
> off all PA-E and BRP related stuff when the toolchain turns off PA-E.
> See https://crbug.com/347976629.
>
> Further additional fixes from the original CL:
>
> The minidump_fuzzer is added to the default toolchain, redirecting to
> the test-launcher toolchain.
>
> The windows host system-allocator toolchain is forced to use the host
> cpu, rather than using the the x86 cpu when cross-compiling.
>
> The Linux-to-Windows cross build avoids putting a `.exe` suffix on
> executables built for the host system-allocator toolchain as targets
> for Linux do not have them, and then GN can't find the requested
> target.
>
> Even more additional fixes:
>
> The previous attempt to get the host system-allocator toolchain to use
> the host cpu on Windows was incomplete. It only updated the non-clang
> toolchain, and it missed changing the environment to point to the host
> cpu's sdk. This is now done correctly.
>
> Removed the redundant output_name field in the windows symupload
> executable target, as the executable target is now named symupload, and
> not symupload_win.
>
> Explicitly add a `$host_toolchain/symupload` alias on Mac ARM so that
> the recipes which explicitly build and run that path will work when
> this lands. Once it reaches stable we can remove those explicit paths
> from the recipes.
>
> Bug: 345514993, b/342251590, 347976629, 349268750
> Change-Id: I35aca8e8d5224aa27bceebb4c815a702a2f50516
> Cq-Include-Trybots: luci.chromium.try:linux-official,android-official,win-official,mac-official
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_msan_rel_ng
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_rel_ng
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5659276
> Reviewed-by: Hans Wennborg <hans@chromium.org>
> Commit-Queue: danakj <danakj@chromium.org>
> Reviewed-by: Mark Mentovai <mark@chromium.org>
> Reviewed-by: Avi Drissman <avi@chromium.org>
> Owners-Override: danakj <danakj@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#1322382}

Bug: 345514993, b/342251590, 347976629, 349268750
Change-Id: I66b9a3fb22627681ae41893a0fc0349ea5ea1a1f
Cq-Include-Trybots: luci.chromium.try:linux-official,android-official,win-official,mac-official
Cq-Include-Trybots: luci.chromium.try:linux_chromium_msan_rel_ng
Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_rel_ng
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5673027
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Owners-Override: Michael Wilson <mjwilson@chromium.org>
Commit-Queue: Michael Wilson <mjwilson@chromium.org>
Cr-Commit-Position: refs/heads/main@{#1322410}
15 files changed
tree: a519282300ed44195a7e982233e1415cd6ce0c3a
  1. android_webview/
  2. apps/
  3. ash/
  4. base/
  5. build/
  6. build_overrides/
  7. buildtools/
  8. cc/
  9. chrome/
  10. chromecast/
  11. chromeos/
  12. codelabs/
  13. components/
  14. content/
  15. courgette/
  16. crypto/
  17. dbus/
  18. device/
  19. docs/
  20. extensions/
  21. fuchsia_web/
  22. gin/
  23. google_apis/
  24. google_update/
  25. gpu/
  26. headless/
  27. infra/
  28. ios/
  29. ipc/
  30. media/
  31. mojo/
  32. native_client_sdk/
  33. net/
  34. pdf/
  35. ppapi/
  36. printing/
  37. remoting/
  38. rlz/
  39. sandbox/
  40. services/
  41. skia/
  42. sql/
  43. storage/
  44. styleguide/
  45. testing/
  46. third_party/
  47. tools/
  48. ui/
  49. url/
  50. webkit/
  51. .clang-format
  52. .clang-tidy
  53. .clangd
  54. .eslintrc.js
  55. .git-blame-ignore-revs
  56. .gitallowed
  57. .gitattributes
  58. .gitignore
  59. .gitmodules
  60. .gn
  61. .mailmap
  62. .rustfmt.toml
  63. .vpython3
  64. .yapfignore
  65. ATL_OWNERS
  66. AUTHORS
  67. BUILD.gn
  68. CODE_OF_CONDUCT.md
  69. codereview.settings
  70. CPPLINT.cfg
  71. DEPS
  72. DIR_METADATA
  73. LICENSE
  74. LICENSE.chromium_os
  75. OWNERS
  76. PRESUBMIT.py
  77. PRESUBMIT_test.py
  78. PRESUBMIT_test_mocks.py
  79. README.md
  80. WATCHLISTS
README.md

Logo Chromium

Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all users to experience the web.

The project's web site is https://www.chromium.org.

To check out the source code locally, don't use git clone! Instead, follow the instructions on how to get the code.

Documentation in the source is rooted in docs/README.md.

Learn how to Get Around the Chromium Source Code Directory Structure.

For historical reasons, there are some small top level directories. Now the guidance is that new top level directories are for product (e.g. Chrome, Android WebView, Ash). Even if these products have multiple executables, the code should be in subdirectories of the product.

If you found a bug, please file it at https://crbug.com/new.