[go: nahoru, domu]

Revert "Reland "Add toolchains without PartitionAlloc-Everywhere for dump_syms et al""

This reverts commit 818e126f4350095dd0a54566ded107f0a5065f6f.

Reason for revert: This brakes the following official builds.
https://ci.chromium.org/ui/p/chrome/builders/official/mac64/8407/overview
https://ci.chromium.org/ui/p/chrome/builders/official/win-clang/8618/overview

`symupload` disappeared from those builds.

Repro steps:
```
./tools/mb/mb gen -m official.chrome -b mac64 --config-file ./internal/tools/mb/mb_config.pyl out/mac64-official
ninja -C out/mac64-official clang_arm64/symupload
[0/1] Regenerating ninja files
ninja: error: unknown target 'clang_arm64/symupload'
```

Original change's description:
> Reland "Add toolchains without PartitionAlloc-Everywhere for dump_syms et al"
>
> This is a reland of commit 38c00784bc98a5dc885b06f5a0e738386c5f7df7
>
> MSAN is disabled in the toolchain with the system allocator as we only
> support MSAN in the default toolchain. If another toolchain has MSAN
> enabled it will try to also generate the MSAN instrumented libraries
> in the default toolchain's directory and they collide.
>
> This is similar to the rust host build tools toolchain, but there we
> disable all sanitizers. For the system allocator toolchain, we disable
> MSAN but retain the ability to build these tools with ASAN or UBSAN if
> needed.
>
> Original change's description:
> > Add toolchains without PartitionAlloc-Everywhere for dump_syms et al
> >
> > When PartitionAlloc is linked into an executable, it takes over the
> > system allocator (malloc, new, etc), which is called PartitionAlloc-
> > Everywhere (or PA-E). When this occurs in dump_syms, we see that PA
> > hits OOM and causes dump_syms to crash while generating the symbols
> > for chrome. It's not at all clear why PA hits OOM but the system
> > allocator does not, it occurs during construction of a std::string (on
> > my machine anyway when I am running it in gdb, maybe elsewhere on bots).
> > This happens on all platforms that we run dump_syms on as part of the
> > official build: on linux and on mac, building for at least linux,
> > android, chromeos, and mac. See also crbug.com/345514993.
> >
> > So we want to build dump_syms and other breakpad executables in a way
> > that uses the system allocator. To do that we need to disable the
> > use_partition_alloc_as_malloc GN variable. As this variable is global,
> > we need a separate toolchain in order to disable it.
> >
> > We introduce a new toolchain with the suffix `_with_system_allocator`
> > that can be used for this purpose. Initially we intended to use the
> > Rust host build tools toolchain for this purpose, however we require
> > careful naming to avoid toolchain collisions. For instance if building
> > on a Linux x64 machine with an Other x64 target, we can have two
> > toolchains:
> > - default_toolchain: //build/toolchain/other:clang_x64
> > - host_toolchain: //build/toolchain/other:clang_x64
> >
> > While these have different labels, it is the name at the end that is
> > used as their output directory (this is hardcoded in GN). But they
> > avoid colliding because the default toolchain is not placed in a
> > subdirectory and uses the `root_build_dir`. However when we add another
> > toolchain with them, they both get subdirs and collide:
> > - for target: //build/toolchain/other:clang_x64_with_system_allocator
> > - for host: //build/toolchain/linux:clang_x64_with_system_allocator
> >
> > Now both toolchains try to write to the clang_x64_with_system_allocator
> > directory which causes errors. To avoid this, we actually make two
> > toolchains per toolchain, one with a `host_` tag inside it.
> > - target:  //build/toolchain/other:clang_x64_with_system_allocator
> > - unused:  //build/toolchain/linux:clang_x64_with_system_allocator
> > - unused:  //build/toolchain/other:clang_x64_host_with_system_allocator
> > - host:    //build/toolchain/linux:clang_x64_host_with_system_allocator
> >
> > Then, when building for the host we choose the `host_` variety, which
> > is specified in the `host_system_allocator_toolchain variable`. And
> > when building for default target, we choose the non-`host_` one, which
> > is specified in the `default_system_allocator_toolchain` variable.
> >
> > More clever strategies that try to avoid creating the unused toolchains
> > above do not seem possible. Inside the toolchain-creating template,
> > it is not clear how to determine which toolchain is being created, as
> > the get_label_info() function on `target_name` does not produce
> > anything that matches exactly with the string in `default_toolchain` or
> > `host_toolchain`. We also tried using the current_cpu and current_os,
> > however the `toolchain_args.current_os` is not actually set correctly in
> > the default toolchain when targeting ChromeOS. The current_os variable
> > is "chromeos" but inside the toolchain_args, it is "linux". So we just
> > make extra toolchains (which can't be used or they'd make build errors)
> > and we don't refer to them from the `host_system_allocator_toolchain`
> > and `default_system_allocator_toolchain` variables, which makes them
> > effectively inaccessible.
> >
> > In the process we learnt many things about how the breakpad executables
> > are built. When you build them for the default toolchain, such as by
> > building `//third_party/breakpad:dump_syms`, it redirects to the *host*
> > toolchain on many platforms, but not on all platforms. This ends up
> > putting a binary that may not work on the target machine in the
> > `root_build_dir` which is highly unusual, but it is required by our
> > testing scripts/infra.
> >
> > The key insight added here is that the toolchain that it should be
> > built with is the platform from where the tests on the target will be
> > *launched*. On Android, iOS, and ChromeOS, the tests are launched from
> > the host machine and that's where the breakpad executables are run. We
> > encode this explicitly in the breakpad GN file.
> >
> > One additional exception is that the breakpad tools do not build on
> > Windows ARM, so when building on Windows x64 for Windows ARM, while
> > the tests are launched from the ARM machine, we target the host x64
> > machine still. This relies on the ARM machine being able to run the
> > x64 binaries through emulation.
> >
> > There's no change here in how the breakpad binaries are built, but it
> > is now more explicitly encoded and documented. What did change is that
> > since we use a separate toolchain for building these tools, we also
> > turn off component build in them. This allows us to replace the use
> > of symlinks with copying (or hardlinking) the binaries from the
> > toolchain's root directory up to the root_build_dir. This enables
> > support for building these tools in the default_toolchain on Windows,
> > something which was not possible before.
> >
> > Bug: 345514993, b/342251590
> > Change-Id: Ic2d1d356452791b8916d6876f3d8d1855a6566b6
> > Cq-Include-Trybots: luci.chromium.try:linux-official,android-official,win-official,mac-official
> > Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5609092
> > Reviewed-by: Achuith Bhandarkar <achuith@chromium.org>
> > Reviewed-by: Nico Weber <thakis@chromium.org>
> > Owners-Override: danakj <danakj@chromium.org>
> > Reviewed-by: Avi Drissman <avi@chromium.org>
> > Commit-Queue: danakj <danakj@chromium.org>
> > Reviewed-by: Mark Mentovai <mark@chromium.org>
> > Cr-Commit-Position: refs/heads/main@{#1316523}
>
> Bug: 345514993, b/342251590
> Change-Id: Ieef03911ad484b234845a6df3f47c4f77136ca6a
> Cq-Include-Trybots: luci.chromium.try:linux-official,android-official,win-official,mac-official
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_msan_rel_ng
> Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_rel_ng
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5639356
> Owners-Override: danakj <danakj@chromium.org>
> Reviewed-by: Nico Weber <thakis@chromium.org>
> Commit-Queue: danakj <danakj@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#1316723}

Bug: 345514993, b/342251590
Change-Id: Id1b999a68902755c06f908a3ab6777720a4dd27e
Cq-Include-Trybots: luci.chromium.try:linux-official,android-official,win-official,mac-official
Cq-Include-Trybots: luci.chromium.try:linux_chromium_msan_rel_ng
Cq-Include-Trybots: luci.chromium.try:linux_chromium_asan_rel_ng
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5641925
Commit-Queue: Junji Watanabe <jwata@google.com>
Owners-Override: Prudhvikumar Bommana <pbommana@google.com>
Commit-Queue: Prudhvikumar Bommana <pbommana@google.com>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#1317267}
15 files changed
tree: 9175a8444303c0930359993ddac787d62f6e992a
  1. android_webview/
  2. apps/
  3. ash/
  4. base/
  5. build/
  6. build_overrides/
  7. buildtools/
  8. cc/
  9. chrome/
  10. chromecast/
  11. chromeos/
  12. codelabs/
  13. components/
  14. content/
  15. courgette/
  16. crypto/
  17. dbus/
  18. device/
  19. docs/
  20. extensions/
  21. fuchsia_web/
  22. gin/
  23. google_apis/
  24. google_update/
  25. gpu/
  26. headless/
  27. infra/
  28. ios/
  29. ipc/
  30. media/
  31. mojo/
  32. native_client_sdk/
  33. net/
  34. pdf/
  35. ppapi/
  36. printing/
  37. remoting/
  38. rlz/
  39. sandbox/
  40. services/
  41. skia/
  42. sql/
  43. storage/
  44. styleguide/
  45. testing/
  46. third_party/
  47. tools/
  48. ui/
  49. url/
  50. webkit/
  51. .clang-format
  52. .clang-tidy
  53. .clangd
  54. .eslintrc.js
  55. .git-blame-ignore-revs
  56. .gitallowed
  57. .gitattributes
  58. .gitignore
  59. .gitmodules
  60. .gn
  61. .mailmap
  62. .rustfmt.toml
  63. .vpython3
  64. .yapfignore
  65. ATL_OWNERS
  66. AUTHORS
  67. BUILD.gn
  68. CODE_OF_CONDUCT.md
  69. codereview.settings
  70. CPPLINT.cfg
  71. DEPS
  72. DIR_METADATA
  73. LICENSE
  74. LICENSE.chromium_os
  75. OWNERS
  76. PRESUBMIT.py
  77. PRESUBMIT_test.py
  78. PRESUBMIT_test_mocks.py
  79. README.md
  80. WATCHLISTS
README.md

Logo Chromium

Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all users to experience the web.

The project's web site is https://www.chromium.org.

To check out the source code locally, don't use git clone! Instead, follow the instructions on how to get the code.

Documentation in the source is rooted in docs/README.md.

Learn how to Get Around the Chromium Source Code Directory Structure.

For historical reasons, there are some small top level directories. Now the guidance is that new top level directories are for product (e.g. Chrome, Android WebView, Ash). Even if these products have multiple executables, the code should be in subdirectories of the product.

If you found a bug, please file it at https://crbug.com/new.