I have tried to add -fPIC in the build flag, but it still reports the same issue. I also tried change '-O2‘ to '-Os', it is the same.
If I add '-mcmodel=large' in the build flag, it will report confliction with '-fPIC' when build python _pywrap_snapshot_utils.cc.

Hi,
I also tried the latest code, it still have the same issue.

Building for the aarch64
INFO: Reading 'startup' options from /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --windows_enable_symlinks
INFO: Options provided by the client:
Inherited 'common' options: --isatty=1 --terminal_columns=122
INFO: Reading rc options for 'build' from /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc:
Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc:
'build' options: --define framework_shared_object=true --define tsl_protobuf_header_only=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --features=-force_no_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false --incompatible_enforce_config_setting_visibility
INFO: Reading rc options for 'build' from /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.tf_configure.bazelrc:
'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3/dist-packages --python_path=/usr/bin/python3 --define=with_xla_support=false
INFO: Found applicable config definition build:short_logs in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:elinux_aarch64 in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --config=elinux --cpu=aarch64
INFO: Found applicable config definition build:elinux in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --crosstool_top=@local_config_embedded_arm//:toolchain --host_crosstool_top=@bazel_tools//tools/cpp:toolchain
INFO: Found applicable config definition build:monolithic in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --define framework_shared_object=false --define tsl_protobuf_header_only=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:linux in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wno-array-bounds --copt=-Wunused-result --copt=-Werror=unused-result --copt=-Wswitch --copt=-Werror=switch --copt=-Wno-error=unused-but-set-variable --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
INFO: Analyzed 4 targets (299 packages loaded, 18526 targets configured).
INFO: Found 4 targets...
ERROR: /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/tensorflow/BUILD:1318:21: Linking tensorflow/libtensorflow_cc.so.2.18.0 failed: (Exit 1): aarch64-none-linux-gnu-gcc failed: error executing command (from target //tensorflow:libtensorflow_cc.so.2.18.0)
(cd /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/build/execroot/org_tensorflow &&
exec env -
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/proc/self/cwd
PYTHON_BIN_PATH=/usr/bin/python3
PYTHON_LIB_PATH=/usr/lib/python3/dist-packages
TF2_BEHAVIOR=1
/home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/build/external/aarch64_linux_toolchain/bin/aarch64-none-linux-gnu-gcc @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow_cc.so.2.18.0-2.params)

Configuration: f86a22a88103843b2f048db8d142631a85f0eddbce9df51d51c468ebb4786d1f

Execution platform: @local_execution_config_platform//:platform

bazel-out/aarch64-opt/bin/tensorflow/c/experimental/gradients/libmath_grad.pic.a(math_grad.pic.o): in function tsl::core::RefCounted::Unref() const [clone .constprop.0]': math_grad.cc:(.text.unlikely._ZNK3tsl4core10RefCounted5UnrefEv.constprop.0+0x14): relocation truncated to fit: R_AARCH64_CALL26 against symbol __aarch64_ldadd8_acq_rel' defined in .text section in /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/build/external/aarch64_linux_toolchain/bin/../lib/gcc/aarch64-none-linux-gnu/11.3.1/libgcc.a(ldadd_8_4.o)
bazel-out/aarch64-opt/bin/tensorflow/compiler/mlir/lite/python/libjax_to_tfl_flatbuffer.pic.a(jax_to_tfl_flatbuffer.pic.o): in function absl::lts_20230802::log_internal::LogMessage& absl::lts_20230802::log_internal::LogMessage::operator<< <absl::lts_20230802::Status, 0>(absl::lts_20230802::Status const&) [clone .isra.0]': jax_to_tfl_flatbuffer.cc:(.text.unlikely._ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT_.isra.0+0x1c): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::OstreamView(absl::lts_20230802::log_internal::LogMessage::LogMessageData&)' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamViewC2ERNS2_14LogMessageDataE section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o)
jax_to_tfl_flatbuffer.cc:(.text.unlikely.ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT.isra.0+0x24): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::stream()' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamView6streamEv section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o) jax_to_tfl_flatbuffer.cc:(.text.unlikely._ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT_.isra.0+0x2c): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::operator<<(std::ostream&, absl::lts_20230802::Status const&)' defined in .text._ZN4absl12lts_20230802lsERSoRKNS0_6StatusE section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/status/libstatus.pic.a(status.pic.o)
jax_to_tfl_flatbuffer.cc:(.text.unlikely.ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT.isra.0+0x34): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::~OstreamView()' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamViewD2Ev section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o) jax_to_tfl_flatbuffer.cc:(.text.unlikely._ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT_.isra.0+0x50): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::~OstreamView()' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamViewD2Ev section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o)
bazel-out/aarch64-opt/bin/tensorflow/compiler/mlir/lite/libtf_to_tfl_flatbuffer.pic.a(tf_to_tfl_flatbuffer.pic.o): in function absl::lts_20230802::log_internal::LogMessage& absl::lts_20230802::log_internal::LogMessage::operator<< <absl::lts_20230802::Status, 0>(absl::lts_20230802::Status const&) [clone .isra.0]': tf_to_tfl_flatbuffer.cc:(.text.unlikely._ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT_.isra.0+0x1c): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::OstreamView(absl::lts_20230802::log_internal::LogMessage::LogMessageData&)' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamViewC2ERNS2_14LogMessageDataE section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o)
tf_to_tfl_flatbuffer.cc:(.text.unlikely.ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT.isra.0+0x24): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::log_internal::LogMessage::OstreamView::stream()' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamView6streamEv section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o) tf_to_tfl_flatbuffer.cc:(.text.unlikely._ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT_.isra.0+0x2c): relocation truncated to fit: R_AARCH64_CALL26 against symbol absl::lts_20230802::operator<<(std::ostream&, absl::lts_20230802::Status const&)' defined in .text._ZN4absl12lts_20230802lsERSoRKNS0_6StatusE section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/status/libstatus.pic.a(status.pic.o)
tf_to_tfl_flatbuffer.cc:(.text.unlikely.ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT.isra.0+0x34): relocation truncated to fit: R_AARCH64_CALL26 against symbol `absl::lts_20230802::log_internal::LogMessage::OstreamView::~OstreamView()' defined in .text._ZN4absl12lts_2023080212log_internal10LogMessage11OstreamViewD2Ev section in bazel-out/aarch64-opt/bin/external/com_google_absl/absl/log/internal/liblog_message.pic.a(log_message.pic.o)
tf_to_tfl_flatbuffer.cc:(.text.unlikely.ZN4absl12lts_2023080212log_internal10LogMessagelsINS0_6StatusELi0EEERS2_RKT.isra.0+0x50): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
ERROR: /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/tensorflow/tools/pip_package/BUILD:266:9 Action tensorflow/tools/pip_package/wheel_house failed: (Exit 1): aarch64-none-linux-gnu-gcc failed: error executing command (from target //tensorflow:libtensorflow_cc.so.2.18.0)
(cd /home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/build/execroot/org_tensorflow &&
exec env -
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/proc/self/cwd
PYTHON_BIN_PATH=/usr/bin/python3
PYTHON_LIB_PATH=/usr/lib/python3/dist-packages
TF2_BEHAVIOR=1
/home/bcmwrt/data/Tensorflow/tf_aarch64/tensorflow/build/external/aarch64_linux_toolchain/bin/aarch64-none-linux-gnu-gcc @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow_cc.so.2.18.0-2.params)

Configuration: f86a22a88103843b2f048db8d142631a85f0eddbce9df51d51c468ebb4786d1f

Execution platform: @local_execution_config_platform//:platform

INFO: Elapsed time: 758.820s, Critical Path: 625.27s
INFO: 6123 processes: 721 internal, 5402 local.
FAILED: Build did NOT complete successfully

Hi,

Add more info when I debug it. This issue ONLY happens at build the target --//tensorflow/tools/pip_package:build_pip_package
Which should be the python package.

Willen

I met with a similar problem when running bazel build //tensorflow/tools/pip_package:build_pip_package with tensorflow version r2.13, GCC 9.4.0. It is on a Jetson Orin NX Jetpack 5.1.

The bazel build always failed at Linking tensorflow/libtensorflow_cc.so.2.13.1. I believe it is because the size of this lib file is too large for a normal ARM jumping instruction to jump somewhere inside/to this lib. There was a similar issue #48919 on an x86-64 platform, resulting from a --dbg flag generating way more debugging information in the lib.

Of course it should be not so easy to shrink the lib size. I used to find some sources that mentioned the -mcmodel and -fPIC compilation flags. Maybe I am gonna report the result here after I try them.

Here is my error logs.

ERROR: /home/user/tensorflow/tensorflow/BUILD:1272:21: Linking tensorflow/libtensorflow_cc.so.2.13.1 failed: (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow_cc.so.2.13.1-2.params
/home/user/.cache/bazel/_bazel_user/b0c16afcd5d9b574b1ee2f726ab4799c/execroot/org_tensorflow/external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc:44: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
  import pipes
bazel-out/aarch64-opt/bin/tensorflow/compiler/aot/libtfcompile_lib.pic.a(compile.pic.o): in function `llvm::object_creator<std::function<absl::lts_20230125::Status (tensorflow::tf2xla::Config const&, xla::XlaComputation*)> >::call()':
compile.cc:(.text._ZN4llvm14object_creatorISt8functionIFN4absl12lts_202301256StatusERKN10tensorflow6tf2xla6ConfigEPN3xla14XlaComputationEEEE4callEv[_ZN4llvm14object_creatorISt8functionIFN4absl12lts_202301256StatusERKN10tensorflow6tf2xla6ConfigEPN3xla14XlaComputationEEEE4callEv]+0xc): relocation truncated to fit: R_AARCH64_CALL26 against symbol `operator new(unsigned long)@@GLIBCXX_3.4' defined in .text section in /usr/lib/gcc/aarch64-linux-gnu/9/libstdc++.so
bazel-out/aarch64-opt/bin/tensorflow/compiler/aot/libtfcompile_lib.pic.a(compile.pic.o): in function `tensorflow::tfcompile::InitializeTargets()':
compile.cc:(.text._ZN10tensorflow9tfcompileL17InitializeTargetsEv+0x14): relocation truncated to fit: R_AARCH64_CALL26 against symbol `LLVMInitializeAArch64AsmParser' defined in .text.LLVMInitializeAArch64AsmParser section in bazel-out/aarch64-opt/bin/external/llvm-project/llvm/libAArch64AsmParser.pic.a(AArch64AsmParser.pic.o)
compile.cc:(.text._ZN10tensorflow9tfcompileL17InitializeTargetsEv+0x34): relocation truncated to fit: R_AARCH64_CALL26 against symbol `LLVMInitializePowerPCTargetInfo' defined in .text.LLVMInitializePowerPCTargetInfo section in bazel-out/aarch64-opt/bin/external/llvm-project/llvm/libPowerPCInfo.pic.a(PowerPCTargetInfo.pic.o)
compile.cc:(.text._ZN10tensorflow9tfcompileL17InitializeTargetsEv+0x40): relocation truncated to fit: R_AARCH64_CALL26 against symbol `LLVMInitializePowerPCAsmPrinter' defined in .text.LLVMInitializePowerPCAsmPrinter section in bazel-out/aarch64-opt/bin/external/llvm-project/llvm/libPowerPCCodeGen.pic.a(PPCAsmPrinter.pic.o)
compile.cc:(.text._ZN10tensorflow9tfcompileL17InitializeTargetsEv+0x50): relocation truncated to fit: R_AARCH64_CALL26 against symbol `LLVMInitializeX86AsmParser' defined in .text.LLVMInitializeX86AsmParser section in bazel-out/aarch64-opt/bin/external/llvm-project/llvm/libX86AsmParser.pic.a(X86AsmParser.pic.o)
compile.cc:(.text._ZN10tensorflow9tfcompileL17InitializeTargetsEv+0x58): relocation truncated to fit: R_AARCH64_JUMP26 against symbol `LLVMInitializeX86AsmPrinter' defined in .text.LLVMInitializeX86AsmPrinter section in bazel-out/aarch64-opt/bin/external/llvm-project/llvm/libX86CodeGen.pic.a(X86AsmPrinter.pic.o)
bazel-out/aarch64-opt/bin/tensorflow/compiler/aot/libtfcompile_lib.pic.a(compile.pic.o): in function `tensorflow::tfcompile::ReadProtoFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message*)':
compile.cc:(.text._ZN10tensorflow9tfcompileL13ReadProtoFileERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPN6google8protobuf7MessageE+0x58): relocation truncated to fit: R_AARCH64_CALL26 against symbol `tsl::ReadBinaryProto(tsl::Env*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::MessageLite*)' defined in .text._ZN3tsl15ReadBinaryProtoEPNS_3EnvERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPN6google8protobuf11MessageLiteE section in bazel-out/aarch64-opt/bin/tensorflow/tsl/platform/default/libenv.pic.a(env.pic.o)
compile.cc:(.text._ZN10tensorflow9tfcompileL13ReadProtoFileERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPN6google8protobuf7MessageE+0x90): relocation truncated to fit: R_AARCH64_CALL26 against symbol `tsl::ReadTextProto(tsl::Env*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message*)' defined in .text._ZN3tsl13ReadTextProtoEPNS_3EnvERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPN6google8protobuf7MessageE section in bazel-out/aarch64-opt/bin/tensorflow/tsl/platform/default/libenv.pic.a(env.pic.o)
bazel-out/aarch64-opt/bin/tensorflow/compiler/aot/libtfcompile_lib.pic.a(compile.pic.o): in function `tensorflow::tfcompile::RegisterQuantizeFn(std::function<absl::lts_20230125::Status (tensorflow::tf2xla::Config const&, xla::XlaComputation*)> const&)':
compile.cc:(.text._ZN10tensorflow9tfcompile18RegisterQuantizeFnERKSt8functionIFN4absl12lts_202301256StatusERKNS_6tf2xla6ConfigEPN3xla14XlaComputationEEE+0x140): relocation truncated to fit: R_AARCH64_CALL26 against symbol `__stack_chk_fail@@GLIBC_2.17' defined in .text section in /lib/aarch64-linux-gnu/libc.so.6
bazel-out/aarch64-opt/bin/tensorflow/compiler/aot/libtfcompile_lib.pic.a(compile.pic.o): in function `absl::lts_20230125::Status tsl::errors::Unknown<char const*, std::basic_string_view<char, std::char_traits<char> > >(char const*, std::basic_string_view<char, std::char_traits<char> >)':
compile.cc:(.text._ZN3tsl6errors7UnknownIJPKcSt17basic_string_viewIcSt11char_traitsIcEEEEEN4absl12lts_202301256StatusEDpT_[_ZN3tsl6errors7UnknownIJPKcSt17basic_string_viewIcSt11char_traitsIcEEEEEN4absl12lts_202301256StatusEDpT_]+0x3c): relocation truncated to fit: R_AARCH64_CALL26 against symbol `strlen@@GLIBC_2.17' defined in .text section in /lib/aarch64-linux-gnu/libc.so.6
compile.cc:(.text._ZN3tsl6errors7UnknownIJPKcSt17basic_string_viewIcSt11char_traitsIcEEEEEN4absl12lts_202301256StatusEDpT_[_ZN3tsl6errors7UnknownIJPKcSt17basic_string_viewIcSt11char_traitsIcEEEEEN4absl12lts_202301256StatusEDpT_]+0x7c): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
Target //tensorflow/tools/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.