bazel-bootstrap fails to build: chokes on -Wl,--package-metadata in LDFLAGS

[bdrung]

Description of the bug:

bazel-bootstrap 4.2.3+ds-11.1build1 fails to build from source on Ubuntu resolute:

Traceback (most recent call last):
 File "/tmp/bazel_xLYnPoI6/out/external/local_config_cc/cc_toolchain_config.bzl", line 1229, column 53, in _impl
  return cc_common.create_cc_toolchain_config_info(
Error in create_cc_toolchain_config_info: Invalid toolchain configuration: expected '{' at position 24 while parsing a flag containing '-Wl,--package-metadata=%7B%22type%22'

It looks like that the build chokes on the colon in the -Wl,--package-metadata= flag.

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Add -Wl,--package-metadata=%7B%22type%22:%22deb%22%7D to LDFLAGS.

Which operating system are you running Bazel on?

Ubuntu resolute

What is the output of bazel info release?

No response

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

Corresponding bug in Ubuntu: https://bugs.launchpad.net/ubuntu/+source/bazel-bootstrap/+bug/2141591

[katre]

This is an interesting issue: Bazel doesn't seem to actually use LDFLAGS anywhere (if you check scripts/bootstrap/compile.sh you'll see it's defaulted to empty, but nothing seems to actually read it).

However, I cloned Ubuntu's repo (the online code browser doesn't seem to work, I assume because I don't have an account), and see that the file debian/rules has the line:

export BAZEL_LINKOPTS = $(subst $(space),:,-lstdc++ -lm ${LDFLAGS})

I then traced where BAZEL_LINKOPTS is used, and it's from rules_cc, specifically from cc/private/toolchain/unix_cc_cinfigure.bzl, where the variable is read, split on the ":" character, and saved as linkopts.

Your problem, then, is that your LDFLAGS contains a colon in the flag (as you noted), and it's not escaped. The split in rules_cc is handled by split_escaped, which requires characters that should not be used for a split to be prefixed with a % character.

So, to get back to where we started, is it possible for Ubuntu's debian/rules code that sets BAZEL_LINKOPTS from LDFLAGS to also escape any ":" characters to "%:"? (I don't know what the existing $(subst) call is actually doing, sorry I can't suggest anything better).

[katre]

To clarify: I'm not sure Bazel should be doing anything different here: while I can't find any actual documentation for BAZEL_LINKOPTS, the implementation clearly expects it to be a colon-separated list, and so anyone generating BAZEL_LINKOPTS needs to properly escape any internal colons that aren't actual separators.

[bdrung]

Thanks for the analysis. The Debian packaging needs to be fixed. I followed your suggestion and escaped : by %: and also escaped % to make split_escaped happy:

export BAZEL_LINKOPTS = $(subst $(space),:,-lstdc++ -lm $(subst :,%:,$(subst %,%%,${LDFLAGS})))

This failed with "Invalid toolchain configuration" as mentioned in the initial report. What worked is double escaping %:

export BAZEL_LINKOPTS = $(subst $(space),:,-lstdc++ -lm $(subst :,%:,$(subst %,%%%%,${LDFLAGS})))

Is that intentional? Could that be documented?

[katre]

Where is the definition of the subst used in debian/rules? I suspect the need to escape multiple times is because of the nested use of subst in

export BAZEL_LINKOPTS = $(subst $(space),:,-lstdc++ -lm $(subst :,%:,$(subst %,%%%%,${LDFLAGS})))

I've filed bazelbuild/rules_cc#893 to add better documentation of these environment variables from rules_cc, and then we can also link that from bazel's bootstrap files.