Lower glibc requirement for pre-compiled `libpineappl_capi.so`

[scarlehoff]

In here #297 due to a problem with github workflows, the glibc support was moved all the way up to 2.29.

This is a problem in some clusters that are using very old installations (e.g., galileo in Milan uses Debian Buster, current oldstable).

Since it doesn't seem to be a restriction of pineappl but just of the CI, it would be nice to revert that change.

[cschwan]

This should concern only the pre-compiled binaries that you can download manually or with install-capi.sh, right?

[scarlehoff]

Yes.

[cschwan]

This seems to be a tough problem. We need at least glibc 2.28 to support Node 20, which in turn is used by the Github actions, but the CAPI itself should be able to compile with 2.17 (see #232). We either

1. need to run a container inside the Github action (containerception)
2. or we figure out how to install an old glibc parallel to the default one in our container and figure out how to link the CAPI shared object to the old library (maybe https://github.com/rust-cross/cargo-zigbuild can help here)

[alecandido]

Just out of curiosity: isn't it an option to just not support pre-compiled binaries on old clusters, and require local compilation?

[cschwan]

That amounts to me just closing this Issue and thus effectively saying 'nope', but I'd like to find out what's possible. Chances are that if your cluster is too old, you're also having a hard time installing new software like Rust and cargo-c.

[alecandido]

That amounts to me just closing this Issue and thus effectively saying 'nope', but I'd like to find out what's possible.

I understood the consequences, it was clear to me.
Finding solutions is always challenging and amusing. But there are times when the gain is very limited (beyond amusement)

Chances are that if your cluster is too old, you're also having a hard time installing new software like Rust and cargo-c.

This is the interesting part: is it actually the case?

My understanding was that the pre-compiled binaries were made available for ease of use, to make users' experience smoother. I never thought it was because of some actual limitation of relevant platforms (for which the source distribution would be impossible).
It's true that, once compiled, you only have to satisfy binary compatibility, and you do not depend any longer on build tools (that's usually the reason for building containers in two stages).

But even cross-compiling has some limits, and the current platform support for Rust seems not to extend before Glibc 2.17. Here it would be just enough, but it's exactly on the boundary.

Is there anything special about Glibc 2.17, other than being adopted by a certain version of certain Linux distros (or other platforms as well) which seems to be a superset of those on the clusters you aim to support?
Otherwise, there's a chance that you may fix the issue somehow, to just get it worse if the support for something (e.g. a certain version of RHEL) expires, and people will move forward (while the outdated clusters stay there).

At that point, it may be more practical (since easier) to stay source-compatible with a certain platform, just by imposing MSRV low enough, such that the build tools compiling that will include those running on the old platform.
Binary-compatibility is in principle more minimal, but breaking source-compatibility may effectively restrict the binary distribution as well[*]...

[*]: up to "magic", of course - you may still start transpiling, and apply whatever sort of compatibility trick by patching things yourself; everything is possible, at some point is just about how much pain...

[cschwan]

Chances are that if your cluster is too old, you're also having a hard time installing new software like Rust and cargo-c.

This is the interesting part: is it actually the case?

I don't know, it was pure speculation on my part. What I wanted to say is that I wouldn't be surprised if that's the case. Maybe @scarlehoff can explain what he did on galileo.

My understanding was that the pre-compiled binaries were made available for ease of use, to make users' experience smoother. I never thought it was because of some actual limitation of relevant platforms (for which the source distribution would be impossible).

The primary concern is indeed to make both the users and maintainers' lifes easier.

It's true that, once compiled, you only have to satisfy binary compatibility, and you do not depend any longer on build tools (that's usually the reason for building containers in two stages).

But even cross-compiling has some limits, and the current platform support for Rust seems not to extend before Glibc 2.17. Here it would be just enough, but it's exactly on the boundary.

Is there anything special about Glibc 2.17, other than being adopted by a certain version of certain Linux distros (or other platforms as well) which seems to be a superset of those on the clusters you aim to support? Otherwise, there's a chance that you may fix the issue somehow, to just get it worse if the support for something (e.g. a certain version of RHEL) expires, and people will move forward (while the outdated clusters stay there).

I don't think there's anything special about GLIBC 2.17 (here are GLIBC's changelogs).

I don't believe there will be any problems if you use a shared object that was compiled with an old version of GLIBC, because GLIBC should be backwards compatible; newer GLIBC should support libraries linked to older GLIBC versions, as long as the SONAME agrees.

[alecandido]

I don't believe there will be any problems if you use a shared object that was compiled with an old version of GLIBC, because GLIBC should be backwards compatible; newer GLIBC should support libraries linked to older GLIBC versions, as long as the SONAME agrees.

I was worried about forward compatibility, not backward (which I assume, though I'm not sure to what extent Glibc is following semver).

If the Rust compiler for your MSRV will move forward, it may step to any Glibc > 2.17, say 2.18. So, you may need to compile with Glibc 2.18 to support Rust internals, but then you will have only 2.17 at runtime, with the same problem you now have, since compiled against 2.29...

[cschwan]

Right, that may become a problem at some point. Let's worry about it when/if it happens.

[cschwan]

If we're OK with lowering the Github action version, we can alternatively

• downgrade all actions in the capi-linux job in release.yml and simply
• use a container with an older glibc version.

That's certainly the easiest path, but I'm not sure how long if at all Github supports these actions.

[alecandido]

That's certainly the easiest path, but I'm not sure how long if at all Github supports these actions.

Not sure, but I tend to think that there is actually no support. You may just try old versions of the actions, and hope to be lucky about their compatibility with the runners
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

• use a container with an older glibc version.

It seems a more reliable alternative.

[scarlehoff]

fwiw, also the cluster in Sevilla requires an older glibc

[Radonirinaunimi]

fwiw, also the cluster in Sevilla requires an older glibc

I don't think there is a suitable solution for this. Usually, University clusters (especially old ones) provides access to the CERN CVMFS repository which provides modern compilers.