[ci] Use sccache in builds - #10246
Conversation
Mousius
left a comment
Mousius
left a comment
There was a problem hiding this comment.
Well this is very exciting 😸!
- update tlc-pack/ci with terraform for public S3 bucket
Why do we need to have this public? Can't the instances assume a role to access it?
- update docker images so they all have sccache
Are we just missing ci_cpu now? I think I got the others you added sccache to 😸
| sccache --show-stats | ||
|
|
||
| # Use ninja as the builder for cmake | ||
| pip install --user ninja |
There was a problem hiding this comment.
Can we not apt-get install ninja-build rather than using pip here?
There was a problem hiding this comment.
When these task_*.sh run, it is with a non-privileged user, so we can't apt-get in these scripts. Perhaps an alternative solution is to add Ninja in the Docker image in the docker/install/*.sh scripting rather install it for every single build job we run.
There was a problem hiding this comment.
Aha! Yeah, that'd be much better than a side channel pip install thanks @leandron!
There was a problem hiding this comment.
Yeah installing in the image is definitely better, #10257 adds it (separate PR so we can land the images and then merge this one). I kept it with pip since the Ubuntu 18 ninja-build is pretty old (1.8, 2017) compared to the binaries in pip (1.10, 2020) and they should be equivalent otherwise
Sorry public wasn't the right word, I meant OSS, we had spun this up in the octo-staging Jenkins to test. The bucket would be private to the runners. Though having a public-readable sccache bucket is potentially on the list of followups since it means anyone could benefit from the cached build.
Yup that's the only one, might be good to roll #10257 into the update too |
driazati
changed the title
|
Closing in favor of #10359 which adds a proper |
…pache#10257) Following on from apache#10246, this moves the `pip install`-at-runtime deps to the docker image install so they are baked in.
This adds the secret to the config so apache/tvm#10246 will work
5 participants
This sets the builds for each platform to use sccache as a build cache. For a fully cached rebuild, this reduces the time (in testing) from 30+ minutes to 1-2 minutes (e.g. if the build isn't touching any C++ files). We can run this for a while and if it is successful then remove the extra code around rebuilding using the existing on-instance build directory.
The sccache server is backed by an S3 bucket configured in tlc-pack/ci-terraform#3.
Example run: https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-10353/5/pipeline/57