Skip to content

[Rust Planner] part 2: devbox build - #104

Merged
savil merged 2 commits into
mainfrom
savil/rustlang-2
Sep 14, 2022
Merged

savil merged 2 commits into
mainfrom
savil/rustlang-2

Conversation

@savil

@savil savil commented Sep 9, 2022 •

Copy link
Copy Markdown
Collaborator

Summary

This PR adds support for devbox build for Rust projects.

  1. Needed to add the gcc nix-package because Rust needs it for libc calls.
  2. We directly execute the binary produced from cargo build --release, instead of cargo run to avoid needing to add the cargo nix package in the final runtime image.

How was it tested?

  1. added plan.json files to testdata.
  2. In each testdata/rust directory, did:
    • devbox build
    • docker run devbox -> inspected output to ensure correct rust version was executed with.

savil commented Sep 9, 2022

Copy link
Copy Markdown
Collaborator Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

Comment thread testdata/rust/rust-stable/plan.json Outdated
},
"start_stage": {
"command": "",
"command": "rustup default stable \u0026\u0026 cargo run \u0026\u0026 rustc --version",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need \u0026. Is that what the go json marshaller does? Can we make it output & instead?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its from json marshaller. I'll see if there's a way to avoid it...

@savil
savil force-pushed the savil/rustlang-2 branch 3 times, most recently from d56ee29 to 70845cd Compare September 11, 2022 14:31
@savil
savil force-pushed the savil/rustlang-1 branch 3 times, most recently from 21a4714 to e72f016 Compare September 11, 2022 14:40
@savil

savil commented Sep 12, 2022

Copy link
Copy Markdown
Collaborator Author

@loreto I added the most minimal Overlays: []string field in the Plan struct in previous PR (#97).

As I look into devbox build implementation for Rust, I think we can get away without having any runtime-overlays, by copying the built binary (similar to golang_planner) from BuildStage to StartStage.

That said, I can see other planners wanting a runtime-overlay. Do we want to distinguish dev-overlays and runtime-ovarlays in the code now?

If so, there are a couple of options on implementing it:

  • DevOverlays: []string, RuntimeOverlays: []string in addition to the existing DevPackages and RuntimePackages
  • Alternatively (and my preference), may be nicer to wrap this into a common struct so we have: Dev: PackageInfo, Runtime: PackageInfo where PackageInfo is a struct of { Packages: []string, Overlays: []string}

WDYT? For now, I'm going to punt on this question (of dev versus runtime overlays) and consider Overlays as just DevOverlays, but wanted to raise it in case folks feel we should add it up-front.

@loreto loreto left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

I would rename Overlays to NixOverlays. I don't think we need to distinguish between Dev and Runtime overlays for the time being.

@savil
savil force-pushed the savil/rustlang-2 branch 3 times, most recently from 72376a4 to 141afe1 Compare September 12, 2022 23:09
@savil
savil marked this pull request as ready for review September 12, 2022 23:10
@Lagoja Lagoja linked an issue Sep 13, 2022 that may be closed by this pull request

@LucilleH LucilleH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try rebasing and run go test ./... again

Comment thread planner/languages/rust/rust_planner.go Outdated
RuntimePackages: []string{"gcc"},
SharedPlan: plansdk.SharedPlan{
BuildStage: &plansdk.Stage{
Command: "cargo build --release",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@savil when you rebase, I removed the default install stage inputFiles field, so you'd have to add

InstallStage: {
    InputFiles: []string{"."}
}

Comment thread planner/languages/rust/rust_planner.go Outdated
BuildStage: &plansdk.Stage{
Command: "cargo build --release",
},
StartStage: &plansdk.Stage{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, add InputFiles for StartStage

@savil
savil force-pushed the savil/rustlang-2 branch 2 times, most recently from a893c9f to fabfbb8 Compare September 13, 2022 22:12
@savil
savil force-pushed the savil/rustlang-1 branch 5 times, most recently from 2f744d1 to 201c02c Compare September 14, 2022 18:48
Base automatically changed from savil/rustlang-1 to main September 14, 2022 18:55
@savil

savil commented Sep 14, 2022 •

Copy link
Copy Markdown
Collaborator Author

image size:

  1. with runtime packages = {rust-overlay.minimal, gcc} = 800MB
  2. with runtime pkgs = {gcc} = 300MB
  3. with no runtime pkgs = 30MB

The current implementation is (1). I want to explore alternate implementations to reduce the size.

@loreto

loreto commented Sep 14, 2022

Copy link
Copy Markdown
Contributor

With that big of a difference, I would say forget the suggestion of checking the version within the testdata example, and let's just generate the small images.

@savil

savil commented Sep 14, 2022

Copy link
Copy Markdown
Collaborator Author

Updated to just have gcc as dependency. This reduces the image size but at 300MB it is still fairly heavy. Will followup with improvements.

@savil
savil merged commit 42b41af into main Sep 14, 2022
@savil
savil deleted the savil/rustlang-2 branch September 14, 2022 22:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Language Request: Rust

3 participants