Skip to content

wb | simplify - #6105

Merged
fmaste merged 14 commits into
masterfrom
bench-simplify
Apr 16, 2025
Merged

fmaste merged 14 commits into
masterfrom
bench-simplify

Conversation

@fmaste

@fmaste fmaste commented Feb 5, 2025 •

Copy link
Copy Markdown
Contributor

Optimize workbench's Nix derivations for cache hits and avoid unnecessary rebuilds

  • Port topology.jq functionality to cardano-topology Haskell lib.
  • Obtain profile files in a single derivation (profile.json, node-specs.json and the top level topology.json).
  • Obtain all nodes topology files (projections) in a single derivation (topology-node-*.json).
  • Don't create a derivation in every supervisor-conf.nix call (needed multiple times for Nomad backends).
  • Add .passthru.noGitRev to cardano-node and cardano-cli and use them to avoid rebuilds.
  • Constrain workbench derivation "src" to avoid unnecessary rebuilds (only bash scripts and it's data files. Skip nix files).
  • Minimal workbench derivation, make it re-usable in different circumstances by including the binaries later.
  • Extract an attrset with the profile's data to use around Nix, create profile/backend data derivations in a different step.
  • Separate profile/backend data from the runner derivations. Not everything should be built if no runner is needed.
  • Remove intermediate derivations / overhead (the .value/__fromJSON and .JSON/__toJSON attributes).
  • Skip generation of both plutus datum and redeemer derivations when not needed by the profile.
  • Remove usage of functions like jsonFilePretty that create extra derivations / overhead.
  • Remove cardano-mainnet-mirror from flake inputs (least used shell).
  • Disable lockfile when using the nomadexec backend.
  • Fix profile forge-stress-p2p P2P flag.

Add your description here, if it fixes a particular issue please provide a
link
to the issue.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. These may include:
    • golden tests
    • property tests
    • roundtrip tests
    • integration tests
      See Runnings tests for more details
  • Any changes are noted in the CHANGELOG.md for affected package
  • The version bounds in .cabal files are updated
  • CI passes. See note on CI. The following CI checks are required:
    • Code is linted with hlint. See .github/workflows/check-hlint.yml to get the hlint version
    • Code is formatted with stylish-haskell. See .github/workflows/stylish-haskell.yml to get the stylish-haskell version
    • Code builds on Linux, MacOS and Windows for ghc-8.10.7 and ghc-9.2.7
  • Self-reviewed the diff

Note on CI

If your PR is from a fork, the necessary CI jobs won't trigger automatically for security reasons.
You will need to get someone with write privileges. Please contact IOG node developers to do this
for you.

@fmaste
fmaste force-pushed the bench-simplify branch 2 times, most recently from 13e663c to 06601cf Compare February 17, 2025 12:23
@fmaste
fmaste force-pushed the bench-simplify branch 2 times, most recently from 4e284ff to f5f73f0 Compare February 17, 2025 18:18
@fmaste
fmaste force-pushed the bench-simplify branch 3 times, most recently from d56158c to b2b32fd Compare March 5, 2025 12:35
@fmaste
fmaste force-pushed the bench-simplify branch 2 times, most recently from 68cf103 to f67948f Compare March 22, 2025 13:43
@fmaste
fmaste force-pushed the bench-simplify branch 3 times, most recently from c9ac502 to 09d8461 Compare April 1, 2025 17:36
@fmaste
fmaste force-pushed the bench-simplify branch 14 times, most recently from 7ac7a40 to 905469f Compare April 9, 2025 15:28
@fmaste
fmaste force-pushed the bench-simplify branch 4 times, most recently from 47ab517 to 4adfcb8 Compare April 15, 2025 15:19
@fmaste
fmaste marked this pull request as ready for review April 15, 2025 18:41
@fmaste
fmaste requested a review from a team April 15, 2025 18:41
@fmaste
fmaste requested review from a team as code owners April 15, 2025 18:41
@fmaste
fmaste force-pushed the bench-simplify branch 2 times, most recently from 5fce09e to c635f85 Compare April 16, 2025 15:22

@mgmeier mgmeier left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😮 🚀
Excellent work - I'm so looking forward to merging this. Thank you!

@fmaste
fmaste added this pull request to the merge queue Apr 16, 2025
@johnalotoski

Copy link
Copy Markdown
Contributor

Awesome, awesome, awesome! 🚀🚀🚀

Merged via the queue into master with commit 605d60b Apr 16, 2025
@fmaste
fmaste deleted the bench-simplify branch April 16, 2025 19:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants