You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[FR] - Log reason when node switches tip to a fork #6014
Describe the feature you'd like
Please log the reason why the node preferred another fork
Describe alternatives you've considered
The independent tool cncli provides logging of block data including the block VRF value into a sqlite database. This database can be queried for insight into whether the likely reason the node preferred another fork was because the other fork terminal block had a lower block VRF and the tie break rule was applied.
Additional context / screenshots
The node knows the reason for switching tip to a competing fork but this reason isn't logged. Rather the logs only confirm that the tip was switched. But, was the reason due to the competing fork being longer? Or was it due to the other fork's terminal block having a lower VRF causing the tie-break rule to be applied? Or did the node prefer another fork some other reason?
This information would be really helpful when trying to identify what caused your own block to not be adopted.
Yes, please: if this is considered impractical to add I would like to hear why. Our SPO operation believes this is important for quality assurance purposes.
@carbolymer what kind of engagement do the developers expect for this issue not to be Stale? There won't be anything new to report until someone in the relevant team finally responds.
This log info has been long & vitally requested for diagnostic purposes and the need for it is never going to go away. Those who need it most would be small stake pools with infrequent blocks... and therefore needed more in the near future if & when the K PoS parameter increases (as is constantly suggested in governance conversations).
If this issue keeps being dismissed, the implication is that QA isn't very important for stake pools and/or chain density isn't that important for Cardano itself. I don't think either of these things are true and I can state from SPO experience dating back to the beginning of the Shelley period that implementing this feature (which we arguably should have had from day 1) would be helpful for SPOs and node infrastructure operators at all levels of endeavour.
The trace messages already contain enough detail to deduce the reason why a node changes its chain. Concretely, whenever we change our selection, we trace the so called "SelectView" of the tip of both the old and the new chain. This SelectView contains exactly the information that is needed in order to determine whether a chain is preferable to another chain. See PraosChainSelectView for more details and assorted documentation.
In Conway, the chain order works like this, via a lexicographical combinationi of the following criteria, when deciding whether to switch from the current chain to a candidate chain:
compare chain length, preferring larger values
when (same slot and same issuer) compare operational certificate issue number, preferring larger values
when (slots differ by at most 5) compare VRF, preferring smaller values
By comparing the old and new SelectView, one can see which of these three rules was responsible for the chain switch:
Here, we can see that the chainLength of newTipSelectView is larger than the one of the chainLength of the oldTipSelectView, so the chain switch happened due to rule 1.
If these would be equal, one would have to look at rule 2 next (and then rule 3) to see whether it is responsible.
For convenience/simplicity, we could further enrich the trace message to directly include eg a reason field that says something like "LongerChain" or "BetterTieBreaker".
cc @mgmeier as you might have already started with this.
In both the old and new tracing system, you should see this when you enable JSON output and use a severity of at least Notice for ChainDB.AddBlockEvent.AddedToCurrentChain/ChainDB.AddBlockEvent.SwitchedToAFork.
@mgmeier@jutaro how can this issue be automatically be marked Stale when it's been assigned to somebody & that label has been deliberately removed? How can it be that the CI here really puts such an issue on track for automatic closure? 🤔
thanks @jasagredo@mgmeier — along with this confirmation, please (for the sake of the SPO community) post a snippet of the log message that we should be looking for... in the new tracing system as well as the old tracing system if applicable. 🙏
The old tracing system does not support this new message field.
In the new tracing system, it's logged under the ChainDB.AddBlockEvent.SwitchedToAFork namespace.
The message is of severity Notice, so it's only filtered out if you set your config to minimum severity Warning or higher.
[2026-04-14 18:57:00.6869Z][0979accfcade:ChainDB.AddBlockEvent.SwitchedToAFork](Notice,25) Switched to a fork, new tip: 6ca9f1fc17c029a120f662531f1dd93148cdddd35d4c386897fd37be754339db at slot 942
Reason: candidate is longer (At (BlockNo 29)) than our selection (At (BlockNo 28))
Activity
github-actions commentedon Nov 15, 2024
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 120 days.
TerminadaPool commentedon Nov 15, 2024
Please stay as an open feature request.
rphair commentedon Nov 15, 2024
Yes, please: if this is considered impractical to add I would like to hear why. Our SPO operation believes this is important for quality assurance purposes.
github-actions commentedon Dec 16, 2024
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 120 days.
rphair commentedon Dec 16, 2024
@carbolymer what kind of engagement do the developers expect for this issue not to be
Stale? There won't be anything new to report until someone in the relevant team finally responds.This log info has been long & vitally requested for diagnostic purposes and the need for it is never going to go away. Those who need it most would be small stake pools with infrequent blocks... and therefore needed more in the near future if & when the
KPoS parameter increases (as is constantly suggested in governance conversations).If this issue keeps being dismissed, the implication is that QA isn't very important for stake pools and/or chain density isn't that important for Cardano itself. I don't think either of these things are true and I can state from SPO experience dating back to the beginning of the Shelley period that implementing this feature (which we arguably should have had from day 1) would be helpful for SPOs and node infrastructure operators at all levels of endeavour.
carbolymer commentedon Dec 27, 2024
There was a slack discussion about this issue some time ago.
@jasagredo @amesgen would you mind sharing main takeaways here?
amesgen commentedon Jan 6, 2025
The trace messages already contain enough detail to deduce the reason why a node changes its chain. Concretely, whenever we change our selection, we trace the so called "
SelectView" of the tip of both the old and the new chain. ThisSelectViewcontains exactly the information that is needed in order to determine whether a chain is preferable to another chain. SeePraosChainSelectViewfor more details and assorted documentation.In Conway, the chain order works like this, via a lexicographical combinationi of the following criteria, when deciding whether to switch from the current chain to a candidate chain:
By comparing the old and new
SelectView, one can see which of these three rules was responsible for the chain switch:Let's look at an example:
{ "at": "2024-12-13T13:50:58.295305594Z", "ns": "ChainDB.AddBlockEvent.AddedToCurrentChain", "data": { "kind": "AddedToCurrentChain", "newTipSelectView": { "chainLength": 11214658, "issueNo": 16, "issuerHash": "8e0bb9b126acd8e65a023602377d9a7a6f15af394d1f7bdd852182b8", "kind": "PraosChainSelectView", "slotNo": 142531567, "tieBreakVRF": "f8f31aabb555124392485cbd7ba02bded4e4eac8980fc36fde0c9a21d7cf2d51c6fd4ddda8113153639db862781dafacbdd77da4e072452a330c32509506762e" }, "newtip": "17d5b0f21e86503b41851640e9558610ebe9b7a62c96a08873eb4d0c7961a460@142531567", "oldTipSelectView": { "chainLength": 11214657, "issueNo": 20, "issuerHash": "c63dab6d780a74cbae2a27696c9723f55b3092b2bd001256df03827f", "kind": "PraosChainSelectView", "slotNo": 142531554, "tieBreakVRF": "905ff37db665e86d60d93dabc92ec86124b1a22165e4968facca226e790d9b7dd3f60502fa84dcd30d95ac28a9278e5e72b8b2eb451759ab59c92fc65909a2b4" } }, "sev": "Notice", "thread": "76", "host": "ramify" }Here, we can see that the
chainLengthofnewTipSelectViewis larger than the one of thechainLengthof theoldTipSelectView, so the chain switch happened due to rule 1.If these would be equal, one would have to look at rule 2 next (and then rule 3) to see whether it is responsible.
For convenience/simplicity, we could further enrich the trace message to directly include eg a
reasonfield that says something like"LongerChain"or"BetterTieBreaker".cc @mgmeier as you might have already started with this.
TerminadaPool commentedon Jan 6, 2025
That sort of log information would absolutely suit my needs. I wasn't aware we could get the "tieBreakVRF" values logged ordinarily.
What do I need to configure in my node config.json file so that such information is logged in my systemd journal?
amesgen commentedon Jan 7, 2025
In both the old and new tracing system, you should see this when you enable JSON output and use a severity of at least
NoticeforChainDB.AddBlockEvent.AddedToCurrentChain/ChainDB.AddBlockEvent.SwitchedToAFork.51 remaining items
rphair commentedon Jan 24, 2026
@mgmeier @jutaro how can this issue be automatically be marked
Stalewhen it's been assigned to somebody & that label has been deliberately removed? How can it be that the CI here really puts such an issue on track for automatic closure? 🤔Expose reason for a switch in ChainSel tracer (#1826)
github-actions commentedon Mar 9, 2026
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 120 days.
rphair commentedon Mar 9, 2026
🤖
github-actions commentedon Apr 13, 2026
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 120 days.
jasagredo commentedon Apr 13, 2026
This feature was implemented in Consensus in IntersectMBO/ouroboros-consensus#1826 and in Cardano-node in #6469.
Once we have confirmation that the new information is being logged correctly, we can finally close this issue.
@mgmeier can I ask you to confirm this is the case?
rphair commentedon Apr 14, 2026
thanks @jasagredo @mgmeier — along with this confirmation, please (for the sake of the SPO community) post a snippet of the log message that we should be looking for... in the new tracing system as well as the old tracing system if applicable. 🙏
mgmeier commentedon Apr 14, 2026
The old tracing system does not support this new message field.
In the new tracing system, it's logged under the
ChainDB.AddBlockEvent.SwitchedToAForknamespace.The message is of severity
Notice, so it's only filtered out if you set your config to minimum severityWarningor higher.mgmeier commentedon Apr 14, 2026
This is an example text output:
mgmeier commentedon Apr 14, 2026
This is an example JSON output (I linted it, it's actually a one-liner):