Skip to content

Add BISDN Linux switcher - #231

Draft
mwindower wants to merge 5 commits into
masterfrom
bisdn-switcher
Draft

mwindower wants to merge 5 commits into
masterfrom
bisdn-switcher

Conversation

@mwindower

@mwindower mwindower commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds a switcher for BISDN Linux, which programs the Broadcom ASIC from the Linux kernel state through baseboxd and OF-DPA. Four commits:

  1. Make the management interface configurable: METAL_CORE_MANAGEMENT_INTERFACE. Empty means the NOS default: eth0, on BISDN Linux enp0.
  2. Add BISDN Linux switcher (cmd/internal/switcher/bisdn):
    • detected by ID=bisdn-linux in /etc/os-release
    • renders systemd-networkd files, deletes stale netdevs and releases ports with netlink where networkd can't, and reloads networkd via D-Bus (new dbus.ReloadNetworkd)
    • renders frr.conf, validates it with frr-validation@ and reloads frr.service
    • front panel ports port1..portN, with their names as identifiers
    • golden tests for both templates
  3. Send LLDP for metal-hammer from metal-core: METAL_CORE_LLDP_SEND=true.
    • BISDN Linux has no LLDP agent. networkd's LLDP sends the machine-id as chassis ID and no port description, so metal-hammer ignores it.
    • metal-core sends chassis ID = port MAC, port ID = port name (locally assigned), port description = port name, system name = hostname.
  4. Render BISDN networkd files from dedicated templates:
    • one file per interface, NN-metal-core-<interface>.netdev|network, each from a small template per file kind (tpl/bisdn/*.tpl), as metal-networker does it
    • the netdevs to delete and the ports to release follow from the file names, no INI parsing
    • when a VRF goes away, all of its files (VRF, SVI, VXLAN, machine ports) are removed and its netdevs deleted; files without the NN-metal-core- prefix (metal-roles' mgmt VRF and management port, anything else) are never touched. Unit tests cover both, the lab checks that no file naming the removed VRF/VNI/VLAN is left and the mgmt files are still there
    • METAL_CORE_INTERFACES_TPL_FILE names a directory whose templates replace the embedded ones with the same name

Notes:

  • FRR 10.7.1 workaround: the BISDN FRR template renders use-underlays-nexthop-weight in every router bgp. FRR 10.7.1 shows that line whenever bgp deterministic-med is set (bgp: use-underlays-nexthop-weight / bgp deterministic-med flip-flop when using frr defaults datacenter FRRouting/frr#22939, fixed by bgpd: Fix flags used by use-underlays-nexthop-weight command FRRouting/frr#22552), so otherwise every reload would flip deterministic-med. It also renders debug bgp updates in/out the way FRR shows them, so frr-reload.py finds no difference after a reload.
  • godbus: networkd is reloaded with its own D-Bus method org.freedesktop.network1.Manager.Reload (what networkctl reload calls). go-systemd only talks to the systemd manager, and BISDN Linux's systemd-networkd.service has no ExecReload, so a unit reload fails and a restart would reconfigure all links. godbus is not a new module: go-systemd already depends on it, the PR only makes it a direct dependency.
  • Breakout and FEC are not handled here: they live in the Broadcom SDK configuration and need an ofdpa restart, so metal-roles provisions them.
  • Pinned pre-releases (why this is a draft):
  • Tested:
    • unit and golden tests, golangci-lint
    • a containerlab lab with BISDN Linux 6.0.0 leaves, configured by this switcher via its TestLabApply: BGP unnumbered, EVPN L3VNI and PXE L2VNI, VNI filter, metal-hammer LLDP content, runtime VRF add/remove with netlink cleanup, port admin state, idempotent frr-reload
    • not on switch hardware yet

Used AI-Tools ✨

  • Claude Code

References:

metal-core read the management address from eth0 only. The new setting
METAL_CORE_MANAGEMENT_INTERFACE (default eth0) names the interface, and
types.Nic carries its name for the templates, for NOSes that name the
management port differently.
BISDN Linux (detected by ID=bisdn-linux in /etc/os-release) programs the
ASIC from the Linux kernel state via baseboxd and OF-DPA. The bisdn switcher
renders systemd-networkd files (NN-bisdn-*) and frr.conf, deletes stale
netdevs and releases ports with netlink where networkd can't, reloads
networkd via D-Bus (new dbus.ReloadNetworkd) and FRR via frr.service, after
validating frr.conf with frr-validation@. Front panel ports are
port1..portN, reported with their names as identifiers; the management
interface defaults to enp0. Networkd's own LLDP is off on all front panel
ports.

The FRR template renders use-underlays-nexthop-weight in every router bgp:
FRR 10.7.1 (BISDN Linux 6.0.0) shows that line whenever bgp
deterministic-med is set (FRRouting/frr#22939, fixed by #22552), so every
reload would flip deterministic-med otherwise. It also renders
"debug bgp updates in/out" as FRR shows them, so frr-reload.py finds no
difference after a reload.

Port breakouts and FEC live in the Broadcom SDK configuration and need an
ofdpa restart, so they are left to provisioning.

Requires metal-stack/api with SWITCH_OS_VENDOR_BISDN (pinned to the branch
commit until it is released).
metal-hammer only accepts a switch neighbor whose LLDP chassis ID is a MAC
address and whose port ID is a MAC address or locally assigned, and uses the
port description as switch port name. BISDN Linux has no LLDP agent, and
systemd-networkd's LLDP sends the machine-id as chassis ID and no port
description.

With METAL_CORE_LLDP_SEND=true, metal-core sends LLDP on all switch ports
itself: chassis ID = port MAC, port ID = port name (locally assigned), port
description = port name, system name = hostname. Ports are rescanned every
10s, so ports created later get a sender too.

The port ID subtype needs metal-stack/go-lldpd#34; until it is released,
go-lldpd is replaced by github.com/mwindower/go-lldpd.
Render one networkd file per interface from a small template per file
kind (tpl/bisdn/*.tpl) instead of splitting one template at markers, as
metal-networker does. Files are named NN-metal-core-<interface>.<kind>,
so the netdevs to delete and the ports to release follow from the file
names and no INI parsing is needed. Files without that prefix, such as
metal-roles' mgmt VRF, are never touched.

METAL_CORE_INTERFACES_TPL_FILE names a directory whose templates replace
the embedded ones with the same name.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant