KNET has several links which are fast right up until they aren’t.
The obvious version is the mobile backup at Leipzig. Its available capacity changes with the radio network, time of day and probably the phase of the moon. The less obvious version is a fixed broadband link with a queue inside the modem, or a tunnel whose provider polices it somewhere beyond my router. They all produce the same irritating failure mode: a large transfer is working exactly as intended while DNS, an interactive session or a routing packet waits behind it.
I had already made the queues visible in Grafana. Over the last few days I moved the other half into knet-ansible: deciding where each queue should form, what should remain responsive when it fills, and how to prove the policy is still attached after a deployment or reboot.
This wasn’t about making a speed test report a bigger number. It was about making the network predictable when there isn’t enough capacity for everything at once.
Owning the Queue
Traffic shaping only helps if my router becomes the bottleneck on purpose.
If I send slightly more than a link can carry, the excess waits somewhere else: in a modem, in a provider policer or in an opaque tunnel path. By the time packets reach that queue, my router has lost the opportunity to decide which flow should wait. A large upload can then add latency to a tiny control packet despite the two having very different consequences.
The fixed-rate links are therefore shaped just below the capacity I can reliably use. That sounds like throwing bandwidth away, but the small margin buys control over the queue. CAKE and FQ-CoDel can then keep independent flows from sitting behind each other, while a small amount of deliberate classification protects traffic whose delay hurts more than its volume.
The useful queue is the one I can classify, measure and change.
I tried to keep that classification semantic. Network control, interactive work and internal services get useful treatment because of what they are. Unknown traffic remains ordinary best effort. Explicitly marked background work remains bulk. A VLAN name is an addressing and firewall boundary; it does not automatically make every packet in that VLAN important.
Leipzig Has More Than One Kind of Link
r3 in Leipzig is where the policy became layered rather than merely rate-limited.
The primary fixed connection uses CAKE on download with per-internal-host fairness. Upload uses an HTB scheduler for broad traffic classes, with FQ-CoDel inside each class so one flow cannot monopolise its share. Control and interactive traffic can borrow available capacity without bulk traffic disappearing completely, and the existing guest limits remain separate beneath the aggregate policy.
Encrypted overlays needed a little more thought. On their logical interfaces, the router can still see the original packets and classify them before WireGuard hides their identity. On the physical underlay, the encrypted outer packet is deliberately treated as neutral transit. Calling every tunnel packet “infrastructure” would let a large transfer become high priority merely because it crossed KNET.
Download shaping also needs somewhere to attach on Linux. Ingress traffic cannot be queued in quite the same way as egress traffic, so VyOS redirects it through intermediate functional block devices. Those interfaces are part of the rendered candidate now, which means the policy and the place it attaches are reviewed together rather than assembled later by a hand-written command.
Letting the Mobile Link Move
A fixed shaper on the mobile backup would be wrong for most of the day.
Set it for the best case and the real queue moves back into the carrier whenever capacity falls. Set it for the worst case and a healthy link spends most of its time artificially slow. r3 therefore runs CAKE in both directions with an adaptive controller measuring load and one-way delay against multiple reflectors.
The controller moves the shaper within explicit safety bounds. When load is high and latency starts growing, it backs away until the queue is under control. When the path has room, it cautiously raises the rate again. The starting point remains conservative, so losing the controller leaves a useful shaped connection rather than an unbounded one.
The controller is pinned alongside its licence, configuration and service definition. A small supervisor makes sure the runtime unit still matches the deployed files, restarts it when those files change, and exposes its state to Prometheus. That makes the adaptive part a managed component of the router rather than an interesting shell script I have to remember is there.
One Queue Shared by Several Networks
r4 in Falkenstein has almost the opposite problem. Its external link is stable, but the same physical interface carries ordinary WAN traffic, service networks and several overlays through a router-on-a-stick design.
Treating those as unrelated queues would ignore the actual contention point. r4 now shapes the shared physical uplink just below line rate and uses CAKE’s host and flow isolation across the combined traffic. A small set of DSCP marks preserves network control, DNS and trusted management traffic; everything else remains best effort unless it has explicitly declared itself background work.
Some tunnel paths also have lower policers farther upstream. They keep their own child shapers beneath the shared physical limit. The shared queue answers “who gets this interface right now?” while the path shapers answer “how quickly can this particular route safely leave?” They solve different bottlenecks, so one does not replace the other.
Watching the Scheduler, Not a Speed Test
A successful VyOS commit tells me the configuration was accepted. It does not tell me the expected qdisc is attached, the adaptive controller is alive or packets are actually entering the classes I intended.
Both routers now export traffic-control state through node exporter’s textfile collector. The collector reads the kernel’s JSON view of qdiscs and classes, writes its metrics atomically, and reports its own success and age alongside policy attachment, throughput, backlog, drops, overlimits and configured CAKE rates.
The live r3 queue view. A flat drop graph is useful here because the collector, attachment checks and controller state prove the measurement path is alive.
The Grafana dashboard makes short queueing events visible next to the selected rates and class traffic. Alerts cover missing policies, stale or failed collection, a stopped adaptive controller and drops in the priority classes. That last distinction matters: drops in a bulk queue under real congestion can be the scheduler doing its job; sustained drops in control or interactive traffic mean the policy is failing where it is supposed to help.
This also closes an observability loop I was missing. The Ansible configuration says what the traffic policy should be. The router’s kernel says what is actually attached. Prometheus records how it behaved under load, and the alerts notice when any part of that evidence disappears.
What Changed
r3 now has one policy for a predictable fixed link, another which follows a variable mobile link, and explicit handling for overlays and constrained guest traffic. r4 has a shared fairness boundary at the physical bottleneck with separate ceilings where individual paths need them. Both are rendered, reviewed and deployed through the same KNET process as the rest of the router configuration.
The part I like most is that this is no longer a collection of plausible queue settings. The policy has a location, the adaptive controller has bounds, the kernel state has metrics and the important failure modes have alerts.
The network will still become busy. It should now be much better at deciding what “busy” is allowed to break.
Previously: Monitoring KNET and HubN.