The Standard Redundancy Mistake
Most teams approaching feed redundancy think about it in simple terms: connect to the same exchange twice via different network paths, and route traffic through whichever connection is alive. This works well enough for disaster recovery. It does not work well for active trading where both feeds are live simultaneously and you need to merge their output into a coherent market state.
The problem is not maintaining two connections. The problem is that two connections to the same exchange will diverge. Not occasionally. Constantly. At the message level.
Sequencing Divergence: The Core Problem
Even when both feeds connect to the same exchange gateway, they receive messages in different orders. Network path jitter, TCP retransmissions, and exchange-side multicast delivery differences all produce sequencing divergence. For equities on NASDAQ ITCH, messages carry sequence numbers that let you reconstruct order. For some other protocols, you get timestamps but no hard sequence guarantee across physical links.
When you run two feeds simultaneously, your deduplication logic has to answer two questions: is this message a duplicate of something already processed? And if you have two different versions of the same event, which one do you trust?
The naive approach is a hash map keyed on some unique message identifier, with a TTL. When a message arrives on Feed B that matches a key already in the map from Feed A, discard it. This works until the same underlying event produces different identifiers on each feed (which happens with derived fields like calculated mid-price) or until the sequence diverges by more than your deduplication window.
Building a Merge Layer That Does Not Become the Bottleneck
The deduplication layer sits on the critical path between feed ingestion and your order book reconstruction. Any synchronization you add there adds latency directly to your market state updates. The instinct to use a mutex-protected hash map is correct for correctness and wrong for latency.
What works better: a fixed-size ring buffer per feed pair, with a wait-free design where the writer thread on each feed writes to separate cache lines from the reader thread doing deduplication. If you have two feeds for the same venue, pin each feed's processing thread to a separate physical core and design the merge buffer so the deduplication read is always from a different NUMA node than either write path.
In our current testbed setup for NYSE Pillar feeds, this approach keeps the deduplication overhead under 400 nanoseconds at the median, measured with RDTSC bracketing on both sides of the merge operation. The overhead is not zero, but it is stable and predictable.
Arbitrating When Feeds Disagree
Deduplication handles duplicate messages. Arbitration handles divergence: situations where both feeds carry a message for the same event but with different field values.
This happens more than you might expect. A common case: an order modification arrives on Feed A with one timestamp and on Feed B with a slightly different timestamp due to how the exchange gateway assigns wall-clock times to different multicast groups. If you use the timestamp field downstream for analytics or compliance, you now have two valid but inconsistent records of the same event.
Our approach is a tiered arbitration policy. For price fields, take the earlier-arriving version (whichever feed delivered it first based on local receive timestamp). For sequence-number-adjacent ordering fields, prefer the feed with the lower gap count over the trailing measurement window. For timestamp fields destined for compliance logging, flag the divergence and log both values with a merged-record marker rather than silently choosing one.
That last point is not about latency. It is about auditability. Compliance systems downstream should know when two feeds disagreed, not receive a silently arbitrated record that looks authoritative.
The Latency Budget for Redundancy
A common question: what is the latency cost of running two feeds versus one? The honest answer is that it depends entirely on what you do with the slower feed.
If your routing engine consumes only the first-arriving message (which is the right design for any latency-sensitive path), redundancy adds zero latency to the fast path. The second feed exists to detect gaps and provide recovery, not to slow down the primary path. Where latency cost appears is in gap detection: when Feed A drops a sequence number and you wait for Feed B to confirm or fill the gap, you are adding a measurement window to your market state publication. In our lab environment, we target a gap-fill wait of under 5 milliseconds before accepting a state update as valid even with a confirmed gap.
If you are tempted to wait for both feeds to agree before publishing a market state update, you are doing active-active with consensus, not redundancy with failover. That is a different architecture with much higher latency cost, and it is appropriate only for specific compliance or settlement use cases, not for execution infrastructure.
What Dual-Feed Does Not Solve
Dual-feed architecture handles transport-level failures and jitter. It does not handle exchange-side data quality problems. If the exchange itself publishes a bad tick, both feeds carry it. Your downstream logic needs to catch that with sanity checking on the field values themselves, not by hoping the second feed disagrees.
It also does not replace gap fill requests. When both feeds drop the same sequence range (which happens during extreme message rate spikes), you still need an out-of-band recovery mechanism. For ITCH-based feeds this is typically a separate TCP retransmission channel. Factor the round-trip cost of that channel into your worst-case market state staleness budget before claiming you have a complete redundancy solution.