30. Lazy buffers
In a tick-to-trade path, heap allocations create latency spikes and unpredictable tail behavior. You want a reusable buffer that allocates lazily and minimizes allocator traffic during bursts. Consider correctness, exception policy, and cache behavior.
struct Buf {
int cap = 0;
std::unique_ptr<char[]> data;
void write(const char* src, int n) {
if (n > cap) { data.reset(new char[n]); cap = n; }
std::memcpy(data.get(), src, n);
}
};
Part 1.
Identify the correctness and latency pitfalls in this design. What concrete changes would you make to minimize allocations and stalls while keeping behavior explicit and predictable on failure?
Part 2.
(1) Should write be noexcept in production HFT? What failure policy would you choose?
(2) Is std::memcpy(nullptr, nullptr, 0) well-defined? Any subtle traps?
(3) Geometric growth vs exact-fit: latency, fragmentation, and cache behavior trade-offs?
(4) Would std::string_view or std::span<const char> improve API/ABI and prevent accidental copies?
(5) How would alignment and false-sharing concerns change this design in multi-threaded ingestion?
Answer
Answer (Part 1)
Allocate lazily but grow geometrically (e.g., next power-of-two or 1.5x) to amortize cost and reduce allocator churn and TLB misses. Fast-path the common case (n <= cap) before branching, and make the exception policy explicit: either pre-reserve at startup and mark write noexcept with fail-fast on OOM, or allow throws off the hot path. Ensure zero-length writes are permitted, document non-overlap expectations, and prefer non-owning views for inputs to avoid hidden copies and improve inlining.
Answer (Part 2)
(1) noexcept avoids unwinding overhead; many HFT systems prefer fail-fast (terminate/log) or pre-reserve to make OOM impossible. If you need recovery, keep it throwing but keep the throwing path cold.
(2) Yes, zero-sized std::memcpy allows null pointers and does nothing. Beware overlap (requires memmove) and that src may be unused but still evaluated.
(3) Geometric growth reduces reallocations and allocator contention, smoothing latency but increasing footprint. Exact-fit minimizes memory yet risks jitter from frequent allocations and fragmentation.
(4) Yes; std::string_view/std::span<const char> express pointer+length without ownership, improving clarity and ABI stability. They reduce accidental copies and enable better inlining.
(5) Use cacheline alignment/padding (e.g., alignas(64)) and per-thread instances to avoid false sharing. Separate hot fields from shared ones, and avoid cross-core contention.