LLM inference thoughtshttps://fergusfinn.com/https://fergusfinn.com/blog/control-layer-benchmarking/https://fergusfinn.com/blog/control-layer-benchmarking/Benchmarking the Doubleword Control Layer Tue, 21 Oct 2025 00:00:00 GMThttps://fergusfinn.com/blog/control-layer-benchmarking/mdhttps://fergusfinn.com/blog/control-layer/https://fergusfinn.com/blog/control-layer/We're releasing our AI Gateway. Why are AI gateways hard to do right, and why do we think we've done it right. Tue, 21 Oct 2025 00:00:00 GMThttps://fergusfinn.com/blog/control-layer/mdhttps://fergusfinn.com/blog/inference-arithmetic/https://fergusfinn.com/blog/inference-arithmetic/Comparing InferenceMAX to the hardware limits Wed, 22 Oct 2025 00:00:00 GMThttps://fergusfinn.com/blog/inference-arithmetic/mdhttps://fergusfinn.com/blog/llm-guided-scheduling/https://fergusfinn.com/blog/llm-guided-scheduling/An idea on how to use LLMs to help with scheduling Tue, 23 Sep 2025 00:00:00 GMThttps://fergusfinn.com/blog/llm-guided-scheduling/mdhttps://fergusfinn.com/blog/paged-attention/https://fergusfinn.com/blog/paged-attention/A discussion of paged attention Mon, 22 Sep 2025 00:00:00 GMThttps://fergusfinn.com/blog/paged-attention/mdhttps://fergusfinn.com/blog/scheduling-in-inference-engines/https://fergusfinn.com/blog/scheduling-in-inference-engines/A look into how inference engines choose which requests to process Mon, 22 Sep 2025 00:00:00 GMThttps://fergusfinn.com/blog/scheduling-in-inference-engines/mdhttps://fergusfinn.com/blog/spacelike-speculative-decoding/https://fergusfinn.com/blog/spacelike-speculative-decoding/Speculative decoding speeds up LLM inference, but using another model works poorly. Mon, 22 Sep 2025 00:00:00 GMThttps://fergusfinn.com/blog/spacelike-speculative-decoding/mdhttps://fergusfinn.com/blog/llm-powered-content-discovery/https://fergusfinn.com/blog/llm-powered-content-discovery/Building a content discovery system using parallel primitives and BST-based ranking with LLM comparisons Fri, 16 Jan 2026 00:00:00 GMThttps://fergusfinn.com/blog/llm-powered-content-discovery/mdhttps://fergusfinn.com/blog/parallel-primitives-blog/https://fergusfinn.com/blog/parallel-primitives-blog/Exploring coordination patterns from parallel computing for multi-agent LLM systems Wed, 31 Dec 2025 00:00:00 GMThttps://fergusfinn.com/blog/parallel-primitives-blog/mdhttps://fergusfinn.com/blog/bst-expensive-comparisons/https://fergusfinn.com/blog/bst-expensive-comparisons/A lock-free binary search tree optimized for expensive async comparisons, with threaded linked list for O(1) sorted iteration Tue, 13 Jan 2026 00:00:00 GMThttps://fergusfinn.com/blog/bst-expensive-comparisons/mdhttps://fergusfinn.com/blog/arxiv-llm-search/https://fergusfinn.com/blog/arxiv-llm-search/Applying parallel primitives to search and rank 2.4 million arXiv papers using LLM judgments Thu, 01 Jan 2026 00:00:00 GMThttps://fergusfinn.com/blog/arxiv-llm-search/mdhttps://fergusfinn.com/blog/weighted-fallback-flattening/https://fergusfinn.com/blog/weighted-fallback-flattening/When a weighted-random fallback rejects samples and retries without replacement, high error rates cause low-weight models to be selected far more often than their weights suggest. Tue, 17 Feb 2026 00:00:00 GMThttps://fergusfinn.com/blog/weighted-fallback-flattening/md