-
LLM powered data structures: A concurrent, lock-free binary search tree
A lock-free binary search tree optimized for expensive async comparisons, with threaded linked list for O(1) sorted iteration
-
Large-Scale Semantic Search Without Embeddings
Applying parallel primitives to search and rank 2.4 million arXiv papers using LLM judgments
-
Parallel Primitives for Multi-Agent Workflows
Exploring coordination patterns from parallel computing for multi-agent LLM systems
-
How fast can an LLM go?
Comparing InferenceMAX to the hardware limits
-
Control Layer Benchmarking
Benchmarking the Doubleword Control Layer
-
The Doubleword Control Layer
We're releasing our AI Gateway. Why are AI gateways hard to do right, and why do we think we've done it right.
-
LLM guided scheduling
An idea on how to use LLMs to help with scheduling
-
Paged attention
A discussion of paged attention
-
Scheduling in inference engines
A look into how inference engines choose which requests to process
-
Using caching for fast speculative decoding
Speculative decoding speeds up LLM inference, but using another model works poorly.