Show HN: Spinf-serve – inference server for bulk scoring and decisions
Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0 - spinfinc/spinf-serve
Spinf-serve is an inference server for bulk scoring and decisions, using one GPU to process multiple questions efficiently.
It processes many questions in one pass, returning probabilities instead of generated text.
Designed for throughput, not latency, with support for specific NVIDIA GPUs.
Runs on a 24 GB NVIDIA GPU and uses an OpenAI-compatible API.
Summarised automatically by AI from the original article by Hacker News. AI can make mistakes, so check the original for details.
spinf-serve reads a piece of content once and answers many questions about it in the same pass. Each answer is a probability over the options you give, not generated text. It runs on one 24 GB NVIDIA GPU and serves an OpenAI-compatible API, the same API as the hosted spinf service. Apache 2.0.
We built this engine for our own work: a daily market index read from millions of news articles, each article asked dozens of narrow, calibrated questions. A serving engine built for chat re-reads the content for every question and generates one token at a time, which made that job slow and expensive. spinf-serve lays every question out as a branch of one read of the content and packs many contents into one forward pass. Bulk classification is where it started and where it is strongest.
Read the full story on Hacker NewsThat's the opening of the story. The full piece is published by Hacker News.
Rivian founder and CEO RJ Scaringe (left) watches Rivian R2s being built after an event at the Rivian assembly plant on May 19th, 2026, in Normal, Illinois. | Photo: Scott Olson…
