OCTOPUS: Towards A Tag Model for Cross-Query Reuse in Semantic Query Processing: what it means for business leaders

OCTOPUS stores every answer a language model gives about a database row as an ordinary column, so later queries read the judgment instead of paying the model again.

1. What the method is

OCTOPUS is a prototype middleware over PostgreSQL that introduces a tag model for semantic query processing. A tag is a column, or a separate relation for pairwise judgments, that holds the value a model returned when asked a natural-language predicate about a row, together with the provenance of that value: confidence from the model's token log probability, model name, timestamp, and the stage of any cascade. Five tag operators form the unit of work: a per-row filter, a per-row map, a per-pair compare, an embedding-based clustering, and a model-free nest. Every common semantic operator from systems such as LOTUS and Palimpzest is translated into a fixed composition of these tag operators and ordinary SQL. Because tags are data, the engine filters, indexes and stores them, and once a row carries a value no model is needed to answer that predicate for it again. The paper is a vision paper with a feasibility study, not a production system.

2. Why the method was developed

Language models let a database answer predicates that no schema column records, such as whether a film suits young children or whether a product is Christmas-themed. The authors observe that nearly every current semantic operator system discards the model's judgments when a query ends. iPDB, PLOP and FlockMTL cache only duplicate rows within one query; LOTUS and AIDB cache across queries only when the rendered prompt is identical, which almost never happens with user-phrased questions. The result is that overlapping predicates are paid for again and again, and no optimizer-style pruning or ordering can use what was already learned. The authors argue the problem is granularity: a whole-table semantic operator produces one answer reusable only by the same question over the same data, whereas a stored per-row judgment serves any later query that needs that row judged. A data model designed for reuse is needed, not a bigger cache.

3. Who should care

Our data platform and analytics engineering teams should care first, because this changes where model calls sit in a query pipeline and makes judgments a managed data asset with maintenance obligations. Product teams building search, recommendation or review-analysis features over unstructured catalogues face the same recurring predicates the paper describes, and would see inference spend fall as a workload matures. Finance and FinOps should note that cost here moves from per-query to per-row-per-predicate, which is predictable and amortizes. Data governance and audit functions gain something they rarely get from LLM features: every answer carries its model, confidence and timestamp, so a disputed classification can be traced and re-judged. Anyone evaluating vendors such as Snowflake Cortex AISQL or DuckDB-based semantic extensions should ask whether judgments persist across queries, since the paper's table shows none of the compared systems do so beyond identical prompts.

4. How the method works

A query arrives in natural language or as an operator plan. One call to a frontier model, the Parser, separates relational predicates from semantic ones, inspects the existing tags, and emits a typed logical plan with relational filters pushed down. If no tag exists for a predicate, a NULL-filled column is created. The Executor then runs three steps. Read first: SQL applies the relational guard, reads the tag, and selects open rows, those with NULL or confidence below a user threshold, while related tags join into the WHERE clause so that, for example, a row already marked not vegetarian is eliminated from a vegan query without a call. Settle: only the open rows reach the model, batched and concurrent. Write back: each answer persists with its provenance. Reuse rules derived by the Parser cover exact match, synonym, complement and taxonomy; the paper also describes residual-test simplification and probabilistic ordering for LIMIT queries, which the prototype does not yet implement.

5. How it was evaluated

The study uses the movie scenario of SemBench, chosen because its data comes from real Rotten Tomatoes reviews and its predicates recur across its eight queries. OCTOPUS is compared with iPDB, which extends DuckDB in-engine, ThalamusDB on DuckDB, and LOTUS and Palimpzest, which run their own engines. All systems use gemini-2.5-flash with concurrent calls, on a Movies table that lists 11 of its 105 films twice, with duplicates left in. Cost is counted as rows, or pairs for joins, that the model judges, which is independent of pricing and batching; where a system packs several pairs into one prompt, the pairs are counted. Quality is reported as a mean score across the eight queries. The queries run once each in fixed order, which the authors note is the least favourable setting for cross-query reuse. We have the full paper, but it describes itself as a feasibility study on a single benchmark scenario.

6. How it performed

Across the eight queries OCTOPUS judged 268 rows against 2,768 for iPDB, 17,240 for ThalamusDB, 17,038 for Palimpzest and 28,910 for LOTUS, with mean quality of 0.82 against 0.87, 0.83, 0.79 and 0.79. Three queries issued no model call at all because earlier queries had filled the tag they needed. On the pairwise join Q7, separability let OCTOPUS judge 108 rows at quality 0.74, the best of the five, while two baselines judged all 16,384 pairs. The authors are candid about the price: Q6 inherited Q5's judgments and scored 0.58, so stored errors propagate, and the trust threshold and Parser matching strictness remain untuned knobs. Rules are beliefs inferred by a model, not declared constraints, tag staleness under model or data change has no treatment, and the single-scenario study cannot show generality. Our call is watch, with a small internal pilot worth considering wherever we already run recurring LLM predicates over a stable table. (Source: arXiv 2610.32586, 2026)