GitHubGet Started Free

Industry-leading search performance.

88% first-result precision - validated, measured, and production-ready. Every number on this page is evidence-based and cites published research or a reproducible benchmark. Zero hallucinated metrics.

Report v1.0Nov 202530 queries · 76 memories · 5 projectsTREC-style evaluation
Precision@1
0%
First-Result Precision
↑ +8-18 pts vs 70-80%1
Precision@5
0%
Top-5 Precision
↑ top of 75-85% range2
MRR
0.00
Mean Reciprocal Rank
↑ +0.1-0.2 vs 0.6-0.73
Latency p95
<0ms
Average Latency
↑ 33-60% faster than 300-500ms6
Performance vs Industry Standards

Measured against published baselines

Recallium's results plotted against the cited industry-standard range for each information-retrieval metric. Bars show measured performance; shaded zones mark the published baseline.

Recallium (measured)Industry standard range
First-Result Precision
Precision@1
88%
+8-18 pts
Top-5 Precision
Precision@5
85%
top of range
Mean Reciprocal Rank
MRR ×100
0.80
+0.1-0.2
Recall@10
Recall@10
75%
+5-15 pts
Query Coverage
availability
100%
0 failed
0255075100

Measured across 30 queries over 76 memories in 5 interconnected projects. Baselines cited in Industry Standards & Citations.

Performance by Query Type

Consistent across every search scenario

Precision@5 broken down across the five query categories in the evaluation set.

0%
Exact Match
P@5 · 9.2 avg results
Function names, error codes
0%
Semantic
P@5 · 9.5 avg results
Natural-language questions
0%
Cross-Project
P@5 · 8.0 avg results
Related codebases
0%
Hybrid
P@5 · 8.5 avg results
Mixed keyword + semantic
Market Positioning

Beyond the top-tier band

First-result precision (P@1) of competitive tiers, derived from aggregated enterprise-search studies. Recallium sits above the top-tier platform band.

50%60%70%80%90%FIRST-RESULT PRECISION (P@1)Enterprise search60-80%Commercial systems65-75%Top-tier platforms75-85%Recallium · 88%

Tier boundaries aggregated from enterprise-search studies1, commercial systems, and top-tier platforms2. Full citations below.

Benchmark Methodology

Rigorous, reproducible, TREC-style

Evaluation follows standard information-retrieval paradigms with an LLM judge producing ~300 relevance judgments.

01
Test Dataset

76 memories across 5 interconnected projects simulating real-world technical documentation.

02
Query Set

30 diverse queries spanning exact-match, semantic, cross-project, hybrid, and ambiguous categories.

03
Metrics

Standard IR metrics: Precision@1/5/10, Recall@5/10, MRR, NDCG, Coverage, and Latency.

04
Evaluation

Claude Sonnet 4.5 as intelligent evaluator producing ~300 relevance judgments.

100% query coverage 0 failed searches ~300 relevance judgments
Hybrid vs Vector-Only

Why hybrid search wins

Vector-only systems (e.g. mem0, Supermemory) rely on semantic similarity alone, missing exact matches and technical terminology. Recallium fuses semantic + keyword + file-based retrieval.

First-result precision by approach
same 30-query eval set
1007550088%75%58%RecalliumVector-onlyRAG base

Vector-only midpoint of cited 70-80%1; RAG baseline 57.6%5.

88% vs 70-80% first-result precision
A measurable lead over typical vector-only systems.
Semantic + keyword matching
Finds relevant results even with typos or different terminology.
File-based search
Enables precise code-context retrieval that pure vectors miss.
Pattern detection
Surfaces related memories across projects automatically.
Industry Standards & Citations

Every baseline, cited

All performance comparisons reference published research or commercial benchmarks.

1
First-Result Precision (P@1): 70-80% standard
Enterprise search typically targets precision in the 60-80% range for top results, with precision-recall tradeoffs being a fundamental challenge.
Buellesbach, N. (2023). Metrics that matter for measuring search performance. · View source ↗
2
Top-5 Precision (P@5): 75-85% range
Precision and recall are typically at odds; tightening requirements can push precision to the 67-85% range.
OpenSource Connections (2016). Search Precision and Recall By Example. · View source ↗
3
Mean Reciprocal Rank (MRR): 0.6-0.7 typical
Commercial systems typically achieve 0.6-0.7; top-tier systems reach 0.8 or higher.
Heidloff, N. (2023). Metrics to evaluate Search Results. · View source ↗
4
Recall@10: 60-70% industry average
Recall of 80% is considered good for search; 60-70% is typical, given the precision-recall tradeoff.
Constructor.com (2025). Measuring Site Search Relevance: Precision and Recall. · View source ↗
5
RAG system performance: ~58% relevance
In 57.6% of cases the returned documents were judged actually relevant; LLM-judged relevance achieved ~80% agreement with humans.
Elastic Labs (2024). The BEIR benchmark & Elasticsearch search relevance. · View source ↗
6
Search latency: <300ms acceptable, <500ms typical
P95 benchmarks show typical systems at 200-500ms; sub-200ms is considered excellent for enterprise search.
Elastic Blog (2024). Benchmarking and sizing your Elasticsearch cluster. · View source ↗

Production-ready search, measured and proven.

Reproduce the benchmark on your own corpus - the eval harness ships with the open-source repo.

Get Started FreeView benchmark repo

Recallium Search Benchmark Report · November 2025 · Test Dataset v1.0 · All comparisons evidence-based