Case study

Simpler RAG architecture: more accurate, faster and cheaper

As evidenced by the recent rebuild of our Stoic Q&A app, less is often more. A simpler architectural structure and RAG tech stack resulted in much more accurate answers, although the LLM was switched as well: from Gemini 2.5 Flash to Gemini 3.5 Flash‑Lite. Median response time fell from 12.6 s to 4.6 s, and LLM cost from $7.23 to $3.72 per 1,000 answers. Although this study cannot establish which changes, and exactly how much, contributed to the app’s overall better performance, simpler architecture and tech stack may be the way to go. It’s also worth experimenting with newer - and often cheaper - LLMs to find the best balance between cost and performance.

Marty Paukstys Published October 26, 2026

The project

Memento Vivere is our Stoic app which bases all of its answers on 4 classic works by Epictetus (Discourses and Enchiridion), Marcus Aurelius (Meditations) and Seneca (Moral Letters to Lucilius).

It is a RAG system (retrieval-augmented generation): it finds the parts of the books that best match a question, and feeds them to an LLM, which provides an answer.

It is a good RAG and LLM testing ground, as the texts are complex and “philosophical”, so to speak.

The app was launched back in 2025, and the last changes were made on November 6, 2025.

We wanted to see if we could shave off latency and cost, as well as increase answer accuracy and overall quality.

Main results and findings

Memento Vivere is a tool grounded in Stoicism’s classic texts, quoting them and citing the sources. The goal is zero hallucinations, and all statements referenced to the actual words of Seneca, Marcus Aurelius and Epictetus.

We compared the live app (v1) with its rebuild from scratch (v2) on the same 140 questions.

v2 took 64% less time to provide an answer. Its median response was 4.6 s, vs. 12.6 s for v1. Most of the difference was in the LLM call: 3.7 s vs. 11.8 s, which the study can’t attribute to the LLM switch alone: v2 also asked for shorter answers and less thinking.

v2 answers cost almost 2x less: $3.72 vs. $7.23 per 1,000 answers. Both LLMs have the same list prices, so the savings came from tokens. v2’s LLM used half as many thinking tokens per answer, 759 vs. 1,519, and its answers had 38% as many tokens as v1’s (404 vs. 1,077).

v2 quoted only the original texts. 86% of v1’s answers had at least one quotation that was not in the texts.

v2’s citations supported their sentences more often than v1’s: 88% vs. 58%. Two AI judges, Claude Opus 5.5 and GPT‑6 Astra, checked whether each cited paragraph backs the sentence it is attached to.

v2’s answers were more faithful to the texts and clearer. Scored by Opus and Astra on a 0 to 2 scale, faithfulness rose from 0.45 to 1.65, and clarity from 1.04 to 1.93. Helpfulness moved less - from 1.21 to 1.45.

The AI judges scored v2 over one point higher overall: 3.47/5.00 vs. 2.37/5.00.

Both versions declined off-topic questions equally well. Each declined all 8 off-topic questions, and wrongly declined 4 of the 129 it should have answered.

v2 got a new feature which points people in crisis to help. For all 3 crisis questions, it showed a fixed message with crisis resources, such as the 988 Suicide & Crisis Lifeline. v1 answered them with Stoic advice, which both AI judges rated as serious harm.

The rebuild (v2) vs. the live app (v1)

88% vs 58%

Citations that support their sentence

v2 is more accurate

4.6 vs 12.6 s

Median response time

v2 takes 64% less time

$3.72 vs $7.23

LLM cost per 1,000 answers

v2 costs 49% less

Answers quoting words not in the texts · v1: 86% 0%

Overall quality, 1 to 5, AI judges · v1: 2.37 3.47

Thinking tokens per answer · v1: 1,519 759

Method

Two versions: v1 - original app. v2 - rebuilt from scratch in October 2026.
Test set: We asked both versions the same 140 questions: 55 about life problems (such as getting angry over small things), 20 about Stoic ideas, 15 the app should decline or handle with care (off-topic, medical or legal, and crisis questions), and 50 follow-ups in 10 conversations. Each version got every question 3 times: 420 requests per version.
Environment: Both versions ran on the same Apple M4 Pro Mac, one question at a time, on October 17–18, 2026.
Judging the answers: Two blind judges: Claude Opus 5.5/high and GPT‑6 Astra/high. 60 answers were rated by a human.
Cost basis: LLM cost is Gemini charges at list prices on October 17, 2026.

Core architecture and RAG stack

The two versions have major differences in both core architecture and RAG stack.

The core. v1 searches principles Gemini wrote from each chapter of the books, and quotes it picked from them. v2 is simpler. It searches the books’ own text: all 2,962 paragraphs, each with a short header naming the author, work and chapter or letter.

The RAG stack. v1 finds the 30 principles and quotes closest to the question, then uses a second step, a reranker, to keep the best 15 for the LLM. v2 finds the 20 closest paragraphs and sends the top 8 to the LLM, with no reranker step.

Table 1. Architecture

Part	Layer	v1 · live app	v2 · rebuild
Content searched	Core	v1 · live app2,685 principles and quotes	v2 · rebuild2,962 book paragraphs
Words per embedded chunk, median	Core	v1 · live app20	v2 · rebuild99
Embedding model	RAG stack	v1 · live appBGE large, local	v2 · rebuildGemini Embedding 2
Chunks retrieved --> sent to the LLM	RAG stack	v1 · live app30 --> 15 reranked	v2 · rebuild20 --> top 8
Words sent to the LLM, median	RAG stack	v1 · live app318	v2 · rebuild1,082
LLM model	LLM	v1 · live appGemini 2.5 Flash	v2 · rebuildGemini 3.5 Flash‑Lite
Thinking	LLM	v1 · live appDefault, dynamic	v2 · rebuildLow
Answer length, median	LLM	v1 · live app648 words	v2 · rebuild155 words

v1’s answer length includes its inline source references. Download CSV

Accuracy

Table 2. Accuracy and quality

Measure	Judged by	v1 · live app	v2 · rebuild
Answers with words in quotation marks not in the texts	Automatic check	v1 · live app86.2%	v2 · rebuild0.0%
Citations that support their sentence	Two AI judges	v1 · live app58.0%	v2 · rebuild88.0%
Faithfulness to the texts, 0 to 2	Two AI judges	v1 · live app0.45	v2 · rebuild1.65
Clarity, 0 to 2	Two AI judges	v1 · live app1.04	v2 · rebuild1.93
Helpfulness, 0 to 2	Two AI judges	v1 · live app1.21	v2 · rebuild1.45
Overall quality, 1 to 5	Two AI judges	v1 · live app2.37	v2 · rebuild3.47
Overall quality, 1 to 5	Human, blind	v1 · live app3.83	v2 · rebuild4.23

v2’s 0% comes from a check that removes any quotation not in the texts. Before it, 4.5% of v2’s answers had one. Human: blind ratings of 30 answer pairs, a difference too small to call. Download CSV

Declines and crisis questions

Table 3. Answers, declines and crisis help

Handling	Questions	v1 · live app	v2 · rebuild
Answered	Should be answered (129)	v1 · live app125	v2 · rebuild125
Declined	Off-topic (8)	v1 · live app8	v2 · rebuild8
Crisis resources given	Crisis (3)	v1 · live app0	v2 · rebuild3

Download CSV

Time and cost

Table 4. Step-by-step response time

Step	Runs on	v1 · live app	v2 · rebuild
Rewriting a follow-up	Gemini API	v1 · live appNone	v2 · rebuild776 ms
Embedding the question	Local CPU · Gemini API	v1 · live app47 ms	v2 · rebuild384 ms
Search	FAISS · Postgres	v1 · live app0.2 ms	v2 · rebuild17 ms
Rerank	Local CPU	v1 · live app694 ms	v2 · rebuildNone
LLM answer	Gemini API	v1 · live app11,825 ms	v2 · rebuild3,682 ms
Total	End to end	v1 · live app12,570 ms	v2 · rebuild4,459 ms

Median times. Each row is its own median, so the steps don’t add up to the total. v2 rewrites only follow-up questions. The 4.6 s and 12.6 s in the findings are per question, so they differ slightly from the totals here. Download CSV

Table 5. Tokens and LLM cost per answer

Measure	Basis	v1 · live app	v2 · rebuild
Input tokens	Mean per answer	v1 · live app2,477	v2 · rebuild2,384
Thinking tokens	Mean per answer	v1 · live app1,519	v2 · rebuild759
Visible output tokens	Mean per answer	v1 · live app1,077	v2 · rebuild404
Price per 1M tokens, input · output	Gemini list price	v1 · live app$0.30 · $2.50	v2 · rebuild$0.30 · $2.50
LLM cost per 1,000 answers	Mean	v1 · live app$7.23	v2 · rebuild$3.72

Thinking tokens are billed as output, and made up about half of each version’s LLM cost. v2’s cost includes its follow-up rewrites and question embeddings. Download CSV

Same prices for both LLM models. Gemini 2.5 Flash and Gemini 3.5 Flash‑Lite cost the same per token: $0.30 per million input tokens, and $2.50 per million output tokens. v2’s lower cost came from generating fewer tokens.

How the answers were graded

Two LLMs graded the answers blind: Claude Opus 5.5 and GPT‑6 Astra, both at high reasoning. Neither knew which version wrote an answer, or what the other judge said. Before grading, both were tested on planted answers with deliberate mistakes, such as a made-up quotation, and caught all of them after one fix to the grading instructions.

A human also rated 60 of the answers blind, and favored v2 less than the AI judges did: 4.23 vs. 3.83, which is a difference too small to call. However, reading them, it was obvious that v2 was more accurate and to-the-point than v1.

Practical takeaways

The most noticeable differences between v1 and v2 were speed, relevancy, and the length of the answers. Overall, v2 felt like a much more polished and sophisticated product.

It also showed that less may be more: a simpler core architecture and RAG stack not only improved the overall user experience, but also cut the app’s LLM cost in half.