In the research agent I built at Doriot, the obvious move would have been to pick one strong model and send every query to it. That's not what I did. The reasons are worth writing up separately from the retrieval side of the system.
GPT-4, through Azure OpenAI, is the default generator. It handles most queries. By the time generation runs, the intent classifier and hybrid retrieval have already scoped the context. What's left is mostly synthesizing an answer. DeepSeek-R1 sits behind it as a fallback when a query needs deeper, multi-step reasoning rather than straightforward synthesis.
What Actually Triggers the Fallback
Two signals matter most. The first comes from the intent classifier. Because it's multi-label, a query tagged with several intents at once already signals more complexity than a single-intent question. Think funding-round details plus competitor analysis plus market trends. The second is the shape of the merged retrieval context. Sometimes both the SQL path and the vector path return substantial results. When those results need to be reconciled, not just summarized, that's a reasoning task. It's not a simple lookup.
Neither is a hard numeric threshold. It's routing logic. Does this query ask the model to hold several independent threads of retrieved information and reason across them? Or does it ask the model to describe what's already in front of it? The former goes to the reasoning model. The latter doesn't need to.
Why Not Just Always Use the Reasoning Model
Reasoning models are built to think longer before answering. That's the point. It also means real latency and cost. Paying that tax on a query like "what was this company's Series B amount" is pure waste when GPT-4 answers it correctly and quickly. Defaulting to the strongest model for everything feels safe. It's rarely the efficient choice once you look at what most requests actually need.
Why Not Just Stay on the Default Model
The opposite mistake is assuming a single capable model plus a better prompt can handle everything. Some questions require holding multiple threads of context and reasoning across them. A single-pass generator doesn't do that reliably, no matter how you write the prompt. At some point the honest fix is a different model built for that work, not a longer system prompt on the same one.
Keeping the Routing Simple
It would be easy to turn this into an elaborate decision tree with many models and many rules. I kept it to two models and one clear trigger condition. A router with two paths is something you can reason about when things go wrong. You can iterate on it without needing a flowchart to explain your own system six months later.
The broader takeaway: which model handles a given request is a design decision made per request. Treat it the same way you'd pick which index or database fits a query. It shouldn't be a single global setting picked once at the start of a project and left alone.
Learn More
The full project is open source: Investment Market Research Agent on GitHub.