Evaluation science
Benchmarks that reflect real tasks, operators, failure modes, and production economics.
Research that changes engineering decisions.
AI moves too quickly for implementation practice to become detached from research. Our agenda focuses on questions that alter feasibility, architecture, evaluation, and production performance.
Working principle
A useful research result should change what gets built, how it is evaluated, or whether it should be built at all.
Research agenda
Four connected technical themes.
Model systems
The architecture around models: routing, retrieval, memory, tools, permissions, and control.
Multimodal intelligence
Systems that perceive and reason across visual, textual, audio, and structured information.
Production learning systems
Infrastructure and operating methods for systems that must be observed, improved, and trusted.
Current field notes
Questions worth resolving.
These are working technical perspectives, not generic trend commentary or claims of completed academic publication.
01Evaluation systems
Evaluating agentic systems under real-world failure conditions
Why task completion is only one part of the evaluation problem, and how permissions, recovery, observability, and operator trust change the benchmark.
02Architecture decisions
When retrieval-augmented generation stops being the right architecture
A practical decision frame for retrieval quality, knowledge volatility, latency, workflow state, and when a different system design is warranted.
03Production economics
The engineering economics of model choice
How to compare frontier APIs, smaller hosted models, and specialized systems across quality, throughput, operational load, and change risk.