How to find RAG pipeline latency regressions instead of blaming the whole stack

End-to-end latency hides the cause

A RAG request may slow down because of embedding, retrieval, reranking, model generation or an external service. A single total-duration metric tells you the user is waiting but not which stage changed.

Keep a stage-level baseline

Capture stage timings under a known workload and compare subsequent runs with the same expectations. The useful output is not only the slowest stage, but whether the bottleneck moved and how much SLO budget remains.

• Track end-to-end and per-stage SLA thresholds.
• Compare current stage profile with the previous baseline.
• Report the largest regression and its contribution to total latency.
• Attach confidence based on observation coverage.

Connect performance to cost carefully

If a team knows the business or infrastructure value of an impact unit, the profiler can estimate economic impact from user-supplied assumptions. It should not invent a dollar value for latency without that context.

Decision-ready output

A GO/WARN/BLOCK gate allows the profiler to sit before deployment or inside an ongoing performance monitor. That makes the measurement directly useful to both CI systems and AI operations agents.