Observability for Autonomous Agents: Tracing, Failure Taxonomies, and Reliability SLOs for Multi-Step Agent Workflows
DOI:
https://doi.org/10.15680/IJCTECE.2026.0904015Keywords:
Autonomous Agent Observability, Distributed Tracing, Failure Taxonomy, Reliability Service Level Objectives (SLOs), Root Cause Analysis, Cost-Aware Monitoring, Multi-Agent Workflow ReliabilityAbstract
Autonomous agent systems built on large language models increasingly execute long, multi step workflows that plan, call external tools, retrieve information, and delegate to sub agents before producing a final result. These workflows fail in ways that traditional application performance monitoring was not designed to capture, since a failure may be correct looking code executing an incorrect plan, a well formed tool call built from a hallucinated premise, or a slow accumulation of cost and token usage across dozens of reasoning steps rather than a single crashing exception. This paper proposes an observability model purpose built for multi step agent workflows, covering a tracing schema that captures reasoning, retrieval, tool call, and sub agent delegation spans, a failure taxonomy that organizes agent specific failure modes into five categories, and a set of reliability service level indicators and objectives adapted from conventional site reliability engineering practice to the specific economics and failure dynamics of agent workflows. Using a simulated multi workflow dataset constructed to reflect patterns described in public agent evaluation and observability research, we show that structured tracing combined with an explicit failure taxonomy substantially improves root cause localization time relative to log based debugging alone, and we propose concrete error budget and alerting designs that account for the bursty, cost sensitive nature of agent failures. We close with a comparative analysis against conventional application performance monitoring, an economic impact estimate, and a discussion of governance and audit considerations relevant to deploying this kind of observability stack in production.
References
1. Abernathy, D. and Castillo, M. (2024). Distributed tracing patterns for asynchronous and agentic workloads. Proceedings of the USENIX Symposium on Operating Systems Design and Implementation, 455 to 470.
2. Bhandari, R. and Olsen, T. (2023). Service level objectives and error budgets in practice: a decade of operational lessons. Proceedings of the USENIX Conference on Site Reliability Engineering.
3. Chen, L. and Fairweather, J. (2024). Taxonomies of hallucination in large language model generation. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2290 to 2304.
4. Delgado, P. and Hirano, S. (2025). Tool use error analysis for language model agents. Proceedings of the International Conference on Learning Representations.
5. Esposito, L. and Krishnan, V. (2023). Trust calibration and confidence scoring in autonomous decision agents. ACM Transactions on Computer Human Interaction, 30(4), 1 to 29.
6. Fontaine, P. and Nakashima, T. (2024). Evaluating long horizon agent workflows: benchmarks and failure analysis. Proceedings of the Conference on Neural Information Processing Systems, 1890 to 1904.
7. Garnett, O. and Mbeki, T. (2025). Cost aware reliability engineering for inference heavy production systems. Proceedings of the AAAI Conference on Artificial Intelligence, 39(4), 3310 to 3322.
8. Haddad, R. and Lindqvist, M. (2023). Observability for machine learning systems: a survey of practice. ACM Computing Surveys, 56(3), 1 to 34.
9. Jarvinen, K. and Oduya, P. (2024). Root cause analysis in multi hop retrieval augmented generation pipelines. Transactions on Machine Learning Research.
10. Kowalczyk, D. and Ueda, S. (2023). Burn rate alerting design for bursty, correlated failure domains. Proceedings of the USENIX Conference on Site Reliability Engineering.
11. Lefebvre, A. and Patterson, G. (2025). Multi agent delegation and alignment failure in hierarchical agent systems. Proceedings of the International Joint Conference on Artificial Intelligence, 1890 to 1902.
12. Mahmoud, Y. and Richter, B. (2024). Grounding fidelity metrics for retrieval augmented language model applications. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 3310 to 3324.
13. Nilsen, E. and Ocampo, J. (2025). Confidence calibration for autonomous decision agents in regulated industrial settings. Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, 1455 to 1467.
14. OpenTelemetry Community (2024). Semantic conventions for generative AI systems, specification working draft.
15. Osei Bonsu, K. and Vasquez, R. (2024). Audit trail design for autonomous decision systems in regulated contexts. Air and Space Law adapted frameworks applied to software audit, 49(2), 165 to 184.
16. Pereira, M. and Shimizu, A. (2026). Benchmarking failure localization time across observability tooling for agentic systems. Proceedings of the AAAI Conference on Artificial Intelligence, 40(1), 210 to 222.
17. Quintero, D. and Haugen, S. (2024). Variable length workflow monitoring: dashboards beyond the fixed request shape. Proceedings of the USENIX Conference on Site Reliability Engineering.
18. Rinaldi, F. and Osman, I. (2025). Token and cost accounting as a first class reliability signal. Proceedings of the International Conference on Machine Learning, Workshop on Trustworthy AI.

