Toward Self-Managing Microservices: Design of a Semi-Autonomous Data Collection Agent

Abstract

As applications, networks and IT systems grow larger and increasingly complex, manual real-time management is often infeasible. As a solution to this problem, autonomous software has emerged as a key area for development. By following pre-configured rules or exercising artificial intelligence, programs can operate more independently, handling tasks such as monitoring logs, allocating system resources, and coordinating devices. This project details one such example, following the design and implementation of a semi-autonomous software agent and its impact as part of a larger data analysis and visualization platform.

Building on the basis of an existing data collection microservice written in Python, the project focuses on the addition of self-monitoring and self-healing features, such as automated state monitoring, configuration updates, and rule-based action handling. The project demonstrates how such features increase resilience, reliability and ease of use in both the service and the larger platform while ensuring that administrators and users retain the capabilities to fully control and customize the system. While noting the various challenges posed by automation, this project seeks to provide evidence that, in many cases, its enormous benefits clearly outweigh its apparent costs.