Data storage handler & technology : evaluation of real-time data handling technology
Bhandari, Ravindrasingh (2024)
2024
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2024121937454
https://urn.fi/URN:NBN:fi:amk-2024121937454
Tiivistelmä
This thesis specifically explores the data storage handler systems and technologies within the ARPA (Applied Research Platform for Autonomous Systems) a collaborative project between Turku University of Applied Sciences and Novia University of Applied Sciences. The ARPA project supports data management challenges in maritime systems by developing advanced data storage solutions. This thesis explores how open-source technologies, such as Apache Pulsar, PostgreSQL and Ambry, can effectively manage large, complex datasets from autonomous vessels and sensors in real-time.
A case study approach was used to design and implement a storage handler system within ARPA’s microservices-based architecture. Key components, including message brokers and secure data handlers, were evaluated to ensure efficient communication across data sources and handlers. REST APIs were employed to manage data flow and maintain high performance, scalability, and security across structured and unstructured data.
The findings show that the ARPA data storage framework successfully supports real-time data processing and provides the scalability and security needed for complex maritime environments. This thesis emphasizes the importance of open-source technologies in building reliable and adaptable data systems for autonomous applications. The insights gained from this study contribute to best practices in data security, real-time processing, and integration strategies that may benefit future autonomous maritime projects and other data-intensive fields.
A case study approach was used to design and implement a storage handler system within ARPA’s microservices-based architecture. Key components, including message brokers and secure data handlers, were evaluated to ensure efficient communication across data sources and handlers. REST APIs were employed to manage data flow and maintain high performance, scalability, and security across structured and unstructured data.
The findings show that the ARPA data storage framework successfully supports real-time data processing and provides the scalability and security needed for complex maritime environments. This thesis emphasizes the importance of open-source technologies in building reliable and adaptable data systems for autonomous applications. The insights gained from this study contribute to best practices in data security, real-time processing, and integration strategies that may benefit future autonomous maritime projects and other data-intensive fields.