A Big Data System for Time-Series Sensor Data
Nguyen, Thang (2017)
2017
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-202404096065
https://urn.fi/URN:NBN:fi:amk-202404096065
Tiivistelmä
The objective of this thesis is building a pipeline of processing and monitoring sensor data in a time-series format, to quickly identify faulty sensors or unexpected issues. The application can be applied in various fields such as: IoT (Internet of Thing), or telemetry systems.
In traditional systems, data is often stored and processed with the relational database management system (RDBMS) model. However, with the increasing of data sources and analytic demands, current systems need to be improved to handle substantial amount of data. Therefore, the main development task would be implementing Big Data technologies to make a scalable, high performance sensors data storage and analytics system.
This thesis described the fundamental knowledge of Big Data tools, and how to apply them to real world application of time-series data monitoring. The expected outcome is a fully working system of storing and processing data on the Hadoop environment.
In traditional systems, data is often stored and processed with the relational database management system (RDBMS) model. However, with the increasing of data sources and analytic demands, current systems need to be improved to handle substantial amount of data. Therefore, the main development task would be implementing Big Data technologies to make a scalable, high performance sensors data storage and analytics system.
This thesis described the fundamental knowledge of Big Data tools, and how to apply them to real world application of time-series data monitoring. The expected outcome is a fully working system of storing and processing data on the Hadoop environment.