Data preprocessing for data mining

Abstract

People have increasing amounts data in the current prosperous information age. In order to improve competitive power and work efficiency, discovering knowledge from data is becoming more and more important. Data mining, as an emerging interdisciplinary applications field, plays a significant role in various trades’ and industries' decision making. However, it is known that original data is always dirty and not suitable for further analysis which have become a major obstacle of finding knowledge. This thesis aims to introduce this new field and data preprocessing as a critical step in a data mining project as well as its practical part using SPSS in order to show the effect of data preprocessing. The thesis first introduces why people need data mining, what is data mining, what kind of data people mine, how to use data mining and what challenges in data mining i The second part compares data mining and relational technologies including data warehouse, OLAP, statistics and machine learning. Then a more detailed theoretical framework is suggested about the task and procedures in data mining especially measures of data preprocessing. At the end, a case study illustrates how to achieve data preprocessing by using SPSS. After preprocessing, the data is clean, integrated and reduced. As a conclusion of the experiment, SPSS can fulfill basically most of the data preprocessing tasks and give a better insight of the data.