Implementing data classification at the database schema level for data categorisation, identification and monitoring : Enhancing data security

Abstract

Managing sensitive information is critical to any organization due to the data protection regulations defined by the government and various compliance institutions. This study aims to address challenges in data classification in diverse applications and locations where data has been saved. Database schema level classification enforces a classification framework such as GDPR (General Data Protection Regulations) at the database level to ensure compliance. The primary solution aims to identify sensitive data and then classify it into different categories based on GDPR thereby enhancing regulatory compliance. Named entity recognition(NER) which is part of the Natural language processing(NLP) technique has been used majorly to build a custom GDPR classification dictionary. The open source library, SpaCy has been used as a framework. Apart from the existing models, additional training has been done in identifying domain specific datasets such as personal health information and payment card details. Anomaly detection and reporting is a part of the compliance framework and it is capable of real time checking. Test results showed significant improvement in the accuracy of the data classification and validation in line with the GDPR. This drastically reduced the manual data identification and worked as a proactive risk management utility. Further, this is an extendable solution to support other regulatory frameworks and enforce the data security of an organization. This work has demonstrated how well schema level classification can solve compliance requirements and also shows future requirements, such as adopting additional frameworks and model scalability. This work contributes to addressing regulatory compliance challenges and advancing data security practices.