Combating Involuntary Information Leakage in Video Conferences Using Computer Vision Algorithms

Abstract

Information security is an endless process. With the evolution of technologies, this topic has been broadly discussed over time. During the COVID-19 pandemic, most activities have changed from personal attendance to online, for instance, meetings. This thesis focuses on information leakage when people are attending online meetings with eyeglasses. As the glass itself has a reflective principle, the work is aiming to use a learning-based method to prevent the reflection of sensitive information, especially information leakage via eyeglasses while attending online meetings. The data was collected through simulation of real-world behavior, and three customized deep learning algorithms have been evaluated. With a customized loss function, the area of eyeglasses can be properly detected and blurred, to add noise to the content reflected on the glasses. And with the combination of transfer learning from an advanced pre-trained facial landmarks detection model, the outcome demonstrated the agility and capability to prevent information leakage under the defined scenario. With the consideration of diverse types of devices running meeting software, for example, phones and tablets, the final result is capable to run on different platforms.