Local AI for Scan-to-BIM : An Experimental Study on LLM-Based Chatbots and Point Cloud Segmentation

Abstract

The aim of this thesis was to investigate whether local open-source artificial intelligence (AI) tools, including large language models (LLMs) and deep learning (DL) methods for point cloud segmentation, could enhance Scan-to-BIM workflows. To achieve this, a proof-of-concept program was developed featuring a local LLM chatbot with function calling capabilities to process text, IFC models and point cloud data. PointNet++ models were trained on public and simulated datasets for semantic segmentation, compared with full fine-tuning and Low-Rank Adaptation (LoRA) strategies. Experiments were conducted using data from SmartLab at Metropolia. The results showed that LLMs improved usability by enabling natural language interaction in the scan-to-BIM process. Fully fine-tuned models achieved higher accuracy, while LoRA offered better training efficiency under limited hardware. Integration tests confirmed the feasibility of executing end-to-end natural language queries with semantic analyses within the chatbot framework. The results can be utilised to create more intuitive and accessible local Scan-to-BIM assistants that allow conversational querying of point clouds. Future work may extend these capabilities with LLM agents, additional IFC-specified datasets and automated IFC element generation to support complete automated scan-to-BIM processes for the construction industry.