Next Generation Audit : Exploring the Potential of a Customized Audit Assistant for ISA Compliance

Abstract

This research provides practical insights for audit professionals regarding the ability of a customized chatbot to provide correct answers, along with the potential to cite relevant ISA standards. The study contributed to the growing field of AI in audit field by customizing a chatbot and assessing its practical applications in real-world auditing environments. The objective of this research was to develop a customized AI-powered chatbot focused on ISA standards compliance and to evaluate the accuracy and completeness of the responses as well as the ISA reference provided by the chatbot as. GPT-4 model was customized for this purpose. The underlying objective was to demonstrate in an exploratory setting whether chatbots, such as ChatGPT, have potential for use in tasks related to legislative compliance, which demand a high degree of accuracy. Given the manual effort involved in ISA compliance, the rapid advancement of AI tools and the specific demands of the audit profession, this topic was deemed highly relevant for exploration in the audit field. As a result of this study, 75% of the answers received the highest accuracy rating as comprehensive in compliance with ISA standards. An additional 19,8% of answers were rated as correct but inadequate, resulting in a combined total of 94,8% for correct responses, whether fully comprehensive or correct but partially lacking information. The reference rating was slightly lower, with 68,8% scoring a correct reference to ISA standards. The results demonstrated that ChatGPT can perform effectively even without extensive fine-tuning involved. Future research and development should prioritize continuous exploration of AI’s practical applications in auditing, with emphasis on empirical and case studies. Specifically, for tasks involving information retrieval related to regulations and guidelines, such as ISA compliance, further fine-tuning of chatbots could enhance accuracy. Additional sources of knowledge, such as IFRS and other relevant guidelines, could be incorporated, positioning chatbots as a primary resource for auditors.