AI in Education: Implementation of an LLM AI chatbot

Abstract

This thesis delves into the implementation of a Large Language Model (LLM) based AI Teaching Assistant (AI-TA) tailored for particular courses. The goal is to improve the learning experience of students. The AI-TA was made using a retrieval-augmented generation architecture: the courses materials were embedded into a vector database to focus the LLMs ’responses, and a minimal User Interface was created to enable students to have easy and simple interaction with the AI-TA. The methodology followed a constructive design and evaluation approach; the AI-TA’s performance was then tested against generic AI models that are not specialized in the courses’ content. The customized AI-TA answered correctly approximately 95% of the questions related to the courses fed to it, with only minor inaccuracies that can be noticeable in particular complex queries. The average response time was approximately 2 seconds, which meets real-time interaction requirements. In comparison with general models such as ChatGPT and Google’Gemini. The AI-TA was the best at determining answers that were closely aligned with the course material. This shows improved accuracy and relevance in subject-focused queries, indicating that the integration of courses-specific knowledge via embeddings can improve the precision of an LLM’s output in an educational environment. The outcome suggests that combining LLMs with specific courses is an effective approach to creating AI teaching assistants. Without the need for additional tuning, the AI-TA was able to deliver accurate responses and context-aware answers. This project underlines the potential of AI-TAs in education by providing immediate aid or support to students. Considerations for ethical concerns such as data privacy and bias mitigation are discussed to ensure responsible integration of AI in learning environments.