Evaluation of AI Tools Suitability for Academic Writing Purposes for Internal Use
Pakkala, Aleksi (2025)
Pakkala, Aleksi
2025
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2025052716910
https://urn.fi/URN:NBN:fi:amk-2025052716910
Tiivistelmä
The rise of generative AI has had a profound impact on higher education, and Metropolia Business School wanted to provide guidance on which AI tools can be used reliably in research and development work by students. The objective of this Thesis was to evaluate the suitability of selected AI tools for academic writing purposes in Metropolia Business School. To achieve this, an AI tool Evaluation Framework was developed.
This research employed an applied action research strategy. Data was collected in three rounds through interviews to support current state analysis, initial proposal development, and validation of the proposal. The current state analysis was conducted first to understand the existing use of generative AI at Metropolia and the existing guidance on generative AI use at Metropolia. The theoretical framework focused on covering key elements of generative AI, accuracy and reliability of generative AI, and evaluation of generative AI output.
Based on the current state analysis, best available knowledge, and input from stakeholders, an AI tool evaluation framework with explicit guidelines for implementation was developed. The evaluation of the suitability of selected AI tools for academic writing purposes was done by utilizing this AI tool evaluation framework. The AI tool evaluation framework was further refined based on feedback from key stakeholder and insights gained during the initial implementation of the framework.
This Thesis presents evaluation results of the suitability of selected AI tools for academic writing purposes by students for year 2025, and an AI tool evaluation framework that can be used to evaluate the suitability of AI tools for a variety of tasks and workflows also in the future. The AI tool evaluation framework and guidelines were developed to be used with individuals without extensive experience with AI, with reasonable amount of work for Metropolia Business School.
This research employed an applied action research strategy. Data was collected in three rounds through interviews to support current state analysis, initial proposal development, and validation of the proposal. The current state analysis was conducted first to understand the existing use of generative AI at Metropolia and the existing guidance on generative AI use at Metropolia. The theoretical framework focused on covering key elements of generative AI, accuracy and reliability of generative AI, and evaluation of generative AI output.
Based on the current state analysis, best available knowledge, and input from stakeholders, an AI tool evaluation framework with explicit guidelines for implementation was developed. The evaluation of the suitability of selected AI tools for academic writing purposes was done by utilizing this AI tool evaluation framework. The AI tool evaluation framework was further refined based on feedback from key stakeholder and insights gained during the initial implementation of the framework.
This Thesis presents evaluation results of the suitability of selected AI tools for academic writing purposes by students for year 2025, and an AI tool evaluation framework that can be used to evaluate the suitability of AI tools for a variety of tasks and workflows also in the future. The AI tool evaluation framework and guidelines were developed to be used with individuals without extensive experience with AI, with reasonable amount of work for Metropolia Business School.