Custom GPT Assistance in Visually Impaired Shopping
Aho, Aino (2025)
2025
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2025091524680
https://urn.fi/URN:NBN:fi:amk-2025091524680
Tiivistelmä
Visual impairment brings many challenges in day-to-day life, including grocery shopping, both online and in-person. This thesis focuses on the in-person aspect, exploring the the creation, testing, and evaluation of a custom-built GPT assistant designed to support visually impaired individuals during in-person grocery shopping. The assistant was created with ChatGPT’s GPT Builder, and was intended to help users with product identification, label reading, weighing produce, and navigating store layouts through text, voice, and image inputs.
The development began with interviews with visually impaired individuals to identify desired functions for the custom GPT. Instruction tuning was carried out by experimenting with different wordings and by testing the GPT’s limits using faulty images, having features such as blurriness, incomplete product views, incorrect brand or flavor combinations, visually similar packaging, and hard-to-read labels. Once the GPT reached a satisfactory level of performance, testing was conducted with visually impaired participants recruited through Satakunnan Näkövammaiset ry.
The findings demonstrate that large language model tools can be adapted for practical assistive purposes, offering new opportunities for enhancing accessibility in everyday tasks. At the same time, the study highlights significant limitations, including accessibility barriers within the ChatGPT application itself, and the issues with dependency on external platform design choices outside of the developer’s control.
The thesis concludes that while GPT-based tools show promise for supporting visually impaired individuals, close collaboration with accessibility communities is essential in their development. These results contribute to the understanding of AI-driven assistive technologies and underscore the importance of inclusive design in shaping the future of AI-powered accessibility solutions.
The development began with interviews with visually impaired individuals to identify desired functions for the custom GPT. Instruction tuning was carried out by experimenting with different wordings and by testing the GPT’s limits using faulty images, having features such as blurriness, incomplete product views, incorrect brand or flavor combinations, visually similar packaging, and hard-to-read labels. Once the GPT reached a satisfactory level of performance, testing was conducted with visually impaired participants recruited through Satakunnan Näkövammaiset ry.
The findings demonstrate that large language model tools can be adapted for practical assistive purposes, offering new opportunities for enhancing accessibility in everyday tasks. At the same time, the study highlights significant limitations, including accessibility barriers within the ChatGPT application itself, and the issues with dependency on external platform design choices outside of the developer’s control.
The thesis concludes that while GPT-based tools show promise for supporting visually impaired individuals, close collaboration with accessibility communities is essential in their development. These results contribute to the understanding of AI-driven assistive technologies and underscore the importance of inclusive design in shaping the future of AI-powered accessibility solutions.