Δημοσιεύσεις

The proposed application builds on the latest advancements of computer vision with the aim to improve the autonomy of people with visual impairment at both practical and emotional level. More specifically, it is an assistive system that relies on visual information to recognise the objects and faces surrounding the user. The system is supported by a set of sensors for capturing the visual information and for transmitting the auditory messages to the users. In this paper, we present a computer vision application, e-vision, in the context of visiting the supermarket for buying groceries.

The latest advancements in computer vision have enabled the development of new applications in numerous scientific and technological fields. The program “e-Vision” builds on these advancements with the aim to improve the autonomy of people with visual impairment by means of visual recognition and audio feedback. This is achieved by a set of sensors, properly embedded to a pair of glasses for capturing visual information and for transmitting auditory feedback messages. Appropriate software running on a mobile device (smartphone) is responsible for transmitting and receiving the audio and visual information to and from the sensors. This paper presents a comparative study of relevant wireless communication protocols that could be implemented for the communication of the sensors with the smart mobile device. The application demands certain criteria to be met, regarding data transmission rates, transmission range, and power consumption. The current study focuses on the comparison of the numerous existing wireless communication protocols. It is shown that Bluetooth and WiFi are the most advantageous in order to implement the audio and visual data transmission respectively.

Computer vision-based assistive technology for the visually impaired is still a field of ongoing research. Its fundamental scope is to extend the frontiers of visually impaired by means of providing a greater degree of independence and autonomy in their daily living activities. Towards this direction, we present “e-Vision”, a hybrid system that couples the convenience and the inherently seamless adoption of an external camera embedded within a pair of eyeglasses with the processing power of modern smartphone devices. The system consists of a pair of eyeglasses integrating a camera and a mobile application that encapsulates computer vision algorithms capable of enhancing several daily living tasks for the visually impaired. The proposed system is a context-aware solution and builds upon three important day-to-day activities: visiting a super-market, going an outdoor walk, and carrying out a work at a public administration building. Going one step further, “e-Vision” also caters for social inclusion by providing social context and enhances overall experience by adopting soundscapes that allow users to perceive selected points of interest in an immersive acoustic way.

The autonomy of the visually impaired, expressed by their ability to accomplish everyday tasks on their own even when help by others is not available, is of paramount importance. Hence, the rapid growth of computer vision in the last decade has given rise to a large number of assistive applications aiming to help the visually impaired in perceiving the world in a similar way to the seeing ones. In this study we investigate the usability of a hybrid system, named e-Vision, that couples the natural and seamless adoption provided by an external camera embedded on a pair of glasses with the processing power and the penetration rate of modern smartphone devices. The ultimate benefit which e-Vision hopes to bring to the visually impaired is greater autonomy, and the increased wellbeing. To assess e-Vision’s performance, a month-long pilot study took place and people with actual visual impairment used the system in their daily lives. This procedure enable the system’s evaluation under real conditions. Although the e-Vision’s evaluation provided indications for a promising system, the obtained results were bellow our expectations with respect to practical usage. Since the employed computer vision modules were based on state-of-the-art deep learning models capable to achieve top-level performance, we identified the shortcomings and the limitations that typical computer vision-based practices set to the creation of assistive technologies for the visually impaired. Therefore, we propose potential remedies capable of overcoming the identified obstacles in existing practices.

Product recognition is a task that receives continuous attention by the computer vision/deep learning community mainly with the scope of providing robust solutions for automatic checkout supermarkets. One of the main challenges is the lack of images that illustrate in realistic conditions a high number of products. Here the product recognition task is perceived slightly differently compared to the automatic checkout paradigm but the challenges encountered are the same. The setting under which this dataset is captured is with the aim to help individuals with visual impairment in doing their daily grocery in order to increase their autonomy. In particular, we propose a large-scale dataset utilized to tackle the product recognition problem in a supermarket environment. The dataset is characterized by (a) large scale in terms of unique products associated with one or more photos from different viewpoints, (b) rich textual descriptions linked to different levels of annotation and, (c) images acquired both in laboratory conditions and in a realistic supermarket scenario portrayed in various clutter and lighting conditions. A direct comparison with existing datasets of this category demonstrates the significantly higher number of the available unique products, as well as the richness of its annotation enabling different recognition scenarios. Finally, the dataset is also benchmarked using various approaches based both on visual and textual descriptors.