The L-CAS Office dataset is created on purpose for the ActiVis project. The pictures are taken from a real office in our university and it contains 17 object images acquired with a mobile phone (Asus ZenFone AR).
If you use this dataset, please cite the following paper, where you can also find more details about it:
@InProceedings{Terreran2020,
author = {M. Terreran and A.G. Tramontano and J.C. Lock and S. Ghidoni and N. Bellotto},
title = {Real-time Object Detection using Deep Learning for helping People with Visual Impairments},
booktitle = {Proc. of the IEEE Int. Conf. on Image Processing, Applications and Systems (IPAS)},
year = {2020}
}
The Virtual Cane app is our attempt at replacing a normal walking cane with a virtual one. Here, the user points the Tango's camera in his walking direction and if there is an oncoming obstacle, the app uses the depth data from the Tango's camera and informs the user of the distance to the obstacle by activating the vibration motors of the device. The vibration intensity is a function of the distance to the obstacle, so the device will vibrate more intensely the closer the obstacle is.
There is a little dot in the middle of the screen that sticks to the last closest location detected in the middle of the camera and changes colour to indicate the distance to the object; this distance is given on the bottom-right of the screen. There is also a button to activate a visualisation of the point cloud data.
The target search app uses sound cues to instruct a user to point the Tango's camera in a certain target direction. This target is presented on-screen as a yellow smiley face floating in an augmented reality setting.
The audio varies in two ways to indicate the pan and tilt angle adjustments that must be made to the camera's orientation to point it toward the target. The tilt angle is conveyed by adjusting the audio pitch: a high pitch indicates that the target is higher than where the camera is currently being pointed and a low pitch tone indicates that its below. The pan angle is given by spatialising the tone to make it sound like its coming from the user's left or right, indicating that the target is to the left or right of the camera's current pointing direction respectively.
A proof-of-concept app that will use the interface we developed earlier to guide the user to a target location in 3D space. The target in this case is a set of QR Code marker placed somewhere in a room that simulate real objects. The app uses an internal representation of its understanding of different objects' relationships wrt each others' location and generates guidance instructions based on that.
An extension of the marker search app that uses the same internal knowledge of object relationships to provide guidance instructions. However, in this case a real object detector, based on the MobileNet V2 was implemented to work in conjunction with the guidance module.