Be My Eyes and See4Me - two new 'lend-me-an-eye' apps for blind people

3. June 2014

With Be My Eyes and See4Me, two very similar helper apps for blind people are currently under development. Both apps aim to lend blind users the eye of a sighted person.

This is how it works in a nutshell: The blind user captures an object with the smartphone camera and sends a request to a community of helpers. One helper accepts the request and responds by explaining the object.

Improving on TapTapSee

The popular iPhone-App TapTapSee has a similar approach. Here, photos are matched to a large database of images, but it stands to reason that human help also contributes when hard-to-identify objects are captured. Identifying 100 images costs 7€ – alternatively the user can pay 9€ a month to use the service without an upper limit on the number of images.

Both Be My Eyes and See4Me intend to build a community of volunteer helpers to be able to offer the service free of charge.

A bit of history: KlickBlick

The approach has already been used in Germany long before the availability of smartphones. Joachim Frank's application KlickBlick / See4Me allowed users to capture objects via a webcam or scanner and access a pool of sighted users via a standard internet connection. At the time of development of KlickBlick, the service was stationary and the idea didn't take off.

Apps can "lend an eye" on the road

With the availability of smartphones with LTE connections, much more has become possible. The service can now become fully mobile (at least where LTE is available). Use cases include getting descriptions of signposts or information displays, identifying sell-by dates on food, or crossing busy streets.

How it works

Both apps use the smartphone's video function and establish a peer-to-peer connection. Users can either access to the pool of volunteer helpers, or address specific requests to known helpers on a buddy list.

A typical process sees the user launch the app, start taking a video image of the object or scene in question, and send a request for assistance to the community (or rather, 10 randomly selected community members). Any helper who is currenty free to help can opt to accept the request.

Via the audio channel between helper and app user, the helper can give instructions such as "turn a bit left", "get closer", or "turn the thing around". This feedback in turn changes the video signal and improves the quality and precision of the description offered by the helper.

An important use case would be 'gap navigation' in door-to-door navigation scenarios. The app can support users on initial or final legs of public transpiort trips that are not (sufficiently) included in general purpose apps like the German mobility app DB Navigator.

The pool of helpers

The pool of helpers starts with friends and acquaintances of blind people and grows as more blind users are added. Volunteers need not worry that they will be overburdened with requests of assistance - they can at any time use the "snooze" mode to set their status to unavailable. In an international application scenario, language settings will ensure that only those helpers will receive requests that will answer in the language(s) selected by the user.

Contact and launch dates

Be My Eyes is a Danish development by Hans Jørgen Wiberg with the help of Danish app developers. The app is open source, the code is available at Github. The iPhone app is set to lauch in August 2014, an Android app is set to follow later. An international launch is also intended in the future. More info and a mail registration to be informed of the launch is available at the Be My Eyes website.

See4Mee is being developed by Joachim Frank, Verein KlickBlick Plus e.V. A simultaneous launch date for both iOS and Android apps is envisioned around the end of 2014. More information about the development is available at the KlickBlick site.