Interface issues when using tablets with screen readers

31. January 2014

There a number of interface issues for blind people using touch devices. We try to generalise them here, drawing on examples from our user tests and the technical tests of the calendar apps.

This is a part of our comparative test of tablets for blind users. An introduction is provided in Tablet test with blind users: Overview. Another related arcticle covers problems related to gestural input: Blind people using touchscreens: The issues.

More details can be found in the technical tests of the three calendars:

Issues are listed in the approximate order of impact, starting with show-stoppers and ending with annoying issues that can mostly be worked out in use.

Controls that cannot be operated

Some touch controls canot be operated when the screen reader is active. An example is the time selection in the Nexus calendar event screen. Even if you know or find out the right gesture (a tricky circular dial movement with two fingers) is impossible to operate reliably in non-sighted use. This one problem calls into question the utility of the Nexus calendar for blind users.

Non-focusable areas / elements

Touch interfaces often show gaps where elements that respond to touch when the screen reader is off are not focusable via swiping or via-touch-explore when the screen reader is on. This problem was mainly found in the Nexus and Windows 8 calendars.

  • Nexus 7: Days in the month view of the calendar cannot be focused (just entire weeks, but without language output). Adding events by double tapping on individual days is theoretically possible but hardly practical for blind users since the day is not announced on touch-explore.
  • Windows 8: Days in the month view cannot be focused by touch explore (unless they contain previously entered events). Swiping between days only works when a day is initially focused, for example, the current date when opening the calendar. Occasionally, important elements are skipped, for example, the element "Save Changes" in the Edit event view, or Narrator does not speak the accessible name when focussing elements. This behaviour seems to change unpredictably (maybe dependent on prior navigation states) and cannot be reliably reproduced.

Non-modal design of dialogues and pop-up controls

Pop-up dialogues used in apps (e.g. in the calendar) often only take a part of the screen area, leaving other areas visible in the margins. While operating these pop-ups is usually possible with swipe gestures that can be performed anywhere on the screen, the likelihood that a local touch is registered accidentally means that blind users very often lost the focus while interacting with a pop-up control, meaning that the focus had to be laboriously rediscovered through touch exploration or swiping.

In a modal design approach, marginal areas outside the pop-up would be disabled, requiring the completion of an interaction with the dialogue that pops up on top. This was especially negative with the Nexus during testing, but has improved with the update to Android 4.4 and/or a recent version of Calendar (201308023).

The iPad at least offers the hint "Dismiss pop-up – Double-tap to dismiss pop-up window". On closing the pop-up, the focus moves back to the beginning of the screen, not back to the triggering element ("Add").

A similar problem exists with date selection in creating a calendar entry in the Windows 8 calendar. An accidental touch outside the the pop-up defocuses it, and announces "close menu - double-tap to activate", i.e., without explicitly identifying the element that is to be closed. Swiping continues to focus positions underneath the pop-up. Refocusing the pop-up required finding it with touch-explore. Here, the focus moves back on the trigger so the widget can be opened again immediately.

Badly named or labelled controls, gaps in voice output

Some very important elements are not clearly labelled, for example the elements called "Add" (iPad) and "New" (Windows 8). Both create a new calendar entry. Nexus has the best label with "New event". Users looking for the function to create a calendar entry repeatedly missed these important elements. This is mainly a problem for novice or occasional users. Another example both in the Nexus and iPad calendars: The value of hour and minute fields when setting the event time is read, but not labeled hour and minute to aid differentiation.

Scarcity of instructions

Instructions or hints provide guidance as to how a contol can be manipulated.

  • iOS/VoiceOver usually provides good and clear instructions. Usually type and name of object are announced as well as hints for interaction. Take, for example, the control for setting date and time in the calendar: "Today - picker item, adjustable. Swipe up or down with one finger to adjust the value."
  • Android/Talkback often announces controls without hints. Sometimes, hints / instructions are provided, but not consistently.
  • Windows 8/Narrator: Instructions generally more comprehensive compared to Android, but at times with odd grammar.

Controls without role

Google Nexus: A number of elements provide no output of their role when receiving focus. Non-interactive elements (e.g. the heading "New event" in the event creation screen) and interactive elements (e.g. Cancel, Done) are read in the same way, i.e. interactive elements do not reveal via their role what actions they afford (e.g., button, text input).

Controls without value

Google Nexus: A number of elements reveal the role but not the currently set value. E.g., when focussing time zone in the Nexus calendar Talkback reads the label time zone, then, on swiping further, button - time zone, but not the currently active value. The same is true for the button repeat event and reminders: the role is read, but not the value.

Focus management issues

When elements trigger modal pop-up widgets, closing these widgets should move the focus back to the triggering element.

  • Nexus 7: Focus management does not work properly in the Nexus calendar for the many pop-up widgets on the new event screen – the TalkBack focus always moves back to the start of the screen when closing widgets, necessitating tedious swiping or touch-explore to regain focus.
  • iPad mini: When an accidental touch takes the focus away from the pop-up for entering an event (leading to the instruction that double-tapping will close the pop-up) swiping returns the focus to the beginning of the pop-up, not to the element that was previously focused.
  • Windows 8: Focus handling is generally quite reliable in the Windows 8 calendar. Closing pop-ups generally restores the focus on the triggering element.

Multiple focus points for one control

This problem was found both in the Nexus as well as the Windows 8 tablet. In the Nexus calendar, swiping runs through redundant focus points such as "what - edit event name - where - edit location". The windows tablet has even three focus points for some of its controls, for example when adding the duration of an event. First, swiping focuses the label "Duration" which has no associated action, next an popup menu, and finally a button. Activating the latter two basically does the same thing (show the pop-up menu). Strangely, there is occasionally just one focus point for the element, which would be better, but as so often with the Windows 8 calendar app, the behavior is unpredictable and cannot be reproduced at will.