Apple has introduced a new way to control your iOS devices, fentirely with your voice. This new feature can benefit several people with disabilities, especially those who have difficulty using their hands including tremors, or those with vision disability, or multiple disabilities. In a nutshell, voice control makes it possible to use the device completely with voice commands and not having to touch it even once.
Here’s a video to understand Voice Control
Somethings are better experienced, then explained. We thought this video from apple summarizes key functionalities of Voice Control. And, yes this video has audio description.
https://www.youtube.com/watch?v=v72nu602WXU
Fascinating, isn’t it?
Here are the salient features
Let’s try to understand this further by deep diving into some of its salient features:
- Comprehensive navigation with voice commands: You can rely entirely on your voice to navigate an app. Comprehensive navigation is provided by navigation commands, names of accessibility labels, numbers, grids, voice gestures and recorded commands. Take for instance:
- Navigation commands give you quick ways to open apps, search the web, press the Home button and more.
- You can easily navigate by telling Voice Control to select the name of an accessibility label for buttons, links and more.
- Say “show numbers” to see numbers appear next to all clickable items onscreen. Use this to quickly navigate complex or unfamiliar apps. Numbers automatically appear in menus and whenever you need to disambiguate between items with the same name. Just say a number to click it.
- Saying “show grid” superimposes a grid on your screen and allows you to precisely do things like tap, zoom, drag and more.
- Use your voice to perform gestures, such as tap, swipe, pinch, zoom, press the Home button and more.
- You can record multistep gestures for apps on your iOS device. So, if you love to send messages with fireworks, you can record the gestures to do this and use the recording to quickly send messages with fireworks.
- Supports text editing: Voice Control supports rich text editing commands. This allows to make corrections. It is quick and easy.
- You can replace phrases by name. Try saying, “Replace I’m almost there with I just arrived”.
- Fine-grained editing also makes it simple to select text. Try saying, “Move up two lines. Select previous word. Capitalise that.”
- If you need to correct a word, simply ask and you’ll be presented with a list of suggested replacements.
- Custom words: Voice control gives the ability to add custom words to ensure that Voice Control recognises the words you commonly use.
- Truly smart: Voice control uses state of art technology such as artificial intelligence, natural language processing that makes it extremely accurate, context aware and safe to use. Take for instance:
- According to Apple, “Voice Control uses the Siri speech recognition engine to give you the latest advances in machine learning for audio‑to‑text transcription”. This makes it extremely accurate. (Please note, as on September 20, 2019, Voice Control supports U.S. English)
- Voice Control understands contextual cues, so you can seamlessly transition between text dictation and commands. For example, say “Happy Birthday. Tap sends.” in Messages, and Voice Control sends “Happy Birthday” — just as you intended. You can also say “Delete that”, and Voice Control knows to delete what you just typed.
- With Attention Awareness, Voice Control goes to sleep when you turn your head away from the TrueDepth camera on an iPhone. It doesn’t activate until you look back at the screen — so you can talk to a friend nearby without affecting your device.
- Local audio processing: All audio processing for Voice Control happens on your device, ensuring that your personal data is kept private.
Getting started with voice control?
By default, Voice Control is turned off. You can either tell Siri to open settings for Voice Control or navigate to Settings > Accessibility > Voice Control. Please note, when you select to turn Voice Control on, before using it, you will need to be on WIFI to download 250MB of data so that Voice Control can work regardless of whether your iOS Device is connected to the internet or not.
Understanding, Voice Control menu options
As you enter Voice Control, you will find several menu options. Further along in this menu, you will find several buttons: Customize Commands, Command Feedback, Vocabulary, Show Confirmation, Play Sound, Show Hints, Overlay, and have the option to help Apple improve Voice Control. A brief explanation of each option follows.
- The first option will explain what Voice Control is. The next options are the language that is selected.
- Customize Commands: The Customize Commands option allows you to not only customize what will cause Voice Control to perform a specific function but will also allow you to create your own. Selecting each option from this menu will allow you to see a full list of what commands are available. Within each of the commands, you will also find the ability to turn that specific command off if you wish.
- Vocabulary: This is a section where you can type in a word or phrase and have it been recognized by Voice Control.
- Show Confirmation and Play Sound: These are features which will show or play a sound to confirm that Voice Control has heard your command.
- Hints: This provides useful hints to users while using Voice Control. It is very useful in early days of using Voice Control. For example, I wanted to select the Mail app with Voice Over enabled. I said, “select Mail”. I was instructed to say “Voice Over, select Mail”. This suggestion selected the Mail app as requested.
- Overlay: It is an option which can speed up your voice interaction depending on your use case. You can select to use numbers or names which will then be displayed over the contents of your screen. With Voice Over enabled, if you say “select 23” for example, you will get a read-out of the number of the various elements on your screen and the associated number. Instead of saying “select 23” I would just say “23” and this would select the designated option for that number. The same is true with the names overlay. The Grid Numbers overlay is also an option, which seems to behave the same using Voice Over. This menu also allows you to control how long the overlay will remain on screen through a feature called “Automatic Dimming”. There is also an option to control the opacity of the overlay.
Finally, Voice Control & Voice Over- can it work together?
The good news is that voice control works well with voice over. It’s nice to see that speech from Voice Over through the internal speaker doesn’t appear to interfere with Voice control very often. However, users from Applies community reports that turning speech off completely, makes it work even better with almost 0 false positives, especially when using braille support.
Check out this related resource
For more detailed examples of how to use Voice Control as a Voice Over user, I recommend checking out the following resources:
- the AppleVis podcast done by Thomas Domville.
- Apple Support YouTube Video on How to navigate with Voice Control on your iPhone – https://www.youtube.com/watch?v=80AyUCjZYZM
- Apple Support YouTube Video on How to use dictation and edit text with Voice Control on your iPhone: https://www.youtube.com/watch?v=k-ykg3jZ9_k