How to control iPhone or iPad with your eyes using Eye Tracking

Learn how to set up and use the built-in Eye Tracking feature in iOS 18 and iPadOS 18 to navigate around your iPhone or iPad with just your eyes — no additional hardware or accessories required!

Eye Tracking is a new accessibility feature on iPhones and iPads for people with physical disabilities who are unable to interact with their devices through touch.

Eye Tracking utilizes your iPhone or iPad’s front-facing camera and artificial intelligence to determine where you are looking on the screen. It then takes action, like highlighting an element, registering a tap for that highlighted item, swiping down, and more, all by just looking at the screen.

This feature is eerily similar to eye tracking on Apple Vision Pro.

Note that iOS 18 and iPadOS 18 are currently in the beta stage, and you can get them by following our simple tutorial.

Tip: When setting up and using eye tracking, keep your iPhone or iPad on a steady surface at a distance of 1 to 2 feet. You can place your device on a phone stand or iPad’s foldable cover.

Set up and activate Eye Tracking

1) Open the Settings app on your iPhone or iPad and tap Accessibility.

2) Select Eye Tracking from the list and turn on the Eye Tracking switch.

3) Go through the setup process. This involves looking at all the dots one by one. Avoid blinking during the process.

After you finish the setup:

• Turn on Dwell Control. This lets you tap the selected element by looking at it for a while.
• Snap to Item is another important setting that highlights the item you’re looking at with a box. I recommend leaving it on.
• Auto-Hide lets you set the duration before the pointer reappears. You can pick from 0.10 seconds to 4 seconds. It also enables you to adjust pointer visibility when you are not using it.

Using Eye Tracking

With Eye Tracking activated, look at the elements/items inside the app, and your iPhone or iPad will highlight it. Keep looking at that element for a while to trigger a tap/touch.

You can perform vertical or horizontal swipes and several other actions by looking at the AssistiveTouch button (which becomes visible on the screen after enabling Eye Tracking) and selecting the Scroll option.

If you do not see Scroll in AssistiveTouch, go to iPhone Settings > Accessibility > Touch > AssistiveTouch > Customize Top Level Menu and add it.

Remember that AssistiveTouch is an important helper for Eye Tracking, ensuring you can do more things just by looking at your iPhone or iPad. So, take a moment to customize AssistiveTouch the way you want.

Set up four Hot Corners

You can use Hot Corners on Mac without activating accessibility features. However, you can set up and use Hot Corners on iPhone and iPad by enabling AssistiveTouch and Dwell Control accessibility options.

With Hot Corners and Eye Tracking activated, you can look at that corner of your device screen to perform that set action. For instance, I have set the lower right corner to go Home. So, when I look at the lower right corner for a while, it automatically takes me to the Home Screen.

Go to iPhone or iPad Settings > Accessibility > Touch > AssistiveTouch and tap Hot Corners. Now, assign important actions to the Top Left, Top Right, Bottom Left, and Bottom Right corners.

Note that you can also use these Hot Corners with a mouse or trackpad.

Eye Tracking can be challenging to use

Apple Vision Pro has incredible eye tracking, thanks to an advanced eye‑tracking system made up of LEDs and infrared cameras. iPhones and iPads do not have such sophisticated tracking cameras, so they use the front-facing camera and on-device intelligence for Eye Tracking.

While impressive in theory, this is far from perfect. iPhone and iPad often fail to figure out the element you are looking at. You will have to keep struggling, making all sorts of eye and head movements, hoping the system highlights the item you intend to. In my experience, it seems a lot of work needs to be done to make it usable.

If Eye Tracking feels terrible (which it will), try turning it off and back on and go through the setup again.

Additionally, I have experienced that Eye Tracking works better if I keep the device further away from my face. While you’re at it, don’t forget to keep your iPhone or iPad still on a table or stand.

Get dedicated MFi eye-tracking devices

If the above-mentioned built-in Apple Eye Tracking isn’t good enough, you have the option to purchase dedicated Made for iPhone/iPad (MFi) eye tracker hardware devices and accessories that work with your iOS device. You can explore options like eyetuitive and TD Pilot and set them up in Settings.

iPhone Mirroring and Eye Tracking

You cannot use iPhone Mirroring to see and control your iPhone from your Mac when Eye Tracking is on. This is because the front iPhone camera is always in use when Eye Tracking is activated. It stays active even when you lock the device. And one of the necessary requirements for iPhone Mirroring to work is that the iPhone should not be in use.

Turn off Eye Tracking

There is no way to temporarily turn off Eye Tracking on an iPhone or iPad from Control Center. You won’t find the Eye Tracking button when editing the new Control Center. You will have to go to Eye Tracking in Settings app > Accessibility and turn it off entirely. If you turn it back on, you must redo the setup process!

All your Eye Tracking data stays on your device

Apple says that Eye Tracking uses on-device machine learning. Eye Tracking setup and other usage data are stored locally on the device and are not shared with anyone, including Apple.

On a related note:

• How to control your iPhone or iPad with head movements
• How to use your Mac camera to control the cursor with head movement

Source link: https://www.idownloadblog.com/2024/07/10/how-to-use-eye-tracking-iphone-ipad/