Apple announces new accessibility features, including eye tracking

Every attention thief is absolutely thrilled at the idea of tracking your eyes. Let’s all imagine the day where the YouTube free tier pauses ads when you’re not actively looking at them.

Shit. I’m turning into one of those negative downers. I’m sorry. I’ve had too much internet today.

I don't love that solid UX gets pushed under the accessibility rug, as an option you might never find.

I don't care how cynical it sounds, user experience became user exploitation a long time ago. Big Tech have been running that gimmick at too-big-to-fail scale for the last decade or so.

I often use them to get around bad UI/UX (like using Reduce Motion), or to make devices more useful (Color Filters (red) for using at night).

Even outside of this, even able-bodied folks can be disabled due to illness, surgery, injury, etc. So it's great to see Apple continuing to support accessibility.

> developed solely with the benefit of the user in mind

Hopefully accessibility features are never artificially segmented to higher priced devices.

Funny, I was just thinking it was so that they can get more attention-economy eyeballs for ads.

That’s a profound and surprising insight. You’re absolutely correct.

Wouldn't a lot of the companies that build in accessibility do it from a viewpoint of gaining an even wider reach and/or a better public image?

I don't see optimizing for that as bad. If they think we'll love the product more by making it better for a given audience, especially if I'm in that audience, I'm happy. Does that mean this company now gets richer? Perhaps, and that's fine by me

Accessibility features can be used to steal attention too

Accessibility features stand out as user-centric developments, love that

Have you noticed any correlation between how hungry you are and how fast motion sickness kicks in?

Vaguely related anecdote.

I used to get bad nausea from aggressive physic-y VR games. But I heard people claim it can be grinded through. So I did that, and they were right. I can do VR games without needed to vomit although it’s still uncomfortable.

However… I am now much more sensitive to non VR motion sickness. :|

A similar app that's on Android: https://play.google.com/store/apps/details?id=com.urbandroid...

Unsure if it actually works though, my personal test results are mixed.

I have motion sickness... It's so hard to movearound for me and I am still not able to find what works best for me

I aimed for the upvote button but they’re so tiny that my fat finger hit the downvote button by accident and then I had to retry the action. This is what people mean by accessibility is for everyone all of the time.

See also https://en.wikipedia.org/wiki/Curb_cut_effect

The image description stuff is already surprisingly good - I noticed when I got a photo text while driving and it described it well enough for me to know what it was.

My friends that use synthetic voices prefer cleanliness of the older and familiar voices. One friend listens at about 900 WPM in skim mode and none of the more realistic voices work well at those rates.

I'm hoping that, one day, we get a tutorial for VoiceOver

Maybe it's not feasible for you but if you're ever near an Apple Store, you could definitely call them and ask whether they have an accessibility expert you could talk to. In the Barcelona Apple Store for example, there is a blind employee who is an expert at using Apple's accessibility features on all their devices. He loves explaining his tips and tricks to anyone who needs to.

I agree. I have a last gen base iPad with Touch ID and a home button. I am pretty tech savvy but actually prefer this form factor.

When there were physical buttons, it was very popular in Asia to enable the accessibility option to put a virtual one on screen instead, because they were afraid the physical one would break. So it was kind of useless in the end.

It’s not physical, but if the problem is they need something visible rather than a gesture, then you can put an always-on menu button on the screen that has a home button in.

https://support.apple.com/en-sg/111794

CarPlay devices aren't really powerful.

Is this really likely to be downstream of the Vision Pro implementation? I would think that eye-tracking with specialized hardware at a fixed position very close to the eye is very different to doing it at a distance with a general purpose front facing camera.

It would be amazing if it gets carried over to the Mac.

Accessibility features tend to be superpowers though, and I'm glad Apple gates them behind permissions and opt-ins. We all know of applications who try to trick the user into granting them inappropriate access to the device through the Accessibility APIs. I think DropBox still begs you to grant them Accessibility access so its tendrils can do who-knows-what to your system.

With great power comes great responsibility.

My favorite is the "allow ANC with just one AirPod in". I have no idea why this would be an accessibility feature. If I turn on ANC, then I don't want it to be disabled just because I'm listening with one ear!

Well, they aren't really limited to accessibility, but they are hidden there. It's sort of like a convenient excuse to get UI designers off your back if you want to ship customization.

FYI you can also make turbo night shift by scheduling toggling of white point balance, yep, in accessibility settings

Eye tracking is absolutely an accessibility feature. Just because you don't need it, and it can be abused, does not mean it isn't an absolutely game changing feature for some people.

My presumption is that apps will not be able to access this data, at least without some sort of permission gate.

Apple are very good about not making this kind of thing available to apps that don't have an explicit reason to need it.

You can say a lot of negative true things about Apple but this is just silly. There is no way Apple is going to expose that data to underlying apps in the same way they refused to do it in Vision Pro. I'd bet a good bit of money it works the same way where it's a layer on top of the app that the app can't access and from the video it looks like that's exactly how it works.

It allows people with ALS to navigate their device with their eyes.

Microsoft added a similar feature to Windows about seven years ago.

The accessibility features are also useful for automated UI testing.

They’ll make features like this because it’s the right thing to do

While the positivity is great to see -- I'd temper that expectation they're simply doing the right thing and definitely acting in their perceived interest. People can and often do the right thing -- large companies rarely do.

This was the right thing to do, but I doubt “the right thing to do” was the primary motivator for it. This is smart marketing that helps position them as the more caring, inclusive, and capable tech brand. It also expands their reach more than a fraction of a percent. 13% of Americans have some form of disability.

Do you realize how bad Apple's accessibility issues were before? This is just marketing and I doubt many people are going to drop thousands of bucks to ditch years of tooling they're already intimately familiar with (aka, Windows), but this is an effort to try and entice some people. That's all it is. Marketing.

I read this very skeptically.

When I hear eye tracking I immediately think of advertisers targeting me based on what I look at, NOT quadriplegics using Apple devices.

Maybe I'm a cynic

I'm hoping they'll add this to macOS too. Eye tracking would be great for end-user UX research.

I'd also like to see what sort of weird stuff people come up with to use eye tracking. Games could use that in interesting ways.

Well then it's both?

Where are you getting this language point from? If I look up any company selling "eye trackers", their products are all meant to track where you're looking, e.g., https://www.tobii.com/

This is how I feel about "face recognition" (it should mean recognizing whether something is a face or not), but it is common to use eye tracking this way.

I think it's more of "clearing the decks" for stuff that didn't make the cut for WWDC. I assume WWDC is going to be all about AI and they couldn't find a good spot to put this announcement. "Clearing the decks" isn't a very kind way to refer to this accessibility tech since Apple has always been better than almost everyone else when it comes to accessibility. I don't see this as "we don't care, just announce it early" as much as "we can't fit this in so let's announce it early".

Tomorrow (today in some timezones) is Global Accesibility Awareness Day: https://en.m.wikipedia.org/wiki/Global_Accessibility_Awarene...

They do it every year at the same time. Also, it’s a small announcement, not a keynote or the kind of fanfare we have at WWDC or the September events. This does not seem calibrated to be effective in an advertising war with another company. All this to say, probably not.

As Terramex pointed out this is tied to a particular, relevant event.

It’s also pretty common for Apple to preannounce some smaller features that are too specialized to be featured in the WWDC announcements. This gives them some attention when they would be lost and buried in WWDC footnotes.

It is also probably an indication that WWDC will be full of new features and only the most impactful will be part of the keynote.

It's better to have an easy way to hold a mouse button via keyboard with your left hand and continue to use one finger on the touchpad rather than do the whole three fingers to drag

The default way to drag on a trackpad is something I’ve never gotten good with so I always enable drag lock the second I get a new laptop. Ideally I would switch to the three finger gesture but after 15 years of drag lock I just can’t get my brain to switch over.

three finger dragger for life here. whenever I see colleagues struggling to drag items on a Mac, I show them three finger dragging and it blows them away. total game-changer!

I'm severely hearing impaired and enjoy going to dance classes - swing, salsa, etc. If I'm standing still, I can easily tune into the beat. But once I start moving, I quickly lose it on many songs; dance studios aren't known for having large sound systems with substantial bass. I don't know that this specific setup would fix anything -- it would need some way of syncing to the instructor's iPhone that is connected via bluetooth to the studio's little portable speaker. But it's a step in the right direction.

While on the topic, I can hear music but almost never understand lyrics; at best I might catch the key chorus phrase (example: the words "born in the USA" are literally the only words I understand in that song).

A few months ago I discovered the "karaoke" feature on Apple Music, in which it displays the lyrics in time with the music. This have been game changing for me. I'm catching up on decades worth of music where I never had any idea what the lyrics are (filthy, that's what they are. So many songs about sex!). It has made exercising on the treadmill, elliptical, etc actually enjoyable.

There's a lot of interesting things we can do with haptics since they're relatively cheap to put in stuff. Hopefully accessibility gets the software and applications further along soon

Using haptics in music to enhance rhythm perception and dance skills. Sounds really cool!

VisionOS has this too. I mapped it to a triple click of the button for when eye tracking becomes inaccurate.

Cool. I'm a bit unsettled that my camera's green dot didn't turn on for it though.

Cool! That works surprisingly well. But how do you click while using this? Clicking on the trackpad doesn't work, when it's tracking my head.

Sure it means something, it’s pretty low on the PR talk scale even. Let’s see:

We believe deeply in

"We", in this case, refers to Apple and as a company can’t believe in anything, it specifically refers to their employees.

the transformative power of innovation

A new feature will bring forth change

to enrich lives

In order to make someone’s life better.

So far so good, at least no synergies being leveraged for 10x buzzword transformation. Let’s see if it passes the test: There’s a bunch of new accessibility features, that certainly fits the bill for innovation. As anecdata, at least one of these will drastically change how I interact with my devices, so there’s the transformative power. I will be able to use these new accessibility features to improve both the way I work and how I privately use technology, so one could argue my life is being positively enriched by this transformative power brought about by innovation. Do they "believe deeply" in this? That is only for Tim Cook and his employees to know, but they’re one of the few companies pushing advances in accessibility past the finish line, so they might!

I always wonder why press releases include claims that a person said a paragraph or more of text that noone would ever say out loud, but this one actually does sound like something he'd say. In a keynote video, anyway.

Don't be ridiculous. Solid hardware and software combos for windows cost a small fraction of that. The convenient and decent Tobii PCEye costs like $1,250 and a very nice TMS5 mini is under $2,000. Your bullshit was off by at least an order of magnitude.

Deaf people I know certainly wouldn't want to be wearing a headset, say, in a restaurant. But a phone app or tablet that can do live text-to-speach would be good enough in many cases if it can separate voices into streams. Anything like this available?

Do you have another idea beyond the accessibility of Users without fine motor control to the phone's screen?

of course it’s surveillance technology.

an iphone constantly broadcasts your location to third parties, can deduce where you work and live, understands the unique shape of your face, has a built-in microphone and multiple kinds of cameras, stores all of your conversations, stores all of your photos, stores your health information, can figure out when you go to bed.. all on a completely closed and proprietary operating system.

it’s like asking “why hasn’t anyone mentioned that we’re all using a website right now”

https://support.apple.com/guide/mac-help/use-your-keyboard-l... - Keyboard navigation

https://support.apple.com/guide/mac-help/navigate-your-mac-u... - Full Keyboard Access (accessibility feature, goes beyond just tabbing between elements)

It's annoying that tabbing between UI elements is off by default on macOS. It's one of the first things I turn on with a new mac.

I share the sentiment. I've long noted in situations with grids of terminal windows where focus-follows-eyes would be so much faster and present less friction than using hotkeys or focus-follows-mouse.

Surely, this would be a headphone feature possible without having to be supported by the player.

It took me a couple read-throughs to understand what you meant, but yes, on-screen gaze-sensitive magnification would be amazing, I agree.

It's basically how the Apple Vision Pro mainly works.

I was thinking the same, it would make having an iPad under my monitor much less cumbersome

At least I put off the phone while I was in the car. Not the case now. Thank you Apple but I'd rather be sick while looking at your phone in a car.

Yes

I know this is a joke, but for for anyone who was wondering,

all data used to set up and control this feature is kept securely on device, and isn’t shared with Apple.

It's not an API adopted which apps can use, it's a system pointer, like a mouse or trackpad