An exploration of Apple business news and technology. We talk about how businesses can use new technology to empower their business and employees, from Leo Dion, founder of BrightDigit.
Leo Dion (00:00)
Hello and welcome to another episode of EmpowerApps. I'm your host, Leo Dion. And today we're joined with Rob Whitaker. Rob, thank you so much for joining us today.
Rob Whitaker (00:11)
Hey Leo, great to be here.
Leo Dion (00:14)
before we begin, I'll let you do your introduction.
Rob Whitaker (00:19)
Sure, my name is Rob. I am an iOS engineer primarily, but my specialism is in accessibility. And I've done a little bit of Android and I've done native accessibility on macOS and Android and iOS and across all those platforms. So yeah, I'm excited to talk about some accessibility with you.
Leo Dion (00:41)
Awesome. So let's just jump right into it. how did you I actually don't know this. How did you get your start in accessibility?
Rob Whitaker (00:51)
I really started with mobile accessibility through the same way that I started making iOS apps essentially. I had been making iOS apps as a hobby with some local businesses. I'd been helping them out making some some applications for them. and someone locally was a user of one of those apps. He knew that I was local.
Rob Whitaker (01:13)
and he sent me a message through the app and said, Look, I'm I'm a blind user. I use this app that you've built, but I'm struggling to do these few things with it. I know you're local. Do you want to meet up and we can I can show you what it is that I'm I can't do on your app and kind of talk you through how I use it. and so yeah, he invited me to a local pub. and yeah, we had a little drink and he showed me how as a VoiceOver user he uses iOS.
Rob Whitaker (01:40)
I was just amazed at all of the things that he could do because he was using my app that I had built, but in a way that I had never even comprehended was possible. and when you y see someone who uses VoiceOver all the time, it's very different to how even you would use it as a developer doing some testing. Like it's it's quite a different experience.
Leo Dion (02:04)
Are there still things that you find that you're like, I didn't know you could do that with accessibility?
Rob Whitaker (02:10)
yeah, all the time. I can't think of a great example, I'm afraid. But yeah, I d like I'm learning stuff all the time. this
Leo Dion (02:16)
Mm-hmm.
Rob Whitaker (02:17)
is why I talk a lot with a lot of other people who work in accessibility. you know, in in kind of the iOS world, people like Danny Devesa Derksen-Staats and various other people like Bas, we often kind of talk about, Hey, did you know about this? Have you heard about this thing? 'cause it's such an enormous topic, right? And you can never know everything. and yeah,
Leo Dion (02:35)
Yes, yeah.
Rob Whitaker (02:36)
we're often saying
Rob Whitaker (02:36)
saying, what do you think about this? How do you think someone would use that? Because a lot of it is still quite new to us.
Leo Dion (02:43)
So let's let's take a step back and kind of define what accessibility is. Like what's your definition and what what are some mistakes that people have when they talk about accessibility?
Rob Whitaker (02:55)
I would say accessibility is about making your app work for everyone, regardless of their ability. it's about including people, about making sure that
Rob Whitaker (03:07)
the software that you make is going to be usable by the largest number of people possible. You're always going to exclude some people just because of the technology or the function of the app. But you can certainly try to make it available for as wide a number of possible people as possible. I think that's important.
Leo Dion (03:25)
So there's kind of the obvious stuff like you know, in like sight and sound, and touch, the way you interact with the iPhone. But are there any that people don't like think about or don't realize?
Rob Whitaker (03:44)
So Apple classify disabilities into four main categories, which I'm probably gonna forget at least one of, so I apologize. but yeah, so vision, motor, which is kind of movement or the ability you know the loss of limb or something like that, a cognitive impairment.
Rob Whitaker (04:03)
I think I said vision impairment, didn't I? or a hearing impairment, an auditory impairment. Those are the four main categories that Apple breaks things down into. and I think if you're a dev, probably the main one that you consider is the visual impairment, which is what VoiceOver users would generally fall into that category. but there's a lot of really interesting assistive technologies and features on iOS that fit into the other categories.
Rob Whitaker (04:28)
So voice control is a really fun way to interact with your device. You essentially just speak to your device. You
Rob Whitaker (04:36)
tell it what you want it to do and it will interact with your device based on those commands those voice commands it's a little bit more direct than Siri where you kind of say hey you know answer this question you're actually saying tap on this button or something like that. It's quite fun to use. But that would be for people who or generally for people who would struggle to touch the screen. So they can see what's on the screen but they can't interact
Rob Whitaker (05:02)
With it because maybe they don't have the motor abilities to do that. but cognitive impairments is the category that Apple have been working on mostly within the last few years, and there's some really interesting features in that. there is a new feature called Assistive Access which allows you to build a cut-down version of your app. So
Rob Whitaker (05:25)
it will allow people to have a phone set up for them with only a certain subset of apps available so that they can't do anything accidentally bad to their device. And then those apps will often have if they support assistive access fully, they will have like a cut-down version which has a much simpler UI, a much larger UI, so it's easier to see, easier to understand.
Rob Whitaker (05:51)
Things like navigating back to get out to things is really simple. Things like making destructive actions is usually removed.
Rob Whitaker (05:59)
This is it does change quite a lot about how you use your phone, so I wouldn't necessarily suggest it's something you should just enable straight off, like do a little bit of research into it first. But I'd really recommend if you check it out with Clemens Strasser who makes a lot of games and apps available on the App Store. several of his apps support that feature in a really nice way. so I recommend checking those out.
Leo Dion (06:23)
So if anybody has ever seen a post online where it's like, you can turn your iPhone into a dumb phone, that's right. That's basically assistive access, right?
Rob Whitaker (06:33)
Yeah, it's it's really intended for anyone with any cognitive impairment, but it's ideal for, you know, if you've got an elderly relative who's you know struggling a little bit in their older age, it's ideal for that. They can still use an iPhone, it's still an iPhone, but it's just everything's bigger, it's simpler. Like you say, it's kind of a dumb phone.
Leo Dion (06:52)
Right, right. I so I just I tried it with one of my phones for the kids and it was not so one of my big problems was I couldn't get it to work along with like screen time. as you might imagine, they kinda conflict with each other. so that that's why I didn't end up going with that for the kids. 'cause, you know, I wanted a phone that was very limited in what they could do. But I I totally get what it's getting at and what it's trying to do. and as we get more
Leo Dion (07:20)
More people, older people using iPhones just because people age. Like I think it's just yeah, gonna be more and more important. what? So let so we'll talk about now what we're discussing at our little what do they call it when you have a group of people, hallway chat, whatever we had at AppDevCon on probably the biggest challenge, and that's getting your manager
Leo Dion (07:46)
to care about accessibility and making it part of the development process.
Leo Dion (07:53)
I don't know yeah, I don't know like w let's take AI out of the equation for now, but like what what are some have you had a lot of challenges? I think we've all had a lot of challenges, and what have you found is the best solution in many of your cases?
Rob Whitaker (08:11)
It's always difficult, I think, getting your manager, getting your leadership at your business.
Rob Whitaker (08:17)
bought into accessibility because accessibility costs time and money. and they will often see it as, there's maybe something that we perhaps should do, but it's let's maybe do it later. But the problem with doing it later is that that's when it really does cost time and money because fixing these issues is so much harder than just not causing the issues in the first place. So that's a great way to kind of win people over really is to say look, you know, if we build our accessibility knowledge, if we you know shift left,
Rob Whitaker (08:45)
Most people who are in tech leadership have heard of the concept of shifting left where you kind of move testing down the process so that essentially you don't make the mistakes and then you don't have to spend time finding them, you don't have to spend time fixing them. It's going to be much more efficient to do that. Things like automated testing, things like designing the UI, UX design, making sure that that is checked for accessibility.
Rob Whitaker (09:13)
before you actually start building it. That's always a good way to achieve that. The main business case really, I suppose, is the fact that roughly 20% of the world's population identifies as being disabled. So whether that's
Rob Whitaker (09:28)
UK, US, Europe as a whole, international estimates, they're all around about the same that roughly twenty percent of people identify as having some kind of disability. so if you're not supporting assistive technologies, if you're not supporting people with disabilities in your app, you're losing twenty percent of your potential audience. and quite often I find that that leadership then wants to go, okay, well let's let's try test that out, let's get some
Rob Whitaker (09:55)
stats from you know how many people use assistive technologies. The problem with that is if you're not supporting an assistive technology, if you don't if your app doesn't support VoiceOver, people who need VoiceOver aren't going to use it. So those numbers are never going to be right.
Leo Dion (10:04)
Yeah. Yeah, I
Leo Dion (10:08)
Yeah, I was gonna say the same thing. That seems like a little bit of a like I don't know what the analogy there's gotta be an analogy or c or metaphor for that. But yeah. If you're not if you don't support it, they're not gonna come, so you're kinda getting the stats you want in a way.
Leo Dion (10:24)
Well so one of the things we talked about at our little session, I think I was the sole American in the group was the whole legal differences between US and EU. Because
Leo Dion (10:37)
Most of the projects I've been on, if it's big enough, they pretty much have to put accessibility in because they don't want to get sued. You want to explain that difference between the US and the EU and what the EU has been doing lately?
Rob Whitaker (10:55)
Yeah, I don't know how exciting this is, but it's quite in the weeds of legal differences between the US and Europe. But in Europe we have the European Accessibility Act, which is a European-wide law. Every state within the EU has to match the level of the European Accessibility Accessibility Act standards, which essentially says you know, apps have to be accessible to a certain standard. and if you're not, then you could be fined a lot of money or an island.
Rob Whitaker (11:23)
can even be put in prison if your app is not accessible.
Leo Dion (11:26)
wow.
Rob Whitaker (11:27)
but then in the US, the the kind of products the projects that you've been working on, the difference there is that there are laws against you know law saying that things have to be accessible. but it's enforced in a different way. So in the US
Rob Whitaker (11:42)
people tend to be more happy to sue other people, right? as opposed to the in the Europe where we will kind of think, well let's let's leave that for the government, let's let the government figure things like that out. The problem there is that there's nothing there's no impetus for the government to prosecute anyone for making inaccessible software because it costs them money to do that. It takes a long time they will then have to put up a case and go into court and
Rob Whitaker (12:09)
And then even if they do get money back in a fine, it often won't cover the cost of the investigation or the people who are doing the investigation don't see any of that money. Whereas if you do it the American way
Rob Whitaker (12:23)
someone somewhere is gonna make some money if you successfully sue someone. And so the people who are making the software, they don't want to lose the money by being sued. So there's a much more of a drive for businesses in the US, I think, to be accessible because there is that real real meaningful financial like risk to them if they're not accessible, which in Europe at the moment just really doesn't exist.
Leo Dion (12:48)
Yeah. So I was just thinking when we were discussing it in Berlin, it was like, how can you incentivize how can you encourage a manager? And my my thing was just like, well, you don't want to get sued. But you had brought up the point, it's not quite the same in Europe, which is really great. Yeah, did we cover everything we can as far as how to convince your manager to spend some time and money on accessibility?
Rob Whitaker (13:13)
Yeah, we covered quite a bit there, I think. I think you can still get sued in
Leo Dion (13:15)
One thing I was gonna say.
Rob Whitaker (13:17)
Europe, but it's just not as common as in the US.
Leo Dion (13:20)
Right, right. So one thing I was gonna say was if we're doing anything with UI tests, accessibility is like kind of not part of it, but like I guess UI tests is a bene extra benefit if you do follow like a lot of the accessibility stuff in iOS or Apple platforms. So
Rob Whitaker (13:41)
Yeah,
Rob Whitaker (13:41)
so UI tests they use the what's called the accessibility tree, which if you're just writing UI tests you don't really need to know about that. But the accessibility tree is the representation of your UI that assistive technologies read from. It's like a a text representation of the UI essentially. And because that's intended for as like a text representation that automation can use for assistive technologies.
Rob Whitaker (14:08)
It also means that UI tests can use that. So if you're writing XCUITest, that's using that same accessibility tree. So if you write a UI test and you check that something's on screen and it you can detect it as the correct element type and it's got the correct value and you can interact with it, then that's essentially an accessibility test. So you kind of get accessibility testing for free by doing good quality UI testing.
Leo Dion (14:36)
Yeah, whatever they're gonna do as far as refactoring it or making it more swifty, I hope they they keep that relationship there because it's beneficial to everyone. okay, so you have a book and you have a new you have a new book that just came out or is coming out or a new edition I should say.
Rob Whitaker (14:57)
Yeah, so I have a book which came out in August, so it's been out a couple of months now. which is although it's just come out in paperback, I think it was available as a PDF before, but not as a paperback. it's called Developing Inclusive Mobile Apps. and it covers accessibility for iOS and Android. It's written from the perspective of someone who's dev, someone whose main job has been writing iOS apps and Android apps.
Rob Whitaker (15:23)
but it covers really everything. So I've tried to give quite a high level view of what accessibility is, what disability is, what the laws and regulations are around it and why it's important. but then
Rob Whitaker (15:35)
As a dev, I'm giving you the tools that you would need in your daily job to build an accessible app. So whether that's reviewing designs, whether that's knowing how to build designs, knowing the APIs that you can use to achieve a certain thing. And importantly, I I'm not covering every API, there's still others that aren't in the book that you can find the Apple documentation. But importantly, what I wanted to do was to tell you what the effect of using that API is and why.
Rob Whitaker (16:04)
you would use it, what benefit that's going to give for the people who are using assistive technologies. because that's not always clear, I think, unless you are really familiar with how these assistive technologies work and how people use them, I think it's often really difficult to understand, okay, well but why would I use that API or when do I need to use that API? And that's really what I'm trying to do with this book. the previously it came out I think in twenty twenty was the first edition, when I covered
Rob Whitaker (16:30)
XML in Android and I covered UIKit in for iOS. And things have obviously changed a lot since then. I think more than I had kind of anticipated. I thought, you know, it's it's only a few years old. But actually when I came to read it, read through it to look at doing a second edition, nearly the whole book needed completely rewriting because everything has just changed so much in terms of the technologies that are available and how you build the UIs for these platforms.
Leo Dion (16:57)
Yeah, yeah, exactly. so you mentioned SwiftUI. a lot of people think that accessibility comes for free with SwiftUI. how do you like how do you address that? 'Cause like it seems obvious to me that no, you need to put you need to do the work and put in the work to make your views accessible, etc. Yeah, where did that idea even come from?
Rob Whitaker (17:22)
it's an idea that I feel slightly responsible for.
Rob Whitaker (17:26)
what an early talk that I did was to try and encourage people to switch from UIKit to SwiftUI because SwiftUI was more accessible than UIKit. And it's true that there are some really good choices that were made in SwiftUI and as well in Jetpack Compose as well, about how accessibility works and what defaults should be. Because I think because mobile accessibility was much more mature by the time those UI paramets
Rob Whitaker (17:54)
Paradigms were they started working on them. And so I think they were able to feed a lot of that knowledge in and kind of they'd learnt a lot from some bad decisions that were perhaps made using UIKit. And so there are some good accessible decisions made in those paradigms, but I think
Rob Whitaker (18:15)
Some people sort of heard that and thought, okay, well, yeah, we should use UI SwiftUI because it's it's gonna be accessible and sort of stopped there. And I think that's sort of natural because you hear someone saying, you know, SwiftUI is more accessible, it's easy to kind of reduce that down to SwiftUI is accessible. And most people aren't gonna be thinking about accessibility every day, right? It's not so it's not for most developers, it's not something that you think about every day. I'd love it to be something
Rob Whitaker (18:44)
people thought about every day. I think about it every day, but a lot of people have other things to think about, right, when you're building features. So I mean in terms of
Leo Dion (18:53)
Is there
Rob Whitaker (18:53)
s
Leo Dion (18:54)
is there any improvements in accessibility when it comes to SwiftUI over UIKit?
Rob Whitaker (19:00)
Yeah, so the two main ones I think for me would be the way it treats images by default. So in UIKit if you added an image to a view, it would be hidden from VoiceOver or from other assistive technologies, which meant if you had a an image button, then that image button was just silent.
Rob Whitaker (19:22)
to a VoiceOver user. It had no label, they didn't know the they couldn't tell what the purpose of that button would be. Whereas in SwiftUI, images are available to assistive technologies by default, which can make
Leo Dion (19:37)
Okay.
Rob Whitaker (19:37)
your app a bit noisy. If you've got lots of decorative images, images that just look good but don't serve a purpose, then VoiceOver users could find that quite noisy. But their decision was well that yes, that's an annoyance, but
Rob Whitaker (19:51)
People can still use that. Whereas if you've got an image that conveys a meaning but is hidden, then that means people can't use it. so it is better, it's not like ideal, it's not gonna fix everything, but it is a really nice quality of life improvement. and really, if you are
Rob Whitaker (20:10)
trying in any way to make your app accessible, you shouldn't be leaving all of those images without labels or hidden you know visible to VoiceOver when they should be hidden and and things like that anyway. But the main one really is a class of accessibility bug which SwiftUI just completely wiped out because of the way that it works being essentially state driven which is in UI kit
Rob Whitaker (20:34)
because it was imperative you could quite easily create a situation where your visual UI and the accessibility tree became out of sync so that you would have
Leo Dion (20:44)
Mm.
Rob Whitaker (20:44)
something that was visible on screen but the actual representation
Leo Dion (20:46)
That's not good.
Rob Whitaker (20:48)
was like completely different to what was on screen. and
Rob Whitaker (20:51)
That was just because UIKit would try and detect when things on screen changed and then update the accessibility tree based on when it thought something had changed. And it wasn't always able to detect that something had changed. And so your responsibility as a dev was to tell UIKit, hey, something's changed sometimes. This is actually more of an issue in AppKit than it is in UIKit. If
Leo Dion (21:14)
Okay.
Rob Whitaker (21:14)
you're still making macOS apps with AppKit, this is something you have to do pretty regularly.
Rob Whitaker (21:20)
But with SwiftUI because it is declarative and state based, if your state change state changes, then your UI changes and that
Rob Whitaker (21:30)
triggers a redraw of the UI but it also triggers a regeneration of the accessibility tree. There essentially it diverges once that state changes. We always think about okay well you know if that state changes then it's going to trigger a redraw but actually it also triggers regeneration of the accessibility tree at the same time.
Leo Dion (21:51)
Yeah, I didn't even think about that. So what is there like an invalidate method or some sort of method you have to call in UI kit for that?
Rob Whitaker (21:59)
Yeah, so it would be an accessibility notification. So I
Leo Dion (22:03)
Okay.
Rob Whitaker (22:03)
think most people would probably be familiar with that as a way to post like a an announcement. So you give it a string and voice you can make VoiceOver speak a string at a particular time. but
Leo Dion (22:14)
Okay.
Rob Whitaker (22:14)
actually the to be honest, the more valuable notification is the one that says, hey, something on the screen changed.
Leo Dion (22:19)
Good.
Leo Dion (22:21)
Man. Yeah, that's one thing I would not miss. going to SwiftUI, like that's a big benefit. so one of the things you mentioned, so there's some things developers can do, right? And one of the things you mentioned is a whole thing with images. So correct me if I'm wrong, is that like accessibility accessibility children type or accessibility child elements to hide and define how they're grouped?
Leo Dion (22:47)
Is that correct?
Rob Whitaker (22:48)
so you can just do accessibility hidden if you just want to hide an image. but yeah,
Leo Dion (22:54)
That's right. Okay.
Rob Whitaker (22:55)
I think th actually you've kind of stumbled on another great feature about SwiftUI that makes it a much better paradigm for accessibility than UI kit. And I you're getting the idea as to why maybe people started to think SwiftUI is accessible, right? Because there's a lot of these kind of n nice little quality of life improvements.
Leo Dion (23:08)
Right. Well it's it's
Leo Dion (23:11)
not that SwiftUI is just better, it's it provides a better API for accessibility. Yeah. That gets lost that gets lost in translation, as you might imagine.
Rob Whitaker (23:15)
Yeah. Yeah. With the Yeah.
Rob Whitaker (23:22)
And that's probably my fault because I'm not I'm not translating it properly. But yeah, the the the accessibility elements children dot combine, I think is what you're thinking of. So
Leo Dion (23:31)
Yep, yeah.
Rob Whitaker (23:31)
that is allows you to combine elements into a single element for accessibility. So on screen it still looks like you know multiple parts of text or you know a button and sort of an icon or whatever. whereas if you add that modifier, then for accessibility it can group it as one element, which
Rob Whitaker (23:51)
makes it much simpler and easier and quicker to navigate. So this is ideal inside a list or something where you've got like a
Rob Whitaker (24:00)
a master detail setup.
Leo Dion (24:01)
Okay.
Rob Whitaker (24:02)
So if you've got everything available on the detail screen, then on your list you maybe want to just group elements together so it's easier to navigate and quicker to navigate. So long as all of that detail is available in full and navigating in the order that people would expect with VoiceOver in the detail screen, then grouping it together in a list is going to make that list much simpler and easier to navigate.
Leo Dion (24:26)
Okay. So let's let's go over some of the other things that developers can do. is there any like thing you can add to like CI to test the stuff out, like you would a unit test?
Rob Whitaker (24:39)
can I would say testing manually is always going to be the most valuable way of doing it. The problem with accessibility what makes it a really kind of knotty problem and makes it really complex is it's the
Leo Dion (24:53)
Do
Leo Dion (24:53)
you mean naughty like misbehaving or naughty like a naught? Which one?
Rob Whitaker (24:59)
I was saying knotty like a knot. a really
Leo Dion (25:01)
Okay, okay.
Rob Whitaker (25:02)
complex problem. the the re the the reason it's it's complex is because it's the interface between technology and humans. and humans are infinitely complex. and so that makes accessibility infinitely complex. So a big part of that of accessibility is understanding intent.
Rob Whitaker (25:21)
Understanding w who designed this, who built it, who intended design the system, how did they mean for this to be perceived and work? And what information did they mean it for it to convey? and y automated tests can't really do that. That's a little fuzzy now with AI, because AI can kind of
Rob Whitaker (25:41)
try and figure out intent to some extent. But the problem with doing automated testing is always going to be that it can only pick out a subset of things. But anything that can be automated should be automated.
Rob Whitaker (25:54)
So if you can add something like we talked about earlier with XCUITest, that's going to essentially if you've got good quality, well-written UI tests, they're essentially accessibility tests. So always worth adding some of those in, particularly for your really high value areas. there are third parties like Evinced or Abra who do automated testing tools which you can add in to your CI C D. They are vendors.
Rob Whitaker (26:20)
so you have to pay for that product. But they are pretty reasonable quality products. Certainly better than they were, you know, five, ten years ago, when the there was kind of tools like that for web, but really for mobile there was nothing available.
Rob Whitaker (26:35)
The paid for ones, to be honest, are what I would generally recommend. there are I would suggest the Point-Free snapshot testing library has a plug-in for it. made by James I can't remember his name now. James, who is one of the organizers of Swift Leads, he had a plug-in for
Rob Whitaker (26:58)
the Point-Free snapshot testing library, which will
Leo Dion (27:01)
Okay.
Rob Whitaker (27:02)
take a snapshot of your accessibility tree as well as the visual snapshot and it can overlay kind of where the accessibility frames are on that snapshot. And I quite like that approach because then it you can use it essentially as a regression test.
Rob Whitaker (27:18)
So I think that's a much better way of instead of it an automated test saying I think there might be a problem here, you kind of say, okay, this is we know we have good accessibility experience here, so then we'll put in that snapshot test that says, okay, if there are any regressions, then this will fail and we know we have to go back and retest it. so that to me seems quite valuable.
Rob Whitaker (27:42)
I did many years ago create a library on top of XCUITest, but the problem I felt with that was that a lot of these automated tests are quite judgment based and so I was never happy with the way I'd managed to work out a lot of those judgment calls. I think it needs someone essentially to spend all their time doing
Rob Whitaker (28:07)
UX research and technical research, which wasn't something I was going to be able to contribute to it, which is why I kind of say that the vendors are a a reasonable option now because they do have people who spend a lot of that time doing that research and making sure that everything does you know validate and make the right decisions.
Leo Dion (28:24)
So I see Reveal by Mantel and AccessibilitySnapshot by Cash App, that's what you have. And then you said there's what
Rob Whitaker (28:33)
Yeah, so
Leo Dion (28:35)
there's one by James from Leeds.
Rob Whitaker (28:38)
Yes. I will find you a link so you can put it in the notes.
Leo Dion (28:41)
Okay. Yeah.
Rob Whitaker (28:42)
but yes, it is a plug-in for the Point-Free snapshot testing framework. but yeah the the other one you mentioned
Rob Whitaker (28:49)
reveal is it's not a CICD tool, it's a view debugging tool, which it's a general view debugging tool not just for accessibility but accessibility is one of the features that they provide in it and that is you can build your apps
Rob Whitaker (29:05)
and in investigate the UI within interrogate the UI within reveal. It's sort of like the in Xcode where you've got the view expander where you can pause the app
Leo Dion (29:15)
Right.
Rob Whitaker (29:15)
in the simulator and it kind of shows you the layout of your app and all the layers and where they all are. It's a more much more fuller featured version of that. But they also have a bunch of accessibility validations in that, some of which I wrote for them.
Rob Whitaker (29:30)
so yeah, I suppose I probably should recommend that one as well. But it is a a great product, but it's not one that you can use within a CI CD, it's one that you would use as a debugging tool while you're building UI.
Leo Dion (29:44)
so reveal is reveal kinda like a power user's version of the accessibility inspector, maybe, or
Rob Whitaker (29:52)
Yeah. I would say
Rob Whitaker (29:53)
The yeah reveal is like a it's a mixture of the accessibility inspector and the view debugger in Xcode in one product. And yeah, very
Leo Dion (30:02)
Okay.
Rob Whitaker (30:03)
much a power version, power users version of that. So yeah, there is the accessibility inspector that you've got bundled as part of Xcode. and because you mentioned CICD, the the checks that the accessibility inspector does, you can also put those into XCUITests. You can call those tests directly from XCUI and
Rob Whitaker (30:23)
And so it will do a subset of automated accessibility testing for you using that. But it's a reasonably small subset of validations that the accessibility inspector will do for you.
Leo Dion (30:36)
Is it have you found the accessibility inspector to be useful? Or is there things that people should know about the accessibility inspector?
Rob Whitaker (30:47)
I feel quite snobby about the accessibility inspector. The reason I say that is because I very rarely use it. But the reason I very rarely use it is because I feel like I
Rob Whitaker (30:57)
have a pretty good knowledge of a lot of the other accessibility stuff and so I can I can kind of judge things myself by doing things manually probably a lot better. But for if you're not someone like myself who spends all their time thinking about accessibility, which I understand that's most people, then the accessibility inspector is pretty good. it does do as I say a subset of tests for you, it does them pretty quickly, you press a an audit button and it will kind of take a moment and just go, ho, hey, all these these things you should probably
Rob Whitaker (31:27)
Take a closer look. you do then have to spend a bit of time investigating: is that a legitimate issue that we need to fix, or is it just something that it's
Leo Dion (31:34)
Right. That's the thing I worry about
Leo Dion (31:35)
is a lot of false positives and noise out of it and you're just gonna throw your hands up and say this is useless.
Rob Whitaker (31:44)
And that's another thing that these vendors actually do a reasonable job at.
Leo Dion (31:49)
Mm-hmm.
Rob Whitaker (31:50)
that they they are pretty good at filtering out false positives. You can never filter out all of them, because again, it's about intent. It's about how users you know perceive what's in your app and how you intended them to perceive it. And that's always going to be difficult for a machine to figure out.
Leo Dion (32:09)
what other tools should people know about that are built into Xcode?
Leo Dion (32:14)
Or is there any anything in the OS that people need to be aware of as far as auditing the auditing by hand, I guess?
Rob Whitaker (32:24)
So all of the assistive technologies and accessibility features are all in iOS. You don't have to download anything or you know add any extras or anything like that. They're all there in iOS. You just need to turn them on essentially. I would say the best thing to do if you want to start testing on device is to use the accessibility shortcut because then you can just enable it's a triple tap on the sleep button and then it brings up a
Rob Whitaker (32:52)
menu that says you know what feature do you want to enable and you can turn it off and turn it on through that really easily so that's by far the easiest and most pain free way of enabling or disabling assistive technologies which is great when you just want to do a bit of quick testing and you know things like VoiceOver change how your device works and so sometimes they can get a bit confusing so it's always good to know how to turn them off really quickly. So I would definitely recommend having a
Rob Whitaker (33:21)
look around the settings, the accessibility settings on your device. But I would recommend probably doing a bit of research first so that you know how that's going to change your device, because as I say, some of them do change how you use your device, but they're all non-destructive, so you can always turn the setting off again afterwards. so yeah, don't need to worry too much, but probably just do a little bit of research first. In terms of Xcode.
Rob Whitaker (33:45)
with when you are running your app in through xcode there are in front the environmental overrides where you can enable things like dark mode and you can change the text size and there's a few other accessibility features you can toggle through those it's only a subset of features but again if you're running your app connected to xcode debugging it that's going to be really quick to just toggle it on
Rob Whitaker (34:09)
Have a quick check and then you can toggle it off again. And the great thing about that is it is completely non-destructive. It doesn't change how your device works at all. It's just you toggle it on, you toggle it off in Xcode.
Leo Dion (34:20)
I was smiling when you were talking about like turning it on and off because I'm like, how many of have you ever run into the situation where somebody's like, my I lost color in my iPhone, what happened? Like and they didn't realize I had an accessibility thing on. Like, have you ever run into that before? Yeah.
Rob Whitaker (34:37)
yeah. Yeah. before I was a an iOS engineer, I used to work in an Apple shop and we would quite regularly get people who would come in and say, Well what have I done with my iPhone? And they just turned on an accessibility feature. you say, they turned on the grayscale mode or they turned on VoiceOver or something like that. And 'cause you know, they were just sort of fiddling around, you know, idle hands, and then they turned something on and then didn't really know what they'd done. so yeah, it I get
Rob Whitaker (35:04)
people get a bit scared about turning those things on because it is a bit confusing because it makes changes but yeah if you enable the accessibility shortcut it's just a really easy to kind of triple tap to turn them on triple tap to turn them off
Leo Dion (35:19)
so I wanted to ask so I've you know I we talked about my app at least how I'm like trying to make it more and more accessible particularly for blind users and stuff because you shouldn't have to use your eyes for the app anyways. But is there anything special about the different Apple platforms besides iOS that folks should be aware of?
Rob Whitaker (35:41)
Every platform is slightly different in the way that it works with assistive technologies. Things like VoiceOver, they do work in a slightly different way. They're mostly pretty much the same. But obviously, if you're using a Mac, then it's not touchscreen, so it's not going to be swipes and taps, it's keyboard gestures to navigate with VoiceOver on a Mac. With the Apple Watch, you do have VoiceOver on there, but it's a reasonably simple version of VoiceOver.
Rob Whitaker (36:09)
over again swipes and taps but you don't really get all the extra gestures and then
Rob Whitaker (36:14)
The Vision Pro, not something I've had a chance to play around with the accessibility on that unfortunately yet, but they do again have a slightly different version of VoiceOver, which uses hand gestures. similar to how you can control the Vision Pro anyway, but just a series of hand gestures. with the Apple Watch actually there is a way that you can control it using again hand gestures because it
Leo Dion (36:34)
Right.
Rob Whitaker (36:35)
can use the movement, can sense the movement in your hand. so you can
Rob Whitaker (36:39)
do gestures like that to to select something. And so yeah, that's quite a fun thing to play around with.
Leo Dion (36:47)
is there anything we should know about when it comes to like the new hardware that's been announced over the last few months? Like I guess we have iPhone Duo. Have you been able to take a look at the accessibility on that device? Is it still the same as it is for iOS typically?
Rob Whitaker (37:06)
Yeah, the iPhone Duo, the accessibility on there is essentially pretty much the same as any other iPhone. because
Leo Dion (37:12)
huh.
Rob Whitaker (37:13)
the software is really the same. The the only difference is it's a different form factor. I think I'm still not a hundred percent convinced how that works for people who are left-handed or people who use left to right, sorry, right to
Leo Dion (37:25)
Yes.
Rob Whitaker (37:26)
left languages.
Rob Whitaker (37:28)
I've not yet had chance to see how people would use it in that situation. I have been assured by people that should know that it actually works pretty well in those situations, but I'm yet to kind of see what users think.
Leo Dion (37:42)
Used to be like a left handed accessibility toggle in for the iPhone Duo for sure. I could I could see that. I mean there was the home view, but we don't really develop for that. We have touch screen max. I mean, I assume that you can tap or you can click with accessibility or I don't I guess I haven't watched those developer videos yet to know how that works.
Rob Whitaker (38:11)
Yeah.
Rob Whitaker (38:11)
Actually seen anything specific for developers on the touchscreen Macs, but I understand that there are instances where controls can get a bit bigger, which I think is quite interesting. I think that's probably
Leo Dion (38:22)
Right.
Rob Whitaker (38:23)
going to be the most, the biggest difference really on that paradigm, because with touch screen devices you would need a 44 by 44 pixel minimum really for a button or anything that's interactive because it needs to be big enough for your finger, right?
Rob Whitaker (38:39)
Whereas on a desktop, generally you're going to be using a pointing device. So the rules around target sizes are a little different because often you're going to have a slightly more accurate pointing device than just your finger. So I yeah, that that would be the main point of interest for me is how is that going to work?
Rob Whitaker (39:01)
how are we going to ensure that people have got enough space for them to tap and not accidentally tap the button next to it? I think would be yeah, the thing I'd be looking at on there. The home view actually, you're right that you know it's not something you can directly develop for. I haven't actually seen any of the accessibility story on that yet, but I'd be really interested to see what how that works because I think there's
Leo Dion (39:21)
Great.
Rob Whitaker (39:21)
some interesting features there and it's intended as like a family device, right? So you know if you've got a family where you know one person in the family
Rob Whitaker (39:29)
needs to use a screen reader that how is that going to switch between you know working for someone who would normally use a screen reader and then working for the rest of the family. So yeah there's
Leo Dion (39:40)
Right, right.
Rob Whitaker (39:41)
definitely some interesting things there to look at.
Leo Dion (39:45)
yeah. So my understanding with macOS touch is like for buttons that are too small, they're supposed to blow up when you tap them. Like with increment and decrement is one I've heard of, where it's like you tap it and then it blows up for your finger, and then you tap either up or down. So luckily they've thought of that without sacrificing what you can do with pointer and keyboard, which is really great. let's see.
Leo Dion (40:11)
Typography. What do people need to know about typography and like dynamic type and fonts and stuff that you think a lot of folks miss?
Rob Whitaker (40:20)
i the main thing is it's really, really difficult to do later on. All accessibility issues are pretty difficult to fix later. But you know, if you
Rob Whitaker (40:31)
Don't think about dynamic type, if you don't think about text scaling when you first make an interface, it's pretty much never going to be added later because you have to rethink so much about your design, about how it looks, about how it functions, about what it's going to do when the text gets longer or bigger. You know, what happens when you've got two bits of text horizontally next to each other and they get bigger, you know, they're probably gonna have to reflow
Rob Whitaker (41:00)
To vertical being a vertical stack, there's a lot of things to think about and to consider, and if you think about this at the start, it's pretty simple because you just build your UI with that in mind. But if you don't, then you've essentially got to rebuild everything. You really should be building in that way to start with, anyway, because you know, if you didn't, you're gonna be pretty stuck now with the iPhone duo because that's a different UI paradigm, it's different shape, things are in different places. so if you had built a pretty
Rob Whitaker (41:29)
Fixed design that didn't work with dynamic type, then chances are you're going to be having a pretty bad time with your app resizing on the iPad or with iPhone Duo support. So yeah, you should be doing that anyway because it's just going to make your life much easier supporting extra devices as Apple bring them on. But also, like I think people often forget that there is screen zoom available on iOS, which isn't really something that people see as an accessibility feature. When you first turn
Rob Whitaker (41:57)
Turn on a new iPhone, it asks you what level of Zoom you want. So a lot of people are just going to pick the higher level of zoom anyway. So if you're a developer who's only ever tested it on the lower level of Zoom, then you're probably going to find that if someone's using it on a high level of Zoom, particularly if they've got a smaller device like an iPhone SE, there's probably a whole bunch of your UI that's broken that you don't even realize.
Leo Dion (42:24)
yeah, was there anything else you wanted to talk about as far as picking the right font? But yeah, I've been I've been in the situation where I've designed an app and then I've realized yeah, that's stupid, you should have you should have thought about dynamic type earlier, but yeah, you can really Of course a lot of people they just design their apps for the web, like to look like the web page.
Leo Dion (42:45)
But yeah.
Rob Whitaker (42:45)
I think custom fonts are are quite important. We should maybe touch on custom fonts. So
Leo Dion (42:50)
Yeah, yeah, yeah.
Rob Whitaker (42:51)
the standard iOS fonts, well the standard Apple fonts, the San Francisco family of fonts, are generally considered to be some of the most readable fonts that are available. They you know if you look at testing for font readability, it's usually San Francisco, the BBC's in-house font, and
Rob Whitaker (43:13)
And I think Arial are like they're kind of super readable, right? makes
Leo Dion (43:18)
Mm-hmm.
Rob Whitaker (43:18)
them you know means that most people are going to be able to read that text with minimal issues. and so I would always recommend using
Rob Whitaker (43:28)
the standard iOS fonts because they are just so incredibly readable for everyone. But I understand if you want to make your app look good and stand out, you're going to need to use a custom font. So the important thing there is to again remember that you know if you're using a custom font that your UI is going to behave slightly differently. So that's obviously important. Another great reason for supporting dynamic type because then if you come to change the font later
Rob Whitaker (43:57)
You're you're gonna have fewer problems with that. but if you are picking a custom font, chances are you are your UI is gonna be less readable than if you just stuck with a standard font, just because the the San Francisco family is so readable. so
Rob Whitaker (44:11)
It's always a good option, I would say, to allow people to turn off your custom font if they would like to. And a lot of really good indie apps do allow you to do this. So switch from their custom font, which they think looks good in the app, to the standard SF Pro font or SF font font.
Rob Whitaker (44:32)
just because it it makes it much more readable for other people. You can then add in other fonts if you'd like to. If there's another font you're like, hey, I quite like this font as well, add that as another option. It's because it's all about customizability, right? allowing people to you know pick an app that looks and works well for them.
Leo Dion (44:50)
would you ever like is it like a branding thing that you would choose the font or do a custom font, or would you say ninety percent of the time you should just use the built in one?
Rob Whitaker (45:02)
I think that's really got to be up for you. I think
Leo Dion (45:05)
huh.
Rob Whitaker (45:06)
yeah, it's it's really got to be kind of your decision when you're building an app. I mean I've got an app that I have built, which is not on the app store 'cause it it's just a very weird, very specific app just for my use, where I've got a like
Rob Whitaker (45:19)
various different L C D style fonts that I use in it. I
Leo Dion (45:22)
Okay.
Rob Whitaker (45:23)
use quite a lot of different L C D style fonts in it. I if I ever shipped that, I think I would have to have the option to turn off though those fonts. I think if it didn't have those L C D fonts, it would look really weird because of the style of the app. And because it's so like stylistic and designed around those L C D fonts, it would look weird if it didn't have those fonts. but
Rob Whitaker (45:46)
It does mean that I'm sacrificing that readability. So yeah, if I ever did ship that, I would definitely have that option for someone to turn off those custom fonts.
Leo Dion (45:54)
Alright, we gotta talk about it because we're legally obligated in twenty twenty six. artificial intelligence. How has that changed the playing field when it comes to accessibility?
Rob Whitaker (46:09)
AI is super powerful for a lot of things, but it does have some really good upsides for accessibility. As we talked about before with testing, the big thing that accessibility testing wasn't able to do before was to understand the intent understand
Rob Whitaker (46:27)
why did what did the person who picked this image pick the image to convey what did the person who built this UI intend for someone to do when they use it and in code that's not always obvious particularly if it's something like an image like what why did you pick that image what what was the meaning you were intending that image to convey
Rob Whitaker (46:49)
So AI can figure out intention in a way that code on its own never could. but I think I would always be a bit wary of when you're using AI to to generate code, trusting what it tells you in terms of accessibility. I'm sure there are a lot of things that will get better.
Rob Whitaker (47:09)
But I of I often find that if you ask an agent to can you make this accessible, it often does a pretty bad job from my experience. and I think that's it's one of those things with AI that like if you you th you always think AI is great until it tries to do your job and then you're like, I it can't do my job, it's terrible at this, this and this that I would have done. so maybe it's partially that. but
Rob Whitaker (47:34)
and you know, like I say, those things will always get better as it learns and improves and gets better skills. but I think there's a fundamental issue with the way that AI works, which doesn't make it a great fit for just asking it to make your software accessible, which is AI really loves verbosity. I mean so do I, I like I talk quite a lot, but it really likes adding
Rob Whitaker (47:56)
extra comments and extra code because it's really trying to be helpful and it's in its mind, being helpful equals adding verbosity. but in
Rob Whitaker (48:08)
If you're a screen reader user, verbosity is actually really not helpful because it makes it noisy, it makes navigating the app really slow and difficult. It means that you have
Leo Dion (48:16)
Yeah.
Rob Whitaker (48:16)
to listen to loads of stuff to understand where you are and figure out what's going on. And so a lot of the time a a good accessible interface actually has v quite short labels, doesn't have loads of hints, doesn't have loads of extra context in it, just as the context you need to understand where you are and what the purpose of something is and what its value is.
Rob Whitaker (48:37)
Yeah, in accessibility we say name, role, value are kind of the two the three like fundamental things. Like what is what's this thing called?
Rob Whitaker (48:45)
what does it do and what is its current state. That's what people want to know if they're using a screen reader. And agents will often kind of add in loads of extra stuff that is just adding noise and getting in the way of VoiceOver users. So I'm sure that will improve, but as far as I see that's quite fundamental to how AI works.
Leo Dion (49:08)
so there's two aspects to it. There's the user aspect where you plug in AI for the user to kind of help with the missing accessibility pieces in an app. And then there's AI for developers, which is basically having either the agent audit your app or having the agent
Leo Dion (49:34)
add the accessibility via code. where where do you stand on both of those and what
Leo Dion (49:42)
What are some ways to make sure you don't get into trouble in either of those cases? As a developer, I guess.
Rob Whitaker (49:49)
You picked up on something I think that is quite interesting that we can talk about in terms of the sort of user features for AI, which is you know one kind of really prominent use of that is for image descriptions. so
Rob Whitaker (50:07)
With image descriptions, if you can't see an image and the image adds value, then someone who is a VoiceOver user still wants to know what the purpose of that image is. and so we would always in the accessibility world we'd always say whoever picks this image should be the person that describes it because whoever picked it is the person that understands why they picked it and what meaning they wanted from it. but
Rob Whitaker (50:30)
That's not always the case, particularly if you have an app that has user generated content, a social media app or something like that where people can submit images, then you can certainly ask them and you should ask them to provide a description. Like if you're using Mastodon, most apps will say, if you add an image, you know, haven't added a description. Most apps will do that on Mastodon. and so you should certainly ask people to do that, but not everyone is going to do that.
Rob Whitaker (50:58)
And so it's those cases where you're missing that information and it's not been provided by the person who should provide it. Well then who's going to provide that? and that is something where AI can help because it can
Rob Whitaker (51:13)
make a description of of that image pretty well now and you can do that on device pretty well. VoiceOver has had the ability for quite some years now to describe an image which it does using models that are built into iOS. Even before foundation models were a thing, that was something that was in VoiceOver iOS. And you can focus on an image and it will describe what it thinks is in the image, whether the image has a label or not, which is quite a useful feature. But it
Rob Whitaker (51:41)
quite generic in terms of the description. So great if the image has text in it, but if you're trying to convey some, you know, if you picked a photo that's got some kind of detailed meaning to it, like you might have a meme on social media, it's not going to get the context of that meme at all.
Rob Whitaker (51:56)
But one thing that has come with foundation models in iOS 27 is there is a new foundation model which can do image recognition and create descriptions of images. So you can as a developer, you can then use that foundation model to generate a description of an image, which you could then suggest to the author, who the person who submitted the image, and say, Hey, I think this is what is in the image. Do you want to edit this? And then if it is something like a meme where it hasn't picked up on that context, they can go.
Rob Whitaker (52:25)
And edit it and add in what the meme means. but if it's just an image of some text or if it's a photo of a person, it can kind of pick that out and you know do a pretty good job there, but it still ideally needs a bit of human intervention. one usage that I think works really well is when you've got user generated content but it's for a specific purpose. So, just as a completely random example, if you were working on a property.
Rob Whitaker (52:52)
portal app, you might have images, photos of a property, an apartment or something that's been submitted by an estate agent or realtor, I think, as you would call them in the US.
Leo Dion (53:04)
Yeah.
Rob Whitaker (53:05)
And they will often not submit descriptions to the photos, but you don't want a generic description. You want it to be clear that this is a description of an image from the perspective of someone who might be looking to rent this apartment.
Rob Whitaker (53:21)
So then, as part of the prompt that you give the foundation model when it's generating a description of the image, you can say, you know, are generating an image, a description of an image for someone who's a screen reader user. This app is a property portal. So describe the features of the room in a way that they might need to know if they're considering purchasing or renting this apartment. And then it just narrows down that description a little bit more.
Rob Whitaker (53:48)
rather than saying this is a photo of a kitchen it might say this is a photo of a kitchen it's got six cupboards and it's got a toaster in the corner and there's a sink with a window and the space for a dishwasher something like that so it's it's going to make it much more tailored towards the usage in your app
Leo Dion (54:05)
Did you list out any good tool like AI based skills or tools that you'd recommend?
Rob Whitaker (54:10)
There are skills available. My friend Danny, Danny Devesa Derksen-Staats has a an accessibility skill available
Rob Whitaker (54:17)
for iOS. So I'd recommend downloading that one. I can give you a link for that one so that people can find that. But yeah, that is you know, he is a real expert in iOS accessibility, so I trust that he's kind of put a lot of thought and knowledge into that skill.
Leo Dion (54:35)
Is there anything else you wanted to cover when it comes to AI?
Rob Whitaker (54:39)
No, I don't think so.
Leo Dion (54:41)
Okay.
Rob Whitaker (54:41)
I started building a skill but I didn't get very far with it. But I came at it from a very interesting perspective. Have you ever built an ac an AI skill?
Leo Dion (54:50)
Yes. but but I would typically just use AI to create the skill anyways.
Rob Whitaker (54:54)
Yeah.
Leo Dion (54:55)
I mean it's just it's just text with some tools attached to it, right? It's not So I just make AI code it for me because it's it knows what it's doing anyways.
Rob Whitaker (55:06)
How do you test it? How do you know the output is gonna be what you expect?
Leo Dion (55:11)
Well, if you listen to the previous episode with Daniel Steinberg, we talked about how AI testing works with foundation models and it's the same idea of like it's all a probability game. It's not something that is gonna work all the time perfectly, because it's AI. So that's just something you have to like be able to like, you know, willing willing to let go of, I guess. Cause that's essentially the way I would test a skill.
Leo Dion (55:41)
before we close out, I do want to cover some of the work you've been doing with the WCAG. I don't remember what it stands for, but it's involving accessibility for mobile, web, excess etc. You want to explain what that is and what is its relationship to nutrition labels that Apple.
Leo Dion (56:05)
Apple has now encouraged us to put on our apps.
Rob Whitaker (56:10)
Sure. so the accessibility nutrition labels I think are something very different, but maybe we'll cover those in a moment. but yeah, the mobile accessibility task force. so
Leo Dion (56:20)
There we go. Thank you.
Rob Whitaker (56:21)
Yeah, that's okay. there's a lot of acronyms. So WCAG or WCAG as it's sometimes called, if you've done any kind of accessibility at any point in the past, it will be based on WCAG. So WCAG is the international standard essentially for how how you should make us accessible software. it is it stands for the Web Content Accessibility Guidelines. So as the name suggests, it's quite web focused.
Rob Whitaker (56:46)
and but it despite it being web focused it is it that is really the standard for all software and so
Rob Whitaker (56:55)
What the MATF does, the Mobile Accessibility Task Force does, is we are trying to provide guidance for how you apply WCAG to native mobile apps. So at the moment very much kind of iOS and Android native, but we do try and cover cross-platform frameworks as much as possible. So essentially we're trying to translate what WCAG says, this is how you make accessible accessible experiences on the web.
Rob Whitaker (57:22)
to say this is how you achieve that same experience on iOS or Android. Because it can be sometimes a little difficult to understand how they fit. So for example there's one they call them success criteria in WCAG.
Rob Whitaker (57:37)
is the the rules for this is how you make assist accessible software. One of those covers page titles which on the web you know you're familiar with having a page title when you've got it open in a tab that's the title of your tab right and so that means as you're switching between windows switching between tabs someone who's a VoiceOver user can tell what the content of that tab is. That obviously is pretty obvious and pretty clear on the web as to what that is and
Rob Whitaker (58:04)
The purpose of it is, but what does that mean on mobile? maybe it doesn't apply at all. So we've had those discussions and we've done that research to kind of say, okay, this is what the analogy for that is on mobile, and so this is how you should apply that and how you should code it up. and so yeah, we've got a guide a document, a guidance for applying WCAG to mobile, which it's currently still in draft. These things take quite a long time because there's quite a lot of research and discussion.
Rob Whitaker (58:30)
involved. We've been working on it for a few years, but we do have a draft that we're looking to publish our first full version of pretty soon. So we are open for comments.
Rob Whitaker (58:42)
there's a group of us who are involved in writing that, but it is open to anyone. It's all done on GitHub. So feel free to if you've got something that's unclear to you or something that you think we need to consider a bit more, open an issue on GitHub or contribute to one of the issues on GitHub and yeah, you can be one of the authors of that report.
Leo Dion (59:01)
If you've ever used it like what what do you how how is it compare the web stuff to like i like iOS? Well how does it web and Android accessibility compare with iOS?
Rob Whitaker (59:14)
they're all quite different, but maybe not as different as you might imagine.
Leo Dion (59:20)
Okay.
Rob Whitaker (59:21)
so
Rob Whitaker (59:22)
I was having a conversation with a someone who worked on web recently who was asking me about mobile accessibility because he assumes that mobile web accessibility would be the same as it would be for an iOS app. but actually it's not, it's the same as it would be for web on a desktop. you know the web accessibility is the same whatever platform you're on. and really the the requirements for accessibility you know they're still WCAG, so they're still gonna be the same.
Rob Whitaker (59:49)
Whatever platform you're on. The difference really is how you implement that. and yeah, the the difficulty just comes from the fact that WCAG is very much wit written in web language. so there's a few places where you do have to kind of go, Well, it's you it's quite easy to go, well, that's very specifically web language, so therefore it doesn't apply to mobile. Actually, most of the time it does apply to mobile, you just have to sort of figure out what the equivalent is on mobile.
Leo Dion (01:00:12)
Right. Right.
Rob Whitaker (01:00:13)
so yeah, there probably there are differences, but it's
Rob Whitaker (01:00:16)
It's mostly about translating it into a slightly different terminology more than big changes in behavior.
Leo Dion (01:00:23)
the thing about nutrition labels is at least Apple has something that will allow you to say and like to say what what your app actually does. do they have anything like that for web or Android?
Rob Whitaker (01:00:40)
So WCAG at the standard is that's the level of like if you reach WCAG standard double A, that's where you're not going you're not in trouble of getting sued or getting prosecuted for not being accessible. So that's kind of the benchmark of like this is accessible or not.
Rob Whitaker (01:01:01)
We can sorry the accessibility nutrition labels are a slight step down from that. So that
Leo Dion (01:01:07)
okay.
Rob Whitaker (01:01:08)
they're a great step if you've never done any any accessibility, then the nutrition labels are a a really great place to start and I'd I definitely encourage people having a look at those.
Rob Whitaker (01:01:19)
But also what they are is like a marketing essentially. They're a great way for you for your app to stand out because it shows up in the app store and it can help your app get indexed higher in the app store as well. So you know if someone searches for your game that supports VoiceOver and you said in your nutrition labels that you support VoiceOver, your app is your game is going be indexed slightly higher than someone who hasn't. So it makes your app look pretty good, it makes you
Rob Whitaker (01:01:48)
improves your search indexing so I definitely recommend looking at any anyway but it provides people a really quick glance of like does this app work for me yeah if you know as a user I can use an app that supports this particular assistive technology then you can go to the app store and you can look now on the ex accessibility nutrition labels yes this app does work for me or no this app doesn't work I'll go and look for something else so makes it really great for customers
Rob Whitaker (01:02:14)
to to know does this work and for you it's going to get gain you some extra downloads as well but yeah it is is not there's no like kind of legal standard or anything to that you know would still if you need to be compliant you still have to be compliant to WCAG but the nutrition labels it covers a subset of assisted technologies and it covers a subset of your app so it's quite tightly scoped so this is why I say it's a great place to start you know if haven't done any actually
Rob Whitaker (01:02:42)
accessibility
Rob Whitaker (01:02:43)
work before take a look at the guidance on the accessibility nutrition labels on Apple's website on the developer website and it will say okay this is how you can scope your app so you decide you determine what you should be testing and what areas of your app you don't need to test and then these are the technologies that you need to test with and how you test them. Again if you're going to do a full audit to WCAG standard you're gonna test everything and you're going to test all the assistive technologies and you're gonna test all of the features of those it's a
Leo Dion (01:03:09)
Wow.
Rob Whitaker (01:03:10)
lot more involved.
Rob Whitaker (01:03:11)
But yeah, with nutrition labels, it's very tightly scoped to it's still a decent chunk of work. Like I'm not gonna say it's gonna be an easy thing to do, but it's much tightly scoped, so particularly if you've never done anything like that before. but yeah, your initial question, sorry, I before I kind of went off rambling about loads of other things, does the web or Android have anything like that? I'm not aware that they do. So there are accessibility nutrition labels are not something that Apple invented. Accessibility nutrition labels are a concept that is
Rob Whitaker (01:03:38)
a around there
Leo Dion (01:03:40)
Okay.
Rob Whitaker (01:03:40)
so that you know businesses can kind of say hey you know much like food nutrition labels you know this is how accessible we are in certain areas on certain measures but I'm not aware of any particular standard way of doing that but I'm sure there are some and I I'm confident there isn't an equivalent on the Android on Google Play Store currently.
Leo Dion (01:04:02)
have you ever used something like Lighthouse? I use that quite a bit for my web stuff. Do you know what I'm talking about?
Rob Whitaker (01:04:09)
Yeah, I I've actually never done any web. never done
Leo Dion (01:04:12)
Okay.
Rob Whitaker (01:04:13)
anything on the web. So yeah, I I've never
Leo Dion (01:04:14)
You haven't truly lived until you've done web.
Rob Whitaker (01:04:18)
I'm sure that's true. Maybe one day. If I
Leo Dion (01:04:19)
No, it's not.
Rob Whitaker (01:04:20)
ever do if I ever need to make a website, I just like get Claude to do it 'cause
Rob Whitaker (01:04:25)
It's gonna do a much better job than I would. but yeah, I have I I am aware of Lighthouse, but yeah, not something that I've ever used. But yeah, let's I suppose for anyone who's watching or listening that or who's in my position and has never done web, Lighthouse is it's like a a browser plugin essentially that will give you an accessibility score for your website.
Leo Dion (01:04:43)
Well I have a hooked up
Leo Dion (01:04:44)
a CI actually. So 'cause you could run Chromium
Rob Whitaker (01:04:47)
Yeah. It's
Leo Dion (01:04:49)
on Node or whatever in a CI environment.
Rob Whitaker (01:04:53)
Yeah.
Rob Whitaker (01:04:54)
And it this is it goes back to what we were talking about earlier with the automated tests for accessibility. the reason that kind of works a lot better on the web is because you've got access to the code that's generating the UI. Whereas when you're running UI testing on iOS or Android, you don't generally have access to that source code. You have the compiled app. You sort of need both to be able to do that properly, right? You need the rendered UI and you need the code, which is
Rob Whitaker (01:05:21)
why it worked well on the web because you've got both of those in the browser. The brand browser does all of that for you. Whereas on native mobile you've got one or the other. You've either got the source code or you've got the rendered UI. You never have both together.
Leo Dion (01:05:33)
Mm, right.
Leo Dion (01:05:36)
Yeah. Well
Leo Dion (01:05:39)
Was there anything else you want to talk about before we close out?
Rob Whitaker (01:05:43)
Talked about a lot.
Leo Dion (01:05:45)
I If there was one thing you wanted a developer to do this week, what would you recommend?
Rob Whitaker (01:05:50)
I would say try out VoiceOver. so VoiceOver we talked about it a lot. It's not the only assistive technology.
Rob Whitaker (01:05:58)
but VoiceOver uses more of the accessibility features on iOS than any other assistive technology. So if you test with VoiceOver, if your app makes sense and you can understand you know what you're supposed to do and how you're supposed to use it, you're gonna get a much you're gonna kind of be testing a lot more of the assistive technology kind of features in your app than with any other assistive technology. So even if you know the very basics of VoiceOver,
Rob Whitaker (01:06:27)
that's going to get you quite a long way with with testing. So I would say the first thing to do is enable the accessibility shortcut in the s accessibility settings on your device so you can turn VoiceOver on and off really easily and then turn on VoiceOver, have a swipe around, figure out what's happening and then turn it off. Just spend five minutes doing it. You don't have to spend long, just kind of get hands on with like how it makes it different different to navigate with and
Rob Whitaker (01:06:55)
what your expectations are with that. And then just kind of build up from there, really. Start testing your app as you get a bit more comfortable with it. But I think a lot of people will not test with VoiceOver or other assistive technologies just because they are different, because they change how your device works. So getting more familiar with them is always going to be a benefit, I think.
Leo Dion (01:07:15)
Rob, thank you so much for coming on. This was such an important episode and really glad to have an expert like you finally on the podcast to talk about this. I'll give you an opportunity to go ahead and plug yourself and where you are on social media.
Rob Whitaker (01:07:31)
Well, thank you very much for having me, Leo. It's been fun talking to you. it was w several years ago now that we met, I think, at 360iDev for the first time. So yeah. So yeah, it good to finally be here.
Leo Dion (01:07:40)
Den yeah, I was gonna say Denver. Rest in peace. Rend rest in
Leo Dion (01:07:45)
peace. I still have the shirt. 360iDev. We miss you.
Rob Whitaker (01:07:50)
I this is a complete tangent. I met up with Ellen Shapiro the other day last week for lunch 'cause she was in London and she had the patch certain on her bag for three sixty I dev that we were at so.
Leo Dion (01:08:02)
That was such a great conference. I miss that conference. where
Leo Dion (01:08:05)
can people find so you got a book and then you're on the socials as well, right?
Rob Whitaker (01:08:10)
yeah, I'm on Bluesky and I'm on Mastodon. I don't use them a huge amount, if I'm honest. so I I think I am iosdev.space slash rob w, I think, on Mastodon. and yeah, I mean you can probably search Rob Whitaker on Bluesky and you'll probably find me there. but yeah, you can reach out to me on either of those. I do kind of check if occasionally. but yeah, my book developing inclusive mobile apps.
Rob Whitaker (01:08:35)
You can buy it from Apress or the publisher. You can get it as an ebook or as a hard copy if you prefer. And yeah, I think reach out to me really if you've got any questions. I'm always happy to talk to people about accessibility and you know how you use it on any platform. It's something that genuinely interests me. So yeah, I'm I'm more than happy, as I've demonstrated on this podcast, to talk about it endlessly. So please feel free to ask me any questions you've got.
Leo Dion (01:09:02)
Rob, thank you so much. People can find me online at leogdion at C.im on Mastodon. my company is BrightDigit. Go to brightdigit.com, subscribe to our newsletter. We're just did a recent piece on the release of my app at least, and you probably should see a new newsletter by the time this is out on AI and software development. So definitely check that out.
Leo Dion (01:09:27)
And subscribe and like if you are either watching this or listening to this episode. Thank you so much, everybody, and talk to you later. Bye.