(applauding)
-: Hi.
I, too, appreciate that you came in.
Although if you had not, I would not have felt bad,
because it is lovely and sunny out,
and I almost wanna do this talk outdoors.
How's the feedback?
Is that me, do I need to run?
I'm good?
Okay.
So I have slides.
I think I have no control over
making them appear at this point.
It's magic; thank you.
So thank you; look, I'm giving this talk.
There is a URL, these slides are not available yet,
but they will be available after my talk at:
ROSEL.LI/WCLDN.
And I will show that URL again later on.
But you're in the talk: Inclusive Usability Testing.
If you think you're at lunch, you missed it.
So there.
I have a slide dedicated to my ego.
This is just so you can say: this is a real person.
You should probably block me now on Twitter.
I have a website where I write a lot of stuff
about accessibility and usability and general code ranting.
Look, there's some things I've done: whatever.
Level setting done, I think.
Okay, so this whole concept of inclusive usability testing
is about integrating people with disabilities
into your testing process; so if you're already bringing
people in, if you're already sitting them down
with software and trying to get a sense of how usable
something is, folding people in with disabilities
is a great way to figure out what other
usability hiccups you might have.
For context, I've listed different types of disabilities.
Hearing impaired, blind, low vision,
physical impairment and cognitive.
I probably should have added a bullet for speech,
because speech is a more and more important interface,
one which we're coming to rely on pretty regularly.
This is the part where I now show you the entire talk
I did last year.
Selfish Accessibility.
I just have to hit the play button.
And it's only 40 minutes, so this'll be good.
That's actually available at WordPress.TV,
so you can do that at your leisure.
Sort of like a homework assignment.
One thing I want to do, I wanted to start with this quote
from Wendy Chisholm from Microsoft,
because I think it's a really important quote
and I think it speaks to where people might struggle
with making that first, initial step to including
people with disabilities.
I encourage you to fail in interacting with people
with disabilities, because you will learn a bunch.
You will learn what not to say and what people care about.
You'll learn about where the obstacles are,
both the designed physical barriers, and the constructed,
emotional ones that exist within yourself.
Very much, she's saying, don't panic.
Just start interacting with people.
People with disabilities will correct you,
if you say something or do something wrong.
You'll learn about, hey, you know,
maybe you'll offer your elbow to somebody who's blind
to guide them, maybe somebody will correct you
not to pet the service animal, because it's working,
and you don't like it when you get pet at work.
I'm just saying.
I mean, I do.
Anyway, so I'm gonna run through these basic nine categories
and there's overlap in these categories.
Not everything is cut and dry, but just generally.
We'll talk about some of the concerns,
planning, payment, which I meant to rename as compensation,
sorry, said I did it and then I lied to you, apparently.
Venue, recruitment, accommodation, technical stuff,
process and privacy, or as I understand many of you say:
privacy.
That's a much better way to say it, by the way.
This photo comes to us from a program called:
Save the Dream.
This was a national sport day in 2017 in Qatar,
and the idea was, using high profile athletes to promote
disability awareness, and at some level,
this is a little bit like disability tourism.
Like the dining in the dark and putting blindfolds on
and you know, trying to enforce empathy exercises
is a little bit tricky, but I think this was still
a really important exercise, particularly in Qatar,
because it sort of reframed this notion of accessibility
and people with disabilities.
Anyway, I have a bunch of these little images
that I sprinkle throughout,
that don't directly tie to anything.
So you don't need to make notes on those, I suppose.
So, big concern: before you start doing
any usability testing, make sure your
accessibility remediation is complete.
You are not bringing people in to do accessibility testing.
You should have already done that.
Your audits should have happened, your bugs should be fixed,
everything should be ready to go.
This should be your accessible version
that you're sitting people down in front of.
To reiterate, it's not accessibility testing.
If you do bring in people to do accessibility testing,
you're wasting their time and you're wasting your budget,
so please don't do that.
And be clear on that with all of the stakeholders.
Make sure that they understand, you're not trying
to uncover accessibility issues.
You're just here to uncover usability issues.
I'd mentioned disability tourism.
You're not here just to see how somebody with a disability
uses your software and be like: wow, that's fun.
That's interesting.
You're doing this because you're trying to gather
good and valuable information.
You're bringing in people for their knowledge
and their expertise.
That's the whole reason you're doing it.
So limit the scope of what you're doing
to just what's needed for testing.
Don't start to fold in other things.
You won't have time, anyway.
But don't fold in other things.
As I've said, be clear with all the stakeholders.
You don't wanna have a fight with them after the fact.
As you're trying to fold in the results of your study,
that can be pretty painful.
And then be wary of anybody in the organization
who is using this strictly as a PR thing.
Like: hey, we brought in people with impairments
because we love the community,
and we wanna show you how much we care.
Yeah, you have an outcome, you have an output objective.
Make sure that you honor that and you're not just
bringing people in for a PR stunt.
Planning.
I like this image a lot.
This is somebody who is taking a sign language class,
who kept drawing a floor plan to a house,
over and over and over, because the person who was teaching
it never set the reference point from one room to the next
and the person couldn't quite figure out
where the kitchen was in relation to the living room.
Because that's pretty important when you can't see.
As the person said: I never know whether the kitchen
is down the hall from the dining room,
or on the other side of the dining room.
It's rather confusing.
This whole concept of making sure that everybody
understands that reference point,
and I think American sign language education
is a nifty way to show that, plus it's just
a hilarious sketch, and I think I would live in this
weirdly stretched out house where the foyer
is twice the length of my house.
I'm just saying.
Make sure, in advance, that you review your tests.
You review the format of those tests.
That will help you prioritize the questions
and the steps that you're going to do.
Especially since you might not get through everything.
So you wanna make sure that you have it all broken down.
Are you doing a prototype?
Is this the live platform, are you testing
a specific feature that you've launched?
Are you just gaging responses?
Make sure that you're not, you know, preemptively imparting
some bias in your study.
Are these tests structured?
Are they informal, are they remote for the scope?
And what I'm talking about, I'm not talking about
remote tests, I'm talking about in-person, on-site tests.
But you have to have this stuff planned in advance.
You have to know if you're just doing questionnaires
or full deep dives into software.
Also, make sure you know how many people are participating.
Are you gonna have three people,
are you gonna have a dozen people?
Have you budgeted for that, both for money and for time?
And then all the recruiting efforts
that have to go into that, especially if
you've never done any recruiting like this?
Please make sure that you budget
for compensating participants.
And I go into a little bit more detail in the compensation
section, which I previously had listed as: payment.
I'd like to point out gift cards are awesome,
because if you give them gift cards with cash value,
not to specific stores, participants can use them
wherever and whenever they want.
They can use only a little bit of it,
they can use it to pay for a service they need.
They can basically treat them as cash.
Be careful to, basically avoid cards
that have a venue requirement.
Like you have to use it at a terrible restaurant.
Or you have to travel across town in order to put it to use.
That's a real barrier already, so don't do that.
Make sure it's more than a free coffee that you give 'em.
And if it costs them $20 just to get to the venue
to put it to use, you've already kind of messed up there.
Fun fact: you can also use these as signature guides
for your blind participants.
You just put the card underneath where they sign,
and they sign above it, and they get to take card home
and start spending.
It's important, it's critically important, that you plan
on paying participants more than you usually do.
And I use the word pay here, and it's the wrong word.
In the US, at least, once you start paying participants
it can be considered income, and it can impact
their ability to get any benefits or any aid packages
that they rely on, which is why this whole section
is called: compensation.
But make sure you set aside more money for them.
There's an outsize cost for them to participate.
There are additional burdens that a traditional participant
might not have to endure.
One of those is transportation costs.
In the states, it's no big deal for somebody
to hop in a car and just go.
We're very used to it in many American cities.
But if I'm going to bring in somebody who's blind,
to do a study, that person's not hopping in a car.
That person has to hire a service
or has to wait for one of the para-transit lines,
or has to go through some other accommodation,
and what might take me 15 minutes to drive
could take somebody else two hours, door to door
transport services, between hopping services,
scheduling queuing everything up.
Even bus routes are gonna require longer waits sometimes,
just to get the accessible bus
or the accessible train to come through.
And they might be borrowing or using their own
or somebody else's mobility aid.
It's also worth knowing that a lot of people
with disabilities are generally underemployed.
And I'll actually even jump ahead to that one.
That doesn't mean their unemployed.
It just means that they are not making the same wage
for their skills that many other people might be making,
for a variety of reasons.
And they're often not in salaried positions.
They might be in hourly positions,
and you have to account for that time off of work,
because they might lose an entire day's worth of wages.
Just to participate in your study.
That could be the cost of one day's worth of wages.
And you should be comfortable compensating them
for that time; maybe not their entire day's worth of wages,
but enough to make it worthwhile,
enough to make it worth the effort,
and consider that transportation cost as well.
Some participants might have aides
that they need to bring along with them.
It could be anything from sign language interpreters,
to people who just carry their stuff, whatever it is.
And they might need that help just to go out in public,
just to maintain a schedule, just to arrange transportation,
and there's a cost associated with that.
It doesn't mean you don't want to include these people,
it's just, again, consider that cost.
So think about gift cards that are accepted where they shop.
Maybe you don't even get a generic gift card off the rack.
Maybe you identify a regular service that they use,
and you just set them up on that service for a while.
Maybe you know that they, there's a mobility aid company
in Boston, and one thing we did once was we just gave them
some money, here, here's a pile of cash.
Here's credit at the facility, rather, just go and pick up
what you need, upgrade your cane, do whatever.
And if not, that's okay too, but make sure you have
that conversation in advance.
You want to compensate people fairly.
You might also be overwhelming them with all the information
you provide, and a gift card is a pretty easy way
for them to not have to panic too much.
Because it's just there, it's just in their pocket
whenever they want it.
Venue.
I hear some chuckles.
I need to qualify this image.
I love to use this image, but this image is a lie.
These ramps that you see on the stairs
are not for wheelchairs.
They are not for wheelchairs.
That's actually for a cart, a dolly to wheel supplies
up and down the stairs.
However, I have been to venues where their approach
to making it wheelchair accessible,
is to just drop ramps on the stairs.
And what you end up with is murder stairs,
because nobody can wheel up a ramp like that without
falling backward and if you try
to go down that ramp: (laughing).
So make sure your insurance is up to date, I guess,
at your venue.
This photo is actually from the Czech Republic
and it's a train station, and what's important to know
is some of you snickered at this,
you know at a glance that this isn't right.
Now, people who rely on these kinds of aids
and services and features, they're gonna know
before you when anything else isn't right.
You might see a ramp that looks perfectly lovely.
It might, you know, it's integrated into stairs,
and it's pretty and it's got handrails,
and that might be a death-trap and you just don't know it.
You might look at a facility that looks great,
it's big and open, and so a blind person is not
gonna struggle by walking into plants,
but an open space is terrifying to a blind person.
Because there's no way to orient.
Way finding is all removed.
So you have to consider these kinds of things,
and remember: you might not be expert
at identifying if your venue is truly accessible.
They will know if it's a token effort anyway.
Ah, it must be accessible.
I was gonna have that slide repeated over and over,
but yeah, and it's not just the venue.
It's not just the building but the entire route.
If you rent a facility where you do the studies,
you know, one of those fancy ones with the two-way mirror
and all the cameras pointing down at people,
and the microphones and everything, great.
Make sure they can actually get to it somehow.
Make sure there's a bus that runs there,
and it's a bus that supports wheelchairs.
Make sure that the para-transit line goes there.
Make sure it doesn't drop them off two blocks away,
and then leaves them stranded.
You have to be able to not just get them in the venue,
but you have to get them to the venue.
And they have to not get killed while they're doing it.
Think about sidewalks, parking lots, parking situations,
things like that.
Bus lines and transport services, ah, these can be
really expensive, if they need to hire out a transport
service so bear in mind the cost.
And bear in mind, just because you've found an accessible
bus route, doesn't mean that they're comfortable using it.
They may have had a bad experience.
I have a friend in Canada who will wait
for the accessible bus, the bus with the wheelchair thing,
and he'll be there with his dog,
and his saddlebags on his wheelchair,
and the bus will say: we don't have any room for you.
And just keep going.
And it's a lie and he's proven it,
but this is why some people aren't even gonna rely
on some of the services that are there,
and they'll have to figure something else out.
Meet them at the door.
Unless you're transporting them yourself, meet them there.
Meet them at the bus stop.
Meet them wherever the transit line drops them off.
Don't leave them stranded.
Walk them in, ah, take 'em to the front desk,
have somebody at the front desk know what to expect.
If it's a straightforward route,
make sure that's explained to them in advance,
and that the person at the greeting area can help them.
If there's an elevator, make sure it can handle wheelchairs.
You know, make sure there's a good turning radius.
Are there bathrooms near where you're doing the study?
Are there bathrooms right inside that entrance?
They may have been traveling for two hours.
Make sure that they can take a break
before they go in there.
Make sure that the lighting is good and variable.
Make sure the place isn't a, you're not walking them
through a Yankee Candle store to get into your building.
Do you have Yankee Candles here?
Think if somebody murdered a perfume store.
I don't know how else to describe it.
Don't ever go in one.
But things like smells and sounds and lighting,
those can all be potentially confusing.
And it's different depending on your audience
that you're bringing in, so bear that in mind, as well.
Definitely make sure you have a relief area
for service animals.
It's bad enough when somebody has to bring their dog
on a bus for two hours, but then the dog can't take a break,
and when they're in there doing a study with you
for a couple hours, what's the dog gonna do?
And it can be as simple as just having a safe tree
in the parking lot, or puppy pads, you know,
depending on your comfort level with that.
And then of course, make sure that when it is time
to wrap everything up, when they're exiting the venue,
that you don't leave them stranded again.
Walk the person out, walk them to their transport.
Maybe wait for the transportation.
Think about when you're releasing them.
Is it during rush hour?
You know, there's extra chaos involved with rush hour.
Are the buses still running on that route?
Are taxis still coming out there?
I did a, ah,
I did a study a couple years ago with a young woman
who was blind who got there via a bus.
The bus dropped her off a half-mile from the facility,
from the venue, and she walked down the street,
literally in the street, because there was no sidewalk.
No pavement, pedestrian, is it pavement?
Is that the term used, ah, there was no pedestrian way.
So she walked in traffic for a half mile with a cane,
and her phone telling her which way to go as she went,
which she had to hold up to her ear to hear.
And when she wrapped up the study,
because I didn't know this was the case,
she told me she could get there just fine.
My mistake.
I walked her to the bus stop.
Walked her a half mile to make sure that she got
to the bus stop and stayed in a pedestrian walkway,
and we got her there and thankfully we'd build enough time
in between the sessions, if we had not, it would have
thrown off the next session, she could have been killed,
it was just a terrible experience.
And that was partly my fault because I didn't identify
the route in advance, because I was in a foreign city.
And the venue's route, because they didn't do their job,
their due diligence of making sure that people could
actually get there, so it's not just getting them there.
It's also getting them out.
It's kind of bad form to kill your participants.
I'm just saying: maybe don't do that.
So now that you've got things sort of set up
and maybe you've identified where you're gonna do it,
you have to think about how you're gonna recruit people.
This photo is from NASA.
They're Goddard Space Flight Center brought in
The Blind Camp of Maryland, and what they did
is they brought in blind and deaf/blind people,
and they set up all of these spacecraft parts on tables,
and they just let people walk around and pick them up
and feel them and ask questions.
There were no kids, because if I was there as like
a 10 year old, I'd be swinging one of those around,
and that would be a mess, but it was the first time
many of them had gotten to experience a lot of these things,
which was awesome.
Recruiting for them, of course, is very easy.
They just went to The Blind Camp of Maryland and said:
hey, come on in!
You'll probably have some different things
that you're going to want to do.
So let's talk about that.
Again, we've covered some of these fundamentals,
so we need to figure out what you have to do
to get the right participants.
In particular, you need to know in advance
what kind of skill level you're looking for.
What technologies you're looking for,
what experience you're looking for,
just like you would for any usability study.
You know, do I want only Mac users,
do I want only Windows Phone users?
Because there's like two left.
What do I do, who do I recruit, what's that skill level?
That still applies here.
So define those characteristics upfront.
Now here's the really important thing.
Lean on community or support organizations.
They can do this heavy lift for you
if you've never done it before.
Many cities have a blind association, just as an example.
Right?
You can just give them a call.
Think about where you're running the study.
Find out what the city has to offer.
Figure out what organizations are there,
and what their constituency looks like.
And once you find one, ask for names of others.
They often have a lot of overlapping constituents, anyway.
Often a person who has some sort of disability
doesn't have just the one; there's often something else
associated with it or linked to it.
So to getting different services
from different organizations in many cases.
These organizations can help you nail down
the demographics that you want.
Everything from age and gender down through skill,
tools, the kinds of jobs that they have.
So if you're testing a travel application
versus a sewing application, you're going to have
different users that you want to get involved with that,
and these organizations can help define some of that.
They also tend to have some brand recognition.
You know, getting an email from Adrian Rosseli, that guy,
saying: hey, how would you like to come to my house
and participate in a study?
I mean, that's lovely if you reply yes.
And a little bit alarming.
But realistically, if it's Lighthouse Organization
of San Francisco, or the, now I've forgotten their name
in Boston, but they have name recognition,
even if I forget it.
They're gong to get a higher return rate on participants.
They're going to get people to participate
in a way that you will not.
They will assume that the study is pre-vetted.
This can only work in your favor, so you wanna ally
yourself with these organizations, wherever you can.
And remember that they have existing relationships.
They already have people that they probably have
participate in studies.
They probably already have done studies before
and can even introduce you to other organizations
or find specific participants.
So lean on those organizations when you can.
You're not an expert in that specific disability.
They also can provide contextual support.
Things like skill level, ability, things that might,
trigger is a little strong of a word,
but things that might cause a lot of stress
for the participants, they can give you
a lot of help on that.
They can also help you with venues, with messaging,
with structuring the test, and in my case,
they did a great job by giving me maps.
Because I did a lot of in-home studies,
and they mapped out all the routes for me to take
to get to the people's houses,
which was awesome because you can't ask a blind
participant how to get to their house.
What road do I take?
Ideally, let the organization do the recruiting for you.
They can send out the emails, they can make sure
that the language is good, they can vet everything.
They can make sure that they're
sending it to the right people.
Let them do that heavy lift.
Some of them do it for free,
some of them charge a fee for it.
Be happy to pay for it.
It's something you don't have to stress about.
Can you hear when it gurgles on the microphone?
It's this, it's not my stomach.
Anyway.
Let the organization recruit.
It pre-validates the study to a lot of the participants.
Accommodation.
Accommodating your participants can be kind of tricky.
There's no real story with this picture,
other than I hope he stopped before he hit her.
Because that would be awkward.
For those who can't see it, it's a toy scooter
with a toy girl in front of it.
No humans were harmed in the making of this photo,
except, you know, the person who's toy fell off,
into the crevasse.
Yeah, anyway, you might find really quickly
that a scooter and a dog can overwhelm almost
any testing facility.
A dog, alone, can send some places in to panic.
So be aware of that.
Make sure that you build in extra time for every task.
Every task.
Bathroom break, extra time.
Drink of water, extra time.
Spending time on a screen to click the one button,
that one call to action that you have,
that's the big orange button in the big white field,
extra time.
Always allow extra time.
You don't know their skill level, you don't know how
easy it is to use that screen.
That's why you're doing this study.
Surfing with assistive technology, in particular,
is a bit slower than you might be accustomed to,
for a variety of reasons.
So don't rush them; allow them to take breaks.
Allow them to rest whenever they're feeling fatigued.
This is not a pressure test.
If you're doing that kind of study,
that's a whole different study.
There are lots of ways you can do that,
that's incredibly painful to watch.
I'm not gonna talk about that here.
Allow them to leave early.
Allow them to start late.
Just allow that kind of flexibility.
You might have discovered that they need more time,
and they might have separately discovered
they need even more time.
Like I said: allow them to be late.
Remember, they're at the whim of everything
from transportation services to your poor instructions.
And everything in between, so don't stress
if they show up late; do the best you can.
This is, this single sentence, there's a lot to unpack
there from experience.
Service animals, canes, wheelchairs,
you name it, they do not play well with tripods,
and they do not play well with cables.
I have had a lot of equipment unplugged
and yanked off tables over the years,
even when I thought I'd taped everything down
and wrapped it all up, and people are really amazing
in their ability to mess up my poorly set-up environment.
But yeah, just remember that you're going to run into
struggles and considerations you had not considered.
It's gonna happen.
Also remember: don't pet the service animal.
The service animals are there to do a job.
I'm gonna have to, I'll stress this a few times
throughout here, but don't pet them.
Let them do what they're there for.
Make sure that the service animal has a clear space,
under the table.
Under the table is a great place to put cables, for example.
My first experience with a cable mishap,
the dog got up, the monitor went down.
Anybody want a slightly broken monitor, by the way?
So make sure the dog can lay at the owner's feet.
If it's a dog, whatever the animal is, typically a dog,
leave that space free of power bricks, as well.
Those can get pretty warm and can be uncomfortable
for an animal, and ultimately the owner is going to tell you
where to let the animal go; some will sit to the side,
some will want to be under the chair, some at the feet.
Sometimes the owner will take the dog off of the harness
and say: go nuts.
In which case, now you have a dog to manage.
So if you didn't pack your lunch properly,
the dog is not working anymore.
The dog is like a four year old.
So be aware of that as well.
They will get into everything.
Consider different seating options for your participants
as well; you have to really think about different body sizes
and shapes, and you might wanna have
a number of chairs available.
Some people will prefer the stability of solid feet,
like the chairs you're in now.
Some people will want wheels,
because they'll need to move around.
Not everybody who comes in in a wheelchair
is going to want to stay in the wheelchair.
I ran a study where one of the participants
was bigger than we had expected,
and the chair that we had provided was tiny.
I could barely fit in it, and it had these little arms
on the side, and it was on wheels.
And she sat on the very front of the chair,
the very front edge of the chair,
and it kept rolling back from her.
So it was a terrible, uncomfortable experience for her.
I scoured the building to find a chair for her,
and we only found one toward the end of the study,
but it also kept her distracted for the entire study.
My mistake.
You know, a valuable lesson there.
It's amazing how much the seating condition
can really, really screw up a person's ability to focus.
I actually saw that, thank you for that.
This is very meta.
He told me I have 10 minutes to go.
I'm gonna go over.
Don't you guys worry, I've got plenty more.
Also, as I said, people who come in in a wheelchair
or a scooter, they might not want to stay in it.
So make sure there's a place for them to park it.
So think about the size of your room now.
You have a wheelchair that's got big saddlebags,
and a dog that's roaming around,
and you have five extra chairs for the different people,
to accommodate them all.
At a baseline, you need a room
about this size for one person.
I mean, that's hyperbole, unless the dog is really wired.
I don't think anybody uses a greyhound as a service animal.
But if they do.
Yeah, okay.
Technology.
This is a setup that probably won't work very well,
but it's a general concept
that I wanted to at least show you.
This is somebody who's playing on a mobile device,
and it's also being recorded and broadcast on a screen,
up here; also consider mobile devices,
people might be holding them to their ear or not, as well.
But the general gist here is you want multiple ways
to capture what's happening on the screen,
just like you would on any particular usability study,
but there are some special accommodations to consider,
and I'm going to run through some of them.
The most critical: use the participant's system.
The person who's coming in for this study,
if they can, have them bring their own kit.
It's already configured, they know how to use it,
there's nothing special they have to do.
By the way, if you're making them test over a mobile
connection, when we talked about compensation,
remember you're hitting their data plan,
so there's some fees associated with that.
For compensation.
You don't wanna waste time getting a user,
getting a participant acclimated to your weird setup.
You know, you're running Jaws at normal, human speaking rate
and they're just gonna pass out out of boredom.
Don't waste time doing all of that.
If you're testing at a community organization,
they've already got systems configured.
You can typically just lean on those.
If you're doing any mobile testing, don't use those mounts,
where the phone rests in the hand and they have a little
camera that points over it.
Think about how many blind or low-vision users
rely on the screen-reader on their phone.
Yeah, that mount's going right into their skull.
So you can't rely on a mount, because they're going to
use the phone in ways you did not expect.
A lot of them will interact, maybe, with the screen down,
and they control it this way, some sideways,
some of them are doing it like this.
So you can't have that phone that they're using
clamped on with gear; that will be a huge problem.
Do not mess with the user's configuration.
Period.
Do not mess with it.
Remember, this piece of equipment they're bringing you
is their lifeline; it's how they do their job,
it's how they interact with their family,
it's how they get food, it's how they do everything
that they need to survive, and if you mess up
their configuration and they leave there
and they turn on that system and it doesn't work,
these are not typically highly-skilled people.
Just like the rest of the world,
they're not typically highly-skilled people.
They get the system configured the way they want.
This is why every time I go to my parents' house
for a dinner, the first half hour is me,
spent reconfiguring their system.
It's not unique to my parents.
But that will also taint your results,
and it will frustrate the users.
If it's absolutely necessary, though,
so there have been times we've had to turn down
the speech rate on a screen-reader.
Okay, make sure you make notes of what you did,
and you bring everything back to where it needs to be.
I see you panicking with that time thing.
I'm panicking now, too.
It's fine, it's fine.
There's no more talks in this room, right?
But definitely return it to the way you found it when done.
You have to make sure everything is functional,
or you've just damaged their ability to, you know,
live their life.
Process.
You might send them any documents and agreements in advance.
You might have not formatted them well.
They might be inaccessible.
They might not have read them, like normal human people.
So be prepared to read them out loud.
Spend some time doing that.
Wherever possible, point the camera at the user,
and all of the interactions.
Ideally, two cameras.
If you can drive a separate display plugged
into their system, even better, so you can watch it.
You don't wanna miss where the assistive technology
is guiding the user.
As I said: driving a second monitor.
Do not interrupt the user when using assistive technology.
Particularly critical for a screen-reader.
A screen-reader, when it comes to a page
or an application, announces all this really
critical information at the start,
and if you interrupt them, they have to start all over.
Don't do that; always let the assistive technology
do what it's doing, let it happen.
The user will make mistakes.
It's your fault, not theirs.
Doesn't matter what they did, it's your fault.
They knocked over a monitor because they waved their arm?
Your fault.
Their dog got into your lunch?
Your fault.
The application caught fire
because they pressed the wrong button?
Your fault.
Never the participants' fault.
They're already going out on a limb to participate,
you wanna get good feedback from them,
make sure that you're getting all the feedback you can get.
And be comfortable taking the blame for everything.
Users will apologize for finding errors.
Whose fault is it?
(muttering)
Thank you.
I have, I tried.
I really tried.
Privacy.
So in the states, we have something called HIPAA.
I don't know what the equivalent is here,
but the general rule of thumb here is:
treat it all confidentially.
Personal health information will be revealed.
You don't need to ask what their situation is.
You can ask the technology they use,
but don't ask them why they use it.
Ask about the technology.
You're going to find things out, this person's blind,
it's gonna be kind of obvious.
Accept that it will be revealed, but be prepared
to treat it as confidential.
It's about respecting their privacy,
making sure that you're not asking too much information,
and in any reporting you do, anonymize the data.
Please, anonymize it; first names, simple information.
Don't give up too much detail.
Especially if you're doing video clips,
clip out things you don't need,
crop out stuff that doesn't belong in the photo.
Avoid photos that show unrelated parts of the scene.
If you're coordinating with a recruiting organization,
they will help with this.
They're going to have some familiarity with the preferences
from their participants, things they want to share,
things they don't wanna share, and they'll give you
some guidance on the best language, and the kind
of language you can use in the materials, as well.
How am I doing on time?
Oh, whew.
Good news is, we're wrapping up.
(laughing)
Yes!
So I'm gonna still go over, just so you know.
CSUN conference for, conference for assistive, ah,
what's it called now?
Assistive technology conference.
They changed the name a couple years ago.
Joe Dollson, he's on the Wordpress accessibility team,
sat down with Leonie Watson, who is with the
Paciello Group, she's blind, she's also a developer,
and she's an expert screen-reader user.
He sat down with her to test Gutenberg.
Gutenberg is not ready for straight usability testing.
There's still a lot of accessibility issues.
So I would never suggest that we do everything I've talked
about yet, that all needs to be hammered out
to do the usability testing, but this setup was very simple.
He did it in a bar.
They weren't even drunk yet.
That might have been a mistake,
but it was a great opportunity to get some immediate
feedback on how Gutenberg performs with a screen reader.
It can be very simple to set up these kinds of tests,
depending on your participants, depending on the context.
A lot of ways to do it, so don't feel too restricted.
I will be posting these slides later.
These are some links to some other resources.
That first one is an article that came out in March,
well after I wrote this talk.
Just so you know, he stole everything of mine.
I will be speaking at Wordcamp Europe, and Wordcamp Buffalo.
You should come; we have chicken wings.
(laughing)
Two different topics.
And again, my slides will be, they're not there yet, at:
ROSEL.LI/WCLDN.
Yeah, and I think I'm done, right?
Am I right on time?
(muttering)
I've got a minute to spare?
(laughing)
Alright, do we do questions or what?
-: Yes, we do, thank you Adrian.
-: Alright, thank you.
(applauding)
(unintelligible murmuring)
-: So, like two?
-: No more.
-: Cool.
Questions?
Sammy, are you just doing it as a courtesy?
Sammy: Yeah.
-: Thank you; I appreciate that.
-: Okay, that's was a great talk.
I was wondering how (unintelligible muttering),
and how many people are in the same group, at the same time?
-: When I do the studies, there's usually two people
running the study in the room, and one participant.
And sometimes an animal.
One person is running the study,
and the other person is the one who's making sure
all the equipment is doing what it's supposed to do.
Keeping the camera working, making sure if the laptop
acts funny we can fix it, or if they drop the phone,
we hand it to them, and it allows the person
who's focusing on just running through the script,
or the study itself, can focus on just that.
So two people, plus a participant,
and then behind that mirror, anybody you want.
Sammy: How many overall do you have?
-: How many overall?
Oh, I'm sorry, how many participants do we have?
(unintelligible muttering)
Overall, it depends on the client.
Typically, we'll do an hour long session,
with an hour break.
So we'll get four to five people in per day.
And we'll do it over three days.
So what is that, 15 people roughly?
Yeah, but it depends on the client and how much
they're willing to spend and the venue and everything else.
-: Thank you.
-: Sure.
-: Any more questions?
(muttering)
-: Thank you.
-: Really, really good talk, and I was wondering,
especially for (unintelligible muttering)
what kind of (muttering) are we testing?
Because there's so many different ways
(unintelligible muttering),
where you kind of cover a lot of stuff,
or could it be specific--
-: It depends, it again depends on the client.
So we've had clients who've said: hey, we've built
this thing and we've done some custom Jaws scripting.
And so we'll find Jaws participants.
We've had other clients who've said:
we just wanna know how this works.
And we'll bring in a variety of disability types,
and we don't ask what their assistive technology is,
in advance; sometimes it's not relevant,
and sometimes it doesn't matter because we're not
going to get specifically the ones we want.
So each study is unique and each one has
different outcome objectives, output objectives,
so it's highly variable.
Did I see another one?
Woman's Voice: Hi, I'm interested in what kind
of training I could do to get into this area.
What kind of, where do you start?
And thank you for your talk, it was great.
-: That's a great question.
I gotta tell you that I got into this accidentally,
because I worked with a lot of agencies over the years
who would run regular usability studies,
and then I would start to fold in people.
So I learned how to do the usability studies
from traditional ad and marketing agencies,
and now I work with a guy, David Sloane,
who is out of Edinburgh, he works for the Paciello Group,
and he has a PhD, which I guess is an impressive thing,
and he's a researcher who does lots of studies.
He's written some good stuff on this.
Sarah Horton has written some good stuff on this.
As for training, I don't know what to tell you,
because I didn't do a structured formal training,
I just kept following people who were smarter than I,
and they told me what not to do,
because I kept doing it wrong.
That's pretty much the best advice I can give you.
Screw it up a lot, and when people stop getting angry at you
you're probably doing it well.
(laughing)
That's how I've survived.
-: Great, any more questions?
No?
-: No, cool.
MC: Thank you, Adrian.
-: Alright, thank you.
(applauding)