Kris Anne Kinney works as an accessibility specialist for Educational Testing Services, a company that is a leader in assessments. The importance of accessibility in online testing can’t be understated. Kris Anne tells us a little bit about the challenges of going from paper-based to online assessments.
Transcript
Nic: Welcome to the Accessibility Rules Podcast. You’re listening to episode 27. This episode has been sponsored by patrons like you. I really do appreciate your support. If you’re aren’t a patron yet and want to support this show, please visit Patreon.com/Steenhout. That’s Patreon.com/Steenhout. I’m Nic Steenhout, and I talk with people involved in one way or another with web accessibility. This week, I’m speaking with Kris Anne Kinney. Hi, Kris Anne.
Kris Anne: Hi, Nic.
Nic: Thanks for joining me for this conversation around web accessibility. I like to let guests introduce themselves, so in a brief elevator pitch style introduction, who’s Kris Anne Kinney?
Kris Anne: Thank you for having me, Nic. Kris Anne Kinney is the accessibility specialist at Educational Testing Service in Princeton, New Jersey, more commonly known as ‘ETS’. I am, along with my full-time job, I’m also a mother of two, small girls. One is getting not so small anymore. Almost nine and four, and I’m I guess what you would call ‘One of the new kids in the accessibility world’. I came from the world of IT, and worked my way with just the passion about accessibility into this field, and I’m really excited to be here.
Nic: Wonderful. A full-time job and two, young kids, that keeps you a little out of trouble, I guess.
Kris Anne: If only. If only that were true. I tend to find myself in lots of trouble anyway and I don’t have the ability to say no to a lot of things, so I end up involved in everything and helping everyone, but I still end up in trouble.
Nic: Cool. Kris Anne, tell me one thing that most people would not know about you.
Kris Anne: One thing people would not know about me is that I should have been in accessibility for a long time, if I had just actually listened to the signals and the signs that the world around me was giving me. I thought that IT and computers was what I was meant to do because I was good at it, but in actuality, I wasn’t really following what my heart was telling me, and that was that using technology can really help someone with a disability have equal access and do things that maybe they could never have done before, and if I had just paid attention to all of the things in my life, even as a kid, as a high schooler, I would have made this my career from the minute I stepped into college. I would have figured out how I could make this a career even though college tracks don’t really lend themselves to accessibility, or that as a college track, I should say, that I would have been in …
Nic: Yeah.
Kris Anne: Yeah. I wouldn’t be the new kid. That’s what they would not know about me. If I had really paid attention to my life, I would not be the new kid. They would already know who I am.
Nic: Right. Do you needed the background before you actually got into accessibility?
Kris Anne: I think I did. I think I really did. I needed that eye-opening moment to just go, “Oh my gosh. This is something that I could do with the skills that I have acquired, and I need to figure out how to do it.”
Nic: We’re talking about web accessibility, and there’s about 15 million definitions of it. How would you define web accessibility?
Kris Anne: To me, web accessibility is just the very basic ability for anyone to get on a computer and buy something that they want or do the research on a topic that they need, or find out services that are available for them in their areas, or for their children in their areas. Accessibility isn’t … To me, it’s also usability, being able to use a site, to understand the information and know that it’s being conveyed in a clear manner.
Nic: Not that you don’t really talk about people with disabilities in terms of accessibility. You say people doing things.
Kris Anne: I don’t really think it’s just for people with disabilities. I think it’s for … It really has a much broader reach than that. Obviously, it’s meant to give equal access to anyone. People who are older.
I mean, my mother is … I won’t give her age away, but she’s aging, and the internet is hard for her. She doesn’t understand a lot of online shopping, things that people my age and younger have been brought up with. She doesn’t understand. It’s not intuitive to her how to add things to her cart and the checkout procedure, and finding all of those things, and so I feel like web accessibility is just as much about people who are new to the internet, people who are aging, people with disabilities, people with low mobility, people with low literacy, parents who find themselves needing …
Nic: Yeah.
Kris Anne: There’s an article that I read once about a mother whose child was born with Down syndrome, and she said sort of like, “Learning French, and then the plane takes you to Spain, and you don’t understand the culture, and you don’t understand the people”, and that’s how I feel about web accessibility. It’s about being able to convey information in a manner that anyone can understand it.
Nic: That’s a great analogy. Thank you.
Kris Anne: You’re welcome.
Nic: Where does your role fall within the work of accessibility as in like day-to-day? What do you do?
Kris Anne: Day-to-day changes every day, and that’s I guess one of the good things about what I do. Some days, I am reviewing assessment content, so Educational Testing Service is a, it’s a research company that writes assessments for education, high-stakes assessments like GRE and Praxis, international assessments like TOEFL or TOEIC. We also do K12 as well. I am reviewing item content, item graphics for accessibility for color, content for usability. How can you use it with the system technologies? Can a score report that’s sent to a student in a PDF format, can that be read by a Screen Reader?
Can it be read by someone who has vision issues? My days are quite different. A lot of it is just reviewing things, educating people about what accessibility is, and helping ETS have the most accessible environment that we can have.
Nic: How’s that going?
Kris Anne: It’s a lot of work. Educational Testing Service has been around for 70 years, and we have 70 years of history and 70 years of the way we used to do it. How about we put it like that?
Nic: Yeah.
Kris Anne: When you go from paper-based testing to computer-based testing, it brings in all kinds of issues beyond the accessibility of it. You have people who want a computer-based test to look like a paper-based test because that’s what test-takers were used to.
Nic: Right.
Kris Anne: In trying to do that, you put up accessibility barriers, and so we have to really get people to understand that, “You know what? Students now who use the internet, they’re used to just things that look like they’re on a webpage. They’re not used to paper-based tests. They don’t know what paper-based tests are.” I mean, I do because I’m old, but students now in K12, they don’t know what a paper-based test is, and so there’s a lot of nervousness is a good word. It’s not really resistance.
It’s nervousness that if we change all of these things, what will that mean for the test scores? What will that mean for the things that we tell people that their students are capable of? How will it change? Sometimes, people get nervous about that. They want to be able to report accurate information to schools, to people, to colleges, to states, and changing the way that we offer things can sometimes change those scores or change all of those things, so there’s so much more than just the actual content.
It’s all the backend stuff about scoring, and there are people here that are beyond brilliant. We call them ‘Psychometricians’, which when I came to ETS, I don’t think I’d ever even heard that word before, but people who really think about how you learn. The research area here is immense, and so a lot of it isn’t just the accessibility of the content. They’re starting to grasp that. It’s all of the other things that come along with it that every day, I get up on a soap box, and I give my spiel about why it’s necessary and why it’s good for ETS moving forward.
Nic: Yeah.
Kris Anne: Then, people go, “Oh, all right. Now I understand it.” “When you put it that way” is actually a phrase I hear quite often. “I guess when you put it that way, yeah”, and now I can see how we can do it. I ask people all the time, “What’s another way to ask this question? What’s another way to do this?”
Think outside of what you’ve always done and try to come up with an innovative way to solve this challenge that we’re facing.
Nic: I love this because we speak, I speak to a lot of people anyway about accessibility, and this is really the first time I’ve spoken with someone who is exploring the issues of online assessment accessibility, and I think it’s brilliant that it’s being done because we hear about a lot of issues with access to computers and education. Just yesterday, there was a Twitter thread that I participated in where I was reminded that when I was taking MBA classes, both the lecturers and the library staff were saying, “Oh, you shouldn’t be taking notes with computers, and you can’t write your exam with computer”, and I kept saying, “Oh yeah, but my hands don’t work so well. I can’t hold the pen for more than five minutes. What am I supposed to do?”
Kris Anne: Right.
Nic: I love that there’s this push from such a big institution as ETS to go electronic, but I can see the issues of, I guess I would compare it to legacy code problems when you have decades of stuff of we’ve all, we’ve done it this way.
Kris Anne: Absolutely. Absolutely. Right. The thing that I fall back on a lot with people is I ask them to think about why ETS was started, and ETS was started so that colleges weren’t just for rich, white men who were legacies. Talk about legacies, that’s pretty much what college was 70 years ago.
It was the Ivy League Institutions that only allowed a guy whose father and grandfather and greatgrandfather had all gone to the school and they’ve given millions of dollars. The building that I sit in is Conant Hall, and James Conant actually have the idea. He at the time was the President of Harvard University. He said, “You know what? Education should not just be for these people.”
“Education should be for anyone who can attain it”, and so he decided to come up with a way to standardize an assessment to see what you know. That’s what an assessment really is just at its core. It’s, “What do you know?”, and based on what you know, you should be able to learn more if you want to. You should be able to go to college, and so when I say that to people, they go, “Oh yeah. That is it”, and so we’re still doing that.
First, it was just giving access to colleges, and then it was making it fair for women, and then it was making it culturally diverse, and now, we’re just getting over that next hump of making it accessible to everyone, so we’re just still working on that original mission of 70 years. It takes some time.
Nic: Yes. Yeah. It does take time, but it’s good.
Kris Anne: Yeah.
Nic: How did you become aware of web accessibility and its importance? You mentioned you had an enlightening moment, but you had a lot of pointers before. Tell me about those pointers. Tell me what suddenly made the penny drop?
Kris Anne: The pointer was when I was in high school, I was at an aid … I hate using that word, but I think that’s actually what my job title was. As a sophomore in high school, I needed a job, and I went into my guidance office, and they said, “You know, Kris Anne, you have a really easy manner about you. We have a student who’s blind in the middle school, and they need somebody to pick him up from school every day, bring him home, make sure he does his homework because mother works full-time, and so the state is willing to pay for someone to help him to get him home, get him to do his homework, make sure that he’s all settled before mom gets home.” Little did I know that I was going to be dealing with probably the most headstrong fifth grader on the face of the planet, who used to run out of school every day so that I wouldn’t be seen picking him up. He used to try to beat me home before I could get to him to walk with him home.
He hated walking home with me because he could do it himself. He was extraordinarily independent, and I love that about him. I still do. I would help him with his homework, and I remember sitting there with a sheet of paper, which was the worksheet, and he would have three braille pages for that one worksheet, and I remember sitting there, going, “This is incredible.” It was amazing to me that they took this one worksheet, and then they’ve turned it into something that he was reading the exact same content that I was, and we were working on it together, except he was reading it in braille, and I would mark his answers on the worksheet that I would do with him, and then, we would submit that for his teacher, and it was just …
Nic: Yeah.
Kris Anne: I should have known then that because I just, I loved that job. It was the best, and when he decided to go to high school, he really decided he did not need, or I think it was seventh grade, he was done with me.
Nic: Right.
Kris Anne: He did not need someone to help him anymore. Then, when I went to college, one of my classmates had a muscular dystrophy. He was in wheelchair, and he had the most wonderful service dog. I think I told you about [Laddie 00:15:53].
Nic: Yeah.
Kris Anne: He’s a beautiful Collie, and David was extraordinarily smart, and people might look at him and think that he wasn’t, but he was. He was so smart. He was always helping us with all of our work, which he was just a great, really easy-going kid, but it wasn’t until I went back to TCNJ. I went to The College of New Jersey as an undergrad. Actually, I went to Trenton State as an undergrad, but then it become The College of New Jersey.
When I went back there to work full-time in IT, one of the professors in the Special Education Department, her name is Amy Dell, she’s quite a woman, and she was talking to me about the computer lab that they have in Special Ed Department, and this was my ‘Ah-huh’ moment, and she had all this assistive technology in this lab. She had scanners so that Kurzweil could read things. She had JAWS. She had things that I can’t even remember anymore because it was 12 years ago, and I’ve had to flush a lot of stuff out of my head, but she had all this assistive technology tools and she wanted my help in making sure that everything worked, and I looked at her with wide eyes like a deer in the headlights and I said, “Okay. I will figure this out, but can you explain to me what it’s for?”, and she started just explaining assistive technologies to me, and it was like someone had smacked me in the face and said, “Idiot. This is where you should have always been”, because I just I thought about all of those kids that I had known, and having all of these tools, how much easier their lives could have been, and so I said, “Okay.”
Nic: Right. Yeah.
Kris Anne: “Yeah”, and so I learned everything that I could from her from a lot of the other Special Education professors at The College of New Jersey. They were all just great people who are willing to help you learn because you’re interested, so they would easily want you to learn. That was my ‘Ah-huh’ moment. It was just this with the computer, the thing that I’m comfortable with, I can make all of these things run and work, and you don’t need braille anymore because you can take a printed sheet, and scan it, and read it, instead of brailling it. It’s just that moment that I continue to have sometimes as new technologies are introduced to me, because for as long as I’ve tried to learn about this technology, I know that there’s a ton that I don’t know about, and I like that.
Nic: Yeah.
Kris Anne: I like learning about new things. I don’t want to know everything because then, what would make me excited about my job?
Nic: Yeah. Yeah. What barriers did you face or are you now facing in terms of implementing accessibility? Yeah. You mentioned a little bit legacy code, and then, what legacy formats, and how are you getting over those?
Kris Anne: Slowly but surely. A lot of the things that we encounter are because things are done without the help of my group, they talk to the product owners and project owners and some of the programs, and they get their requirements, and they don’t include accessibility, so that really is one of our major barriers, and we’re trying to get ourselves in planning stages of programs and assessments, rather than towards the end where they’re like, “Can you check this?” It’s like, “No. Now is not the time.” It shouldn’t be now when you’re asking us to look at this.
It should be from the beginning. When you’re designing, you’re developing this. Say, “How can we make it accessible going out the door?”, and some of our programs, some of the newer programs are doing that, and we’re trying really hard to get the word out to the older, the more entrenched programs because that’s the only word that’s come into my brain right now, but to say that, “Look. You have refreshes, so as you have that refresh, let’s start talking when you’re making new questions, when you’re designing a new interface, when you’re trying to develop new science assessments. Let’s talk about it now before anything gets written, before any word gets written so that you can understand how to create an accessible question.”
Nic: Yeah.
Kris Anne: “Let’s start at the basics”, and then we’ll work with the test delivery system, because you have two pieces.
Nic: Yes.
Kris Anne: It’s the item itself and how the item interacts with assistive technology, but then, you have the test delivery system and the accommodation the test delivery system can or cannot offer. Those are the two kinds of the battles that we fight simultaneously. I don’t like the word fight because it’s not really a fight. It’s more just an enlightening if you will.
Nic: Yeah.
Kris Anne: Enlightening the test developers, enlightening IT that these are all important things.
Nic: Yeah. What’s your favorite word?
Kris Anne: Possible.
Nic: Why?
Kris Anne: Because anything is possible. You just have to want to do it, and I don’t like the word ‘Impossible’ because I feel like that just stops you in your tracks. I like the word ‘Possible’. I don’t like ‘Definitely’ or ‘Half dues’. I like possible because anything is possible.
Nic: Thank you.
Kris Anne: You’re welcome.
Nic: What would you say your greatest achievement is in terms of web accessibility?
Kris Anne: My greatest achievement thus far is bringing a greater understanding to people and having them want to make changes to accessibility. I say that because I’m not the one who’s actually making those changes, so I’m not the one writing code. I work with very brilliant programmers who do that. My job is to educate people. My job is to show them there’s always that moment when someone hears JAWS for the first time in that Eloquence voice, not one of the really nice vocalizer voices, but in that Eloquence voice.
They go, “Oh my God. What is that?”, and you smile and you say, “That’s how someone who can’t read reads this page”, who has no vision, who has low vision.
Nic: Yeah. Yeah.
Kris Anne: They go, “But it [denounce 00:22:50] … What is all that stuff?” That’s what they say. “What’s that stuff it’s saying?” It’s saying all of the things that are non-visual that are behind it, all this junk.
Nic: Yeah.
Kris Anne: I lovingly refer to it as ‘Junk’. When you show them that versus something that’s done well, that watching someone have that ‘Ah-huh’ moment is a wonderful thing.
Nic: It’s funny you mentioned that because next week, on Monday actually, I’m going to be speaking in Vancouver at the ConFoo Conference, and my presentation is just that using a screen reader to compare bad code and good code, and go through that and demonstrate to the people how it works, and I’ve given this conference a couple times before, and the face on people is just amazing as you were talking about an ‘Ah-huh’ moment, and that’s really what it is for them, so I can imagine the reaction you’re having when you’re presenting JAWS to your developers.
Kris Anne: Right.
Nic: That’s cool.
Kris Anne: Right, and it’s interesting. When you present it to anyone, when you present it to the marketing department and they understand how a PDF sounds or doesn’t sound if it’s not correctly formatted to be accessible. I had someone say to me, “Oh, I know this PDF is fine, but can you look at it?” I love fine. I’m like, “Okay”, and it read nothing.
It said, “The content on this page is an image.” Would you like me … I went back to her and I said before I did the Adobe’s build-in, OCR to try to get any information about it, I said to her, “Okay. This is an image.” She said, “What do you mean?” I said, “It’s an image of text. It’s not text. There’s nothing a screen reader can read on it.”
Nic: Yeah.
Kris Anne: “What do you mean?” People don’t understand what a PDF truly is, is that it’s kind of a stagnant if you will image of text initially when you create it, and then you have to do things to it in order to add those elements in.
Nic: Yeah.
Kris Anne: It’s moments like that where people think, “Oh, we’re fine”, and you have to be the dark rain cloud and say, “No. No, you’re not fine, but you will be if you learn how to do it or if you know what to look for.” Unfortunately sometimes, people are very trusting of people or external companies when they say, “Oh, we create accessible things, code. We create accessible widgets. We create accessible code. We create accessible documents”, and they get something back that isn’t quite there.
Not quite there could mean somebody not hearing what their final test score was because that element wasn’t readable, so it’s sometimes the little things.
Nic: Yeah. How can companies trust third party vendors that say, vendor X, Y, Z says, “We make accessible products”, and how can a company trust that statement?
Kris Anne: I find that it’s really hard to trust that. I hate to sound untrusting, but at the same time, I’ve seen it not be accessible on a few occasions. I’ve gotten VPATs from companies that say, “Oh, we do this, this, this and this”, and they send me examples, and none of those examples do the things that they say that their VPAT does. Now, that’s not to say that their software can’t do those things, but in all honesty, if your software can do that, why wouldn’t someone want to implement it if you’re …
Nic: Yeah.
Kris Anne: I think the way to ensure that you get something accessible is, especially in the beginning and as you continue with contracts, you have to check their work. You have to show them that you’re paying attention on that. Sometimes you’re not taking them from their word, that you’re actually going to call them out on something that’s incorrect, and I’ve done it before. They get really angry, but at the same time, we’re paying you a lot of money to do something like this, and you’re not doing it correctly, and if you put this out there and say that it’s accessible and it’s not, who does that come back on?
Nic: Yeah.
Kris Anne: That comes back on us as a company, not necessarily you, the vendor who created it, and so I think a lot of times, people have to be, for lack of a better term, strong-armed into being careful about what it is that they produce, being careful about who they hire and actually making sure that they’re training their staff correctly to do this, not just saying, “Oh, do you know how to make PDF? You know how to tag it? Okay. Great. You’re hired.”
Nic: Yeah.
Kris Anne: That’s not enough to know about accessibility, especially PDF is just stuck in my head now because that’s what we were talking about.
Nic: Yeah. Cool. That’s wonderful. Thank you. Look, I think we’re going to wrap this segment of our chat for this week. Kris Anne, thank you for your fantastic and candid answers to my questions.
Kris Anne: Yeah. Thank you for having me.
Nic: We’ll finish our chat next week.
Kris Anne: See you next week.
Nic: Thank you for listening, and until next week. That’s all. Before I go, I want to thank my patrons once again, and remember that if you need a hand ensuring your site’s accessibility, I’m available. Contact me on my website at Incl.ca.