While virtual reality (VR) has been on the radar for decades, it wasn’t until recently that its use has become less of a novelty, and more of a consumer reality. Now, as VR technology becomes more ubiquitous, the big question is: How will this emerging technology enter the classroom and assistive technology space? In this session, Jaclyn Wickham, founder of AcclimateVR, explored some of the current and future applications for virtual reality (as well as some info on applications of Augmented Reality) to support students with autism and other disabilities. (Get the PowerPoint slides and other webinar resources in the Download Here section).
- [Todd] Hello everyone, and welcome to the Center on Technology and Disability's latest assistive technology webinar. Today we are looking at some new technology possibilities, with presenter Jaclyn Wickham, a teacher turned technologist and founder of AcclimateVR, an ed tech startup developing virtual reality based learning modules to help students on the spectrum develop independent learning skills, practice social skills, build safety awareness, and navigate challenging community locations. Today, she will present on the many ways virtual and augmented reality can offer new learning opportunities in the classroom. We'll let her run through her presentation first, though feel free to ask questions via the chat box as we go. We will have a Q and A session at the end, and make sure we get as many questions answered as we can. Just keep in mind CTD always records and archives the webinar, PowerPoint and resources on the CTD website by the following day. If you're looking to revisit the webinar and materials. Welcome everyone and let's get started. Jaclyn, the floor is yours.
- [Jacyln] Hey everyone. My name is Jaclyn, and I am an education technology specialist and founder of AcclimateVR. As Todd mentioned, we are at AcclimateVR, we are an ed tech startup focused on creating virtual reality content learning modules for students with autism and other developmental disabilities. So just to give you a bit of background as to the journey that kind of led me to found AcclimateVR, I have a background in education. I started as an elementary teacher, and then transitioned into education technology and instructional design.
So I transitioned into ed tech after teaching elementary school for about five years. And realizing, after realizing the benefits of technology in the classroom. And as an ed tech specialist, I had the opportunity to travel to schools across New York City, helping teachers implement technology into their instruction. And it was during that time that I became really interested in ed tech and app development, and ed tech entrepreneurship. I had a lot of ideas for educational apps. So I ended up at a graduate program called ITP over at NYU which was essentially a coding boot camp, where I learned to program, and I had the opportunity to explore a lot of new and emerging technologies, like augmented and virtual reality.
And while I was there, I ended up taking a course on developing assistive technology, and that's when I really started to become really interested in assistive tech. And I ended up teaming up with an occupational therapist there for a couple of projects in that course. And through our collaboration, was born the idea for AcclimateVR. So it started as my graduate thesis project, and now has sort of grown into a full-time passion. So our mission at AcclimateVR is to leverage virtual reality technology to help students with autism and other developmental disabilities experience real world situations and develop everyday living skills in a safe environment. And our VR teaching tools are designed to help children and teens on the autism spectrum develop independent living skills, practice social skills, build safety awareness, and navigate challenging community locations.
And so we're focusing on these four different areas, and I'll elaborate more on this later in the presentation. But first I'd just like to start with a quick agenda just to let you know what we'll be covering in today's webinar. So I wanted to start out with just talking a little bit about the difference between VR and AR. Because we will actually be talking about some augmented reality applications as well as virtual reality today. And I want to also make sure that everybody's comfortable with those definitions, and talk a little bit about the different types of VR. And then we will transition into talking more about the practical applications of VR in the assistive technology space. And then I'll tell you a little bit more about AcclimateVR and what we're building, and what we're learning from our pilot studies.
And at the end, we'll transition into talking about augmented reality and some specific applications in the assistive technology space. So to start out with, I'd love to, as I mentioned, we're just gonna start about talking a little bit about VR versus AR. So I'd like to just open up the, take a quick poll, and open up the chat box here. So hearing from you, from the audience, what do you think the difference is between AR and VR? I'd love to hear your thoughts first, before we jump into this section. I'll give you just a few minutes to answer the poll. Okay, so we've got some answers coming in. Some people are not sure at all, which is completely fine. We're gonna talk about the difference in just a minute.
Okay, some of you seem to be pretty clear on understanding of one being totally immersive and the other partially immersive. Right. Okay, so so far, it looks like we are about half and half. Some of you seem pretty clear on the definition, and some of you aren't sure. So it's a good thing that we're gonna stop and just talk about the difference before we jump into looking at different applications. So thank you for those of you who submitted a response. So this is actually an infographic by Earthly Mission. And it does a really nice job of explaining the difference between augmented and virtual reality. So the way to really think about it is to kind of think of the difference between the definition of augmented reality, we think of augmented it means improved or enhanced, whereas virtual means something that's not physically existing but that's made by software to appear so.
So AR is the blending of virtual reality and real life. And so developers will actually basically create images within app locations that blend in with the contents in the real world. Whereas virtual reality is all about the creation of a virtual world that users can interact with, and it's closed and fully immersive. And so you can think of it as augmented reality essentially puts virtual things into users real worlds, augmenting them, whereas VR puts users inside virtual worlds, immersing them. So, in AR, the user still continues to be in touch with the real world, whereas with VR, the user is completely immersed. And you, I'd like to just take a look at some examples of both, just so that that distinction is really clear.
You are all probably familiar actually with some common applications of augmented reality. If any of you have ever been on Instagram or SnapChat, you will have probably seen some of these face filters. This is a perfect example of augmented reality. You can see the image of yourself through your smartphone's camera, and then you have the ability to add on these sort of silly face filters. So you're overlaying digital elements onto the real world, augmenting it. And just another example here. If some of you may remember the Pokemon Go craze from a couple years ago, and that's another perfect example of augmented reality, where we have the ability to point your smartphone, use the camera on your smartphone and you will see that there are these sort of digital avatars layered onto the real world scene there, augmenting your view. And you may also be familiar with, there are a lot of different AR apps for education like this one, where this one uses, it's essentially like anatomy flash cards. And they superimpose small images on the normal field of vision to help students learn vocabulary.
So that's another example of augmented reality. Now, virtual reality, on the other hand, you can see that the user wears a headset. Sometimes you'll hear it called googles or a head-mounted display, but the point is that they're very isolated or immersed in a virtual world. So virtual reality is simply an artificial experience that kind of tricks your mind into thinking that it's somewhere else by stimulating your senses. So it transports you to another world that can be experienced or interacted with through the use of sensory devices. And depending on the type of headset that you are using, sometimes you can, users can actually interact with the virtual world using hand controllers or buttons on the headset. So just another quick poll out of curiosity. I'm curious if any of you have ever tried VR, and if so, which headset you've tried. I'll give you just a minute to see the responses come in here. So we'll start with the yes and no. Okay, so it looks like, still coming in here.
Okay, so we have, it seems that quite a few of you have actually tried it. We also have some who have never tried it. And it looks like we're, I do see some answers coming in on the chat box as well. We have, okay, it looks like we've had some people try the Oculus, Google Cardboard, oh great. Okay, yeah, Second Life, somebody asked if Second Life counts. That would absolutely count. That's a perfect example of a computer-generated VR, which is actually what we're about to transition into in just a second. Samsung Gear, great. So yeah, you'll see there are a lot of different VR headsets out on the market right now. And they all offer different features. Google Cardboard is essentially the most, the easiest to get started with, as it's just basically a cardboard set of goggles that you place your smartphone into. And then there are some more advanced headsets like the Oculus Rift out there as well.
Most of the examples here use mobile VR, where you can just place your phone into the headset. Okay. Great, that's really helpful. Thank you for those of you who submitted a response. The reason I was asking is I was curious just to see if the types of VR that you've tried out, and so if you've ever tried VR, as it looks like some of you have, there are, you've probably experienced one of two different types of virtual reality. So there is a difference. There is what we call computer-generated VR. and computer-generated VR environments can be created in two different ways. I'm sorry, or basically, are created by computers, and they can look and feel like the real world. Such as a flight simulator, or they can be completely artificial. Think of an imagined distant planet. But any case, they are computer-generated simulations created by computer, and you can think of it almost like a video game type atmosphere.
And Second Life, which somebody mentioned, is a perfect example of computer-generated VR. And often in computer-generated VR environments, you'll see different avatars that look like this, that the user can interact with. So there's also another, you may have also heard of VR, 360 degree images and videos referred to as virtual reality content. And this is basically immersive or spherical videos or images. They are recordings where a view in every direction is recorded at the same time, shot using an omnidirectional camera or a collection of cameras. You can see some examples here on the slide. And 360 degree video is typically recorded using either a special rig of multiple cameras or using a dedicated camera that contains multiple camera lenses embedded into the device and then filming overlapping angles simultaneously. So you can see here, this is an example of a video captured by one of these 360 cameras.
So it starts out, you can see the video footage is recorded with a fisheye lens, and then through a method known as video stitching, basically they stitched the footage together into one spherical place. But I wanted to show you a quick example of that, just so you get an idea of the difference. Sorry, let me see if I can. We'll go through this way, sorry about that. The, you can see here, this is where it's converted. And the end result will look something like this, where you'll get a video, and it's often referred to as VR, as if you were watching this content through a VR headset, you can imagine that it would, if you were able to turn your head and look in all different directions, it would look like you were actually at this real world scene.
So we've talked a little bit about the differences between VR and AR and the types of VR. So I'd like to jump into talking about what most of you are probably here for, is to hear a little bit more about some of the practical applications of VR, specifically in the assistive technology space. Both of these technologies, VR and AR, have the potential to create new applications that support people with disabilities. So often we hear about virtual reality games and advertisement, but VR really has a huge potential beyond just its entertainment value. It's making a huge impact in the health sector. Examples include cognitive behavioral therapy, exposure therapy, treating PTSD, treating phobias and anxiety disorders, as well as stress and meditation.
And it's also being used as a way to help people with disabilities explore the world that might be difficult or impossible in real life. So I'd like to talk a bit about some specific examples of applications, and also a bit, a bit about the research behind VR therapy. I should mention before you jump into this section that a lot of my research, my personal research and experience has focused on how VR can support individuals and children and teens on the autism spectrum. Which is mainly what we'll be looking at examples of today. But please keep in mind that many of these examples and applications that we will be looking at could also support people with other intellectual or cognitive and developmental disabilities in addition to autism.
So we'll be kind of looking through a narrow lens today, specifically at examples to support people with autism. But just keep in mind that a lot of these could be used or applied more broadly. So, why consider using VR therapy for autism? First of all, it creates a safe and controlled environment. So as a lot of you probably know, chaotic situations can be extremely stressful for those on the spectrum. And they sometimes tend to have trouble ignoring irrelevant stimuli, and can easily get overwhelmed in unfamiliar places that are full of new sights or sounds or smells. So a VR learning environment is much more controlled. The student can enter and leave at will, simply by putting on or taking off the headset. So there's a low risk of overstimulation. And learning is much more productive under low stress conditions. So not only are the students much more comfortable, but they're also more likely to make progress.
This second point here is a big one, accessible practice and unlimited repetition. Opportunities to practice social interaction are much more limited for students with autism often, than for their neurotypical peers. And going out into the community can often be really difficult, because more supervision is required to ensure their safety. They also get, sometimes can get less exposure to everyday social situations in school if they're separated from the mainstream classroom, either deliberately or otherwise. But in contrast, VR can be at their fingertips anytime, both at school and at home. And it also lets them practice these same situations as many times as they want, in a safe environment, over and over again, until that skill, until they're comfortable with the skill.
And that's something that doesn't always happen in the real world. A couple other reasons to consider using VR for autism, it can create an individual, I'm sorry, I skipped ahead there. There's an emphasis on visual and auditory cues. And sometimes people with autism will tend to respond better to sight and sound than touch. So VR can be a great match. The ability to create individualized experiences. So, as many of you know, autism includes a wide spectrum of symptoms. You know, it can be from slight social and intellectual differences to a total inability to communicate. And not every student needs the same kind of teaching. So VR allows for a complete individualization for each student. And the therapist can select the simulations that are right for the student, and the simulation itself can respond according to what each individual does.
So that no two students will necessarily have the exact same experience. And as I'll mention, I'll mention these two in just a bit, when we get into the research. But it's also, VR is also beneficial for people with autism, as a lot of them have shown preference for computer interaction, and it also allows non-verbal interaction possibilities. So we'll get into that in just a moment here. So because of all of these considerations, which we just spoke about, academic researchers, as it turns out, have actually been investigating the use of VR as a teaching tool for individuals on the autism spectrum since the 1990s actually, when VR head-mounted displays were huge and clunky, and tethered to a computer. And I'll show you kind of what they used to, they used to look a lot like this. So, and the reason that researchers have been exploring, as I mentioned is kind of intuitive, kids with autism need to practice basic life skills in the real world, and it's hard.
They need a place to go, someone to take them there, time to learn and support. With VR, we can make a world that's convenient and controlled, and perfectly designed for learning some of these essential skills. So one of the first studies on the use of VR with individuals with autism was conducted by a pioneering researcher named Dorothy Strickland, and this is actually just a quick writeup of her case study. They were, sorry I'm hearing some feedback there. Okay. So, one of the, as I mentioned, this is one of the first studies. And they, in the study, they examined whether children with autism could tolerate and respond to VR. So they had two children, age seven and nine, both mild or moderate on the spectrum, and they used VR headsets that displayed a computer generated simulation of crossing the street.
So this is a perfect example of what we were talking about earlier, which would be computer-generated VR, you can see it's sort of a video game-like atmosphere. And the results were actually really promising. The students were able to wear the headset, which of course was much bulkier and heavier. And I can actually show you just another quick picture here from the same study. You can see right here, it looked a lot like that. So the fact that the students were able to, to wear that was pretty impressive, considering what they looked like and felt like back then. Because of course it's probably natural for everybody, most people typically wonder if students with autism would experience any negative sensory effects. So they were able to wear it and verbally identify the objects and walk around the computer generated world. But it wasn't really clear whether the children would translate or generalize their experience to a real street crossing situation.
So there's still some research to be done in that area. And there have been a lot of studies in between. But more recently, we have, yeah, I just saw the question come in about what about vertigo or dizziness, which is a great question. It's natural to ask that, especially considering some of, that some students have some sensory issues. And it depends on the headset that you're using. Some people do experience vertigo with VR. But the study that I actually have pulled up right now, that was one of his focuses. So this is Dr. Nigel Newbutt, from the University of West England. And the main focus of his pilot study was to basically test, to test whether the students were able to, or I'm sorry, the participants were able to wear the headsets and, so they tried it with the Oculus, Oculus Rift. He tested it with 29 individuals with autism. And he did it in two phases.
So the first phase had a virtual scenario for about 10 minutes. And the second phase was 25 minutes. And the scenarios were computer-generated, so you can see the graphics looked something like this here. This is one of the graphics that he showed. And the participants for the most part, as I mentioned, there were 29, four of the participants were unable to complete the first phase because of dizziness and tiredness. Which was the question that just came in. But most experienced no negative physiological effects. So overall, the group reported a strong sense of presence and immersion. And I'll just come back here for a second. So, across, there have been as I mentioned, there have been a lot of studies in between. I showed you one of the oldest studies and one of the most recent.
But basically, across technologies and studies, study participants have demonstrated their ability to engage with simulations with actually very few negative emotional or physiological reactions. And beyond that, VR learning experiences have been shown to produce learning gains measured not only by tests of social judgment, but also by assessments of activities performed in the real world. There is a lot more work to do in order to find out if students are actually transferring or generalizing these skills to the real world. But if you're interested to learn more about some of these studies, you can actually go to our website, acclimatevr.com/research, and you'll see a form there that will ask you to sign up for our blog.
If you sign up for the blog, you'll hear about all the different, different things we're working on and building at Acclimate, and what we're up to. And once you sign up for that, you can actually, you will be sent a free downloadable version of our Research Roundup. So we basically combed all of the research that's out there about VR and autism, and did essentially like a literature review of all the research. So if you are, if you are interested in learning more, please feel free to head over to our website, and take a look at our Research Roundup. So as I mentioned, research has shown that virtual reality is very promising, and has a lot of potential as a tool for therapists to be able to administer virtual reality exposure therapy in a safe and controlled manner.
But we haven't really, it hasn't really been widely available to date until recent years. So, you know, as before, like I said, they looked a lot more like space helmets. But in the last few years, we've seen huge advancements in the technology. It's cheaper than ever. And you now only need something as simple as a Google Cardboard and your mobile phone, which is a $12 viewer, and your phone, to be able to experience it. So the first time ever VR is actually becoming a reality in school settings. So I'd like to talk about some of the applications that are being used, how it's being used in school settings specifically, and also to of course support people with autism and other disabilities. So, sorry. So these are some of the specific applications of VR in the assistive tech space.
It's being used for joint, to work with students on joint attention skills, sensory-based experiences, exposure therapy to reduce anxiety, social stories, community-based instruction, social skills, executive functioning, daily and independent living skills, safety awareness, and social modeling. And I have a few examples of each. Not of each, I'm sorry, a few examples of some of these. But I will also show you what we're building at AcclimateVR, which incorporates a lot of these different skills. So the first example I wanted to just show you is, let me just pull this up, so this is actually, where this plays, this is a startup called Floreo, and they are DC-based, I believe. And they are exploring, doing a lot of exploration with computer-generated VR. And you'll see this is actually an example of an activity to build joint attention skills. So for those of you who don't know, joint attention is an early developing social communicative skill in which two people, usually a young child and an adult, use gestures and gaze to share attention with respect to interesting objects or events. And that can sometimes be really difficult for kids with autism. So I'll just show you this quick. This is I think one of their early prototypes. And you'll see that this is computer-generated VR here.
- [Woman] There's an elephant over there. All right, let's see. Let's play with one of the animals.
- [Emma] Wow, I wonder what's gonna happen.
- [Woman] All right, let's look at Emma. What's the elephant doing?
- [Emma] Neat, look at that!
- [Woman] Nice job watching the elephant with Emma.
- [Jaclyn] Okay, so that's just, again, if you're interested in learning more about them, they're called Floreo, and they're doing a lot of cool things with computer-generated VR. And I think a big, very big possibility for VR, which we're actually working on at AcclimateVR is taking social stories. A lot of you, especially if you're an occupational therapist or work with students with special needs, a lot of you have seen these types of social stories. And imagine being able to take something like this, which students typically use to learn through, you know, these paper social stories and scaffolding skills or working through role playing. But imagine if we could take this and actually make it into a virtual reality experience where they actually see a view of the railroad and being able to go over the crosswalk, and listen for cars, and be able to hear the environmental sounds. So, the ability, so social stories is a big one that we're seeing different uses of for VR applications. So, one other really interesting use that's kind of a little bit different is this video, which was put out by the, let me just pull this up here, Autism TMI Virtual Experience. And this is posted or published by the National Autistic Society. And basically what they did was they created a virtual reality experience using 360 degree video to show other, to show people what it's like to have autism. So they tried to recreate the experience and show what it would be like to have sensory overload. So it was kind of interesting. I'll show you that here.
- [Woman] I'm just gonna get a ticket, okay?
- [Jaclyn] So we can't slow it down. So you can imagine that if you were wearing a headset and watching that, it would be a really interesting way to explore that perspective of somebody with autism. Obviously, that's a pretty extreme piece, but I thought that was an interesting example. So those are just a few, a few examples of some of the work that's being done with computer-generated VR. And now I'd like to transition into AcclimateVR's approach, and talk a little bit about what we're building over at AcclimateVR. So, I'd like to start by just talking a little bit about my main inspiration for Acclimate, and my inspiration for building Acclimate. As I mentioned, it started as my graduate thesis. And it all kind of started over a conversation I was having over coffee one morning with a colleague of mine, Kaitlin.
And she's an occupational therapist. She works with students with autism on a daily basis. And she was telling me about her morning, which she had spent at the subway station across from her school with a group of her middle school students, teaching them how to buy and swipe a Metrocard, read a subway map, ask for help if they're lost, and basically how to safely ride public transportation to and from school. And this is a normal morning for Kaitlin. Trips like these occur on a weekly basis and vary in location, ranging from subway station to the cafe or a restaurant down the street, to the local grocery store. Interacting with the cashier, or ordering from a restaurant menu, or going to a doctor's office, these are all skills that many people take for granted.
But for children and adolescents on the autism spectrum, developing independence and building the skills that go along with it can be hugely challenging. And for many of the spectrum, it's not so easy to pick up things on the fly just by observing others. Additionally, many students with, the nuances of navigating unfamiliar places and interacting with strangers are not always intuitive or self-explanatory. So additionally, a lot of students with autism have challenges with sensory processing. So, for a child who is oversensitive to stimulation, an ordinary trip to the supermarket or a neighborhood can be very anxiety-inducing. So what Kaitlin's doing when she's taking her students to these community locations is called, it's a very widely known instructional strategy called community-based instruction.
So like many, it's used by a lot of occupational therapists and educators who work with students with autism. And so it's an evidence-based instructional strategy that helps students develop functional independence as they approach the transition into adulthood. So evidence shows that the best way to help students develop skills to become more independent is to take them out of the classroom and into the real world, or the natural settings where a skill is ultimately expected to be demonstrated. And a lot of times these field trips, sorry, not field trips, these trips are very structured. And by no means a field trip. As you can see from the data collection form above. So, Kaitlin the occupational therapist, she invited me to observe one of these community-based instruction trips.
And it was really interesting to watch. The therapists each had a clipboard and a Velcro wallet for each of their 15 students. And as we walked around, they worked with individual students on specific tasks, based on their individual goals in their IEPs, keeping track of how much prompting each student needed. But what I learned from Kaitlin, and from numerous conversations and surveys and interviews with a lot of other occupational therapists, is that community-based instruction presents a lot of challenges. One of the biggest challenges is access. Because a lot of schools don't have access to community places nearby where they can take students to practice. So what I saw at some of the schools I visited were rooms like these with mock grocery stores or cafes where students could practice the skills through role playing. And time is also a big issue, as many sessions only last for 30 minutes.
So it's not always an option time-wise for therapists to take students out into the community. Safety is also a concern. One therapist told me that sometimes what they'll do is stand 15 feet back around the corner from the student to see how the student will do independently without the teacher or the therapist standing over their shoulder. But they do have to worry about safety. And lastly, logistics, organizing the trips is quite a process. It requires permission slips, and reimbursement forms, and partnerships with local businesses in the community. And it can really just be a logistical nightmare. So Kaitlin, the therapist, and I began kind of thinking about how we might address some of these challenges. And we thought what if we could create an immersive environment to simulate a physical presence in the real-world community settings, and allow students to repeatedly practice some of these targeted skills within a safe, controllable environment.
And what if they could have repeated exposure to real world places and situations before going out into the community without real world consequences. So that's when, which led us to consider virtual reality as a platform. And with VR, students can practice far more often and with less stress, gradually building towards readiness to go out into the world and try their new skills. So after, after we did the research, I submitted an institutional review board proposal form, and we ended up doing a usability study at the Manhattan Children's Center. It's a school that provides education to children with autism spectrum disorder. And throughout the development and design process, I had access to and was able to collaborate with a team of five occupational therapists throughout the design and development. And so the OTs were involved in every step of the process.
And what we ended up with was... We tried to figure out which community setting would be best to build out for our initial prototype. A cafe, or a grocery store. And we ended up deciding on a grocery store, just because the shopping experience requires students to use so many different skills. And here you can see a chart of all the executive functioning skills which can be applied to the experience of a shopping trip. And our next step was to figure out what these instructional scenarios would look like. So over and over again, throughout the surveys and interviews, we kept hearing how important it was for virtual experiences used during therapy to mimic as closely as possible what students would hear and see in real world settings. And while computer-generated simulations allow for a wide range of user interactions, they don't always capture the visual intensity or the ambient noise or environmental sounds or chaos of the real world.
And that's exactly what a lot of students with autism struggle with so much. So we decided to try something different. As you've probably noticed, a lot of the examples that I've shown you so far were computer-generated VR. But we wanted to try using 360 degree images and videos so that students, we could actually prepare learners for the conditions that they would encounter in real life. And complete with real people and environmental sounds. So you can think of it almost as, what we're building at AcclimateVR is almost like immersive video modeling. Many of you may be familiar with the concept of video modeling. It's a well-known therapy technique that has been shown to help teach students communication, social, cognitive, and play skills. And basically we're taking that concept, and making the video immersive. And so I can show you just very quickly here what some of the video footage that we took looks like.
So you can see here that with the 360 video, the student is able to see all the way 360 degree view, and hear the sounds of the grocery store, and see the people in the grocery store to get used to that environment. So it's almost like exposure therapy. And we wanted to take it a step further, the video modeling, and unlike traditional video-based modeling, which is often used as a tool for students with special needs, we wanted to give the opportunity for students to actively participate in tasks within the 360 degree video scene. So what we did was we built in embedded assessments into each VR experience to provide opportunities for practice and to check understanding of the student's targeted skill. So I'm just gonna show you very quickly what those embedded assessments look like. So here you can see a user testing session that we had with a student with autism. And you'll notice that he is presented with a sort of check for understanding as he's watching the videos. So I'm just gonna turn the volume up here.
- [Narrator] Look down to see your cart. Check your list. Do you need anything in the bread section?
- [Woman] Great job getting the wheat bread. It's crossed off your list, and in your cart.
- [Cashier] Great, you have a great day.
- [Narrator] Now, it's your turn to pay.
- [Jaclyn] Okay, so you can see from that example where the student was working on finding items on the grocery list, that he was presented with positive feedback when he made the correct choice. And the bread was suddenly in the cart. So that was a check for understanding to make sure that he was understanding the skills that he's being taught. Another feature that we wanted to include, based on surveys with therapists was the ability for teachers or therapists to see what the student is seeing. So when a student participates in a VR experience, the therapist or teacher has the ability to observe the student's experience mirrored on a separate device's screen. And this feature allows educators to track whether students are meeting learning objectives for targeted skills, and provide feedback to students in real time. So I'm just gonna show you one more usability testing, and we'll quickly look at an example of AR, and then we'll get to questions, I know there have been some questions coming in. So in this case, this is an actual student user test. And you can hear, you can actually hear the student's responses. So listen carefully, and you'll hear the student actually interacting with the cashier. Which indicated to us that he was definitely experiencing a sense of presence or immersion. And this is another example of one of the embedded assessments that a student would see within the 360 degree video scene. This skill was working on paying at the cashier.
- [Cashier] Thank you very much, you have a great day.
- [Narrator] Now it's your turn to pay.
- Hi, How are you today?
- [Student] I'm doing okay! Yeah, I have one friend.
- [Student] Apples.
- All right, your total is $18.36 please.
- [Student] Okay. Bye.
- [Narrator] Your money from your wallet is on the counter. How much will you give the cashier? $1 is not enough. Try again. $5 is not enough. You need to give the cashier more money.
- [Student] 19.36, $20.
- All right, out of $20, thank you. Okay, 64 cents is your change. And here is your receipt. Thank you very much. Have a good day now.
- [Student] I did it!
- [Jaclyn] So you'll see the student's very excited at the end after completing the task successfully. So we're running at that sharing time, so I'm just gonna skip ahead to the quick example of some AR. But just to give you an idea of what we're working on for the future before we do that, we're looking at actually being able to differentiate the VR experiences by controlling some of the sensory inputs. So for example, some students might see just a still photo with no noise, or a list with pictures or without pictures, you receive more prompting or less prompting. So that's something that's on the roadmap for the future for us. And we're currently doing some pilot studies. And if you're interested in learning more about what we're learning from those pilot studies, you can check out our website on www.acclimatevr.com. And as I mentioned, we're running a bit short on time, so I'm just gonna skip over to AR. So we talked a little bit about the difference between AR and VR. And I wanted to just look at a couple practical applications of VR in the assistive technology space. So this is a really interesting project, called the Color Black Moment. And you'll see it's used to increase emotional responsiveness in children. So I'll just show you really quickly, I won't show you the whole thing. But just to give you an idea. It's a virtual reality cube that you place your camera on the device.
- Hi my name's Kevin Child, CEO for Color Black. And I'm gonna show you my software, Moment. It works off this thing right here, called a Merge Cube. You can get it on Amazon or Walmart for only $15. It comes with little mini kid games in augmented reality. And my wife and I created the first professional app for it. Me being a software developer, my wife being a school psychologist. I'm gonna launch it right here on my iPad, but it works on any ILS and active link device. There's three tabs on here, and the first one is emotion. This is for autism and mental health. And all you do is just put the cube right in front of it, and now you have all the emotions in the palm of your hand. And so now it becomes a straight talking piece and you can engage with each emotion. So you can go right here and you can talk to sad, and you can figure out, hey, why is he sad? And as you see from the side, I'm using the unique movement of the cube itself. And while we're working on sad, if you want to divulge even further, you can click on it, and it's six sub-emotions of that main emotion. So you have more, you're sad because you're in front of a lost dog sign. You're sad, is he being bullied or talked about? You're sad, maybe it's self-image. He's looking at himself and doesn't feel comfortable and he's sad about it. All of these are straight talking pieces.
- [Jaclyn] So that's a really interesting project to check out, if you'd like to learn more, you can check out their website. And the last example I'll give, this is actually an article that we came across by Craig Smith. And he was talking about the potential for what's called the ARKit. And if you're not familiar with the ARKit, it's basically, it's basically a kit for developers to use that they can create augmented reality experiences. And so what Craig Smith did in this article, was he basically, and booted the link, I'll have to share that link with you. He talked about different possibilities for using the ARKit for students with autism. So looking at things like being able to create visual schedules overlaid on a real-world scene like this. Like if a student was preparing a meal, and had to do the different steps of the meal.
You would be able to see that overlay onto the real world. Emotional regulation, being able to identify a feeling in real life during a, and then being presented with opportunities or suggestions for how to handle that feeling. And also, social modeling that's actually out in the real world. So if a student was out on the playground, and you could have a video overlaid on top of the swing, where they could learn to share and take turns. And this is another example. Just skip down to this one here. With visual focus. So, being able to demonstrate social skills or emotional regulation skills. And use visual timetables to help organize the day. So visual schedules overlaid on top of what you're seeing in the real world. And this was just an idea that we had for a bravery game. And I think that would be, sensory customization, this one is also very interesting. So being able to, to show, to adjust the color of the ceiling light. And have customized texture in front of your eyes, and have each of these different sensory elements be customized.
So we're running short on time. If you're interested in learning a little bit more about it, I'm just gonna kind of scroll down through some of these pictures, so you can keep seeing some of them. Some of these ideas for how augmented reality could be used with students with autism, definitely take a look at this article. It's definitely interesting. And I believe we are about out of time, so I will open up, I know we've had some questions come in on the chat box here. Just let me scroll up here. If you have any recent questions, I know some people asked throughout the presentation. But if you have any questions, feel free to type them now. And if I don't get to your question, feel free to contact me after the session, and I'd happily answer over email as well. So let's see. It looks like we had some questions about... Okay great, so we had another example of a first-person perspective of a child with autism, thank you for sharing that. All right, so question just came in. Let's see.
- [Todd] Jaclyn, there was a question about are you aware of any protocol that helps determine which students are good candidates for VR?
- [Jaclyn] Oh thank you, I missed that one, perfect. So we, during a lot of our, it's a great question, during a lot of our pilot studies and usability testing as we're working with schools, we typically will leave that up to the therapist. Usually the occupational therapist or the teachers will know best the needs of their students. We haven't actually developed like a specific protocol, but we have found that students who have really extreme sensory processing issues are sometimes not the best candidates, because the, like, there was one student that we worked with, one who did put the headset on, it kind of bothered him having that on his face. And that's actually only happened one or two times. And for the most part, we haven't had any negatives sensory issues. But again, usually we just typically will leave it up to the discretion of the therapist or the teacher working with the students. That's a great question. Any other questions coming in before we leave?
- [Todd] There's some new comments towards the bottom about double view, and whether or not it's easily overcome by most students? There's some questions about accessibility standards.
- [Jaclyn] I don't know why I'm not seeing those questions come in. Okay, so it was about. Let me see. Yeah, I'm still just seeing the Merge is the last part I'm seeing. So it was about, can you repeat the question one more time, Todd?
- [Todd] Yeah, there was one question about is the double view, quote unquote, double view easily overcome by most students? And then there was also a question on, do you have any insight into the upcoming accessibility standards for individuals with other disabilities related to VR and AR?
- [Jaclyn] Okay, so for the double view, I don't, we haven't experienced any issues with that during our usability testing. So I would say yes, the answer to that would be yes, it is easily overcome. Interestingly we, the students that we've actually, we've done a lot of usability testing. And with most of the students we've worked with, like I said, they haven't really experienced any dizziness or double view, or any sort of vertigo or anything like that. In fact, it's kind of the opposite. Most of the time we have to, to beg them to take the headset off. They are really, they tend to be very, very, very engaged when they have the headsets on. And very sort of entranced by the content. So we haven't really experienced, as I mentioned, there was the one student who had some issues wearing the headset. But other than that, we haven't really had any issues with double vision or vertigo or dizziness or anything. And, let's see. I'm so sorry, Todd, I'm still not seeing that one other question.
- [Todd] Yeah, that's okay. You were asking about accessibility, and I feel like honestly, I don't want to presume, but my guess is super early for accessibility standards to be out there. I haven't read, I've done some research. I haven't, do you know anything? Have you seen, I'm not even sure where that would be as a resource.
- [Jaclyn] I don't, I mean I've seen a lot of resources being posted in terms of best protocols, you know, for developers and things like that, in terms of developing apps to meet standards. But no, I think it might be too early for that still.
- [Todd] A bunch of thank yous in here for the presentation and the links. Some other people have shared some great stuff here. So if you haven't looked in the chat box, hopefully nobody else is having issues with the chat box. We will probably post a bunch of these links that other people have added for visual impairments and such. Thank you so much for sharing that everyone. Somebody's suggesting that the FCC may have some stuff.
- [Jaclyn] Oh, my chat box just, sorry to interrupt. My chat box just got caught up for some reason. So yeah, I see somebody just asked where they would find the developer protocols. I don't have that offhand, but I do have it saved somewhere. So I can see if I can find that link to share after the presentation. And thank you so much, I'm--
- [Todd] Are you--
- [Jaclyn] Sorry, go ahead Todd.
- [Todd] I was just gonna ask Alice if she was asking about the AR developer protocols or VR, or probably both.
- [Jaclyn] Yeah, I would guess both. And I think I've seen some resources for both. So I can take a look and see if I can find those links. Perfect, yeah.
- [Todd] I think that's it, I think we're there. Jaclyn so much for this great presentation. I learned a lot today. As we said, we will share the other information that the folks have shared. Please, it always helps us to get survey information back from you, feedback from you folks, if you could click the link on the SurveyMonkey, we're always happy to hear your comments and criticisms and suggestions for other topics. So thank you so much everyone for joining us today. We will be posting this on the CTD website tomorrow. Jaclyn, again, thank you so much for your expertise.
- [Jaclyn] Yes, thanks so much for inviting.
- [Todd] And I think we'll let you go.