KinderTEK: Research and Evidence-based Technology for Early Math Instruction

In this webinar, Dr. Strand Cary presents KinderTEK - a technology-based intervention system that provides engaging and individualized instruction to support all students struggling with math concepts. KinderTEK is a complete Math support system with embedded reporting tools and student management, individualized instruction for exploration and sequencing modes. Participants will have full access to KinderTEK app as an independent beta test or study participant. (Get the slides.)


- [Woman] Good afternoon, everyone. We're gonna be getting started. We're pleased to welcome Dr. Strand Cary, KinderTEK project director and research associate at the Center on Teaching and Learning from the University of Oregon. We wanna hear from you, so at the end of the webinar please fill out our survey and you can receive a certificate of participation. I'm gonna pass this to Dr. Cary.

- [Strand] Thank you so much. Excellent. Thank you to CTD, first of all, including Jackie, Anna Maria, and Noah for helping me get set up today and taking care of the logistics for the call and certainly for inviting me to present our research to you. Second, thanks to everybody who logged in. It's going to be great to be able to share with you what we're working on, what we have worked on, during this exciting time for the KinderTEK project. As you can see, we're scaling up efforts that have been going on for a long time. So we're going to be talking about that today. I'll also be providing an overview of the app itself. It's an instructional math app and it's kind of a systematic approach.

My colleagues and I have developed this over a series of years, and so then I'll also talk about that process of developing it. At the end I'll be sharing information with you about how you or your students can try KinderTEK on your own. As we're talking today, if you have questions that are impacting your understanding of what I'm saying, please put that in the chat for sure. Otherwise, I'd ask you to hold your questions till the end and then we can cycle back to things that might be of interest to others as well. And of course, if you're having technical difficulties put that in the chat as well, and Anna Maria'll probably help get that sorted out.

Okay, so first of all, why did we come at this KinderTEK project? Well, first of all, it's essentially that society needs citizens with a strong understanding of math. That's not just, though, that we have engineers and doctors and computer scientists, but so that the general population can navigate and fully participate in everyday life. For example, we want people to be able to consider and question statistics they hear on the news or they read in a tweet on their phone. We wanna help them figure out what's the best deal at the grocery store when money's a little tight or whether to pay off the credit card this month or not. And to do that, we need to keep all kids thinking about math throughout their education careers, and we wanna keep all kids in the STEM pipeline. STEM stands for science, technology, engineering and math. If you haven't heard that acronym before, now you know it and you'll hear it a lot more as you listen to the news and hear educators speak.

Then the pipeline essentially extends all the way from when kids first enter school through graduation, and then we hope beyond. We need to keep students engaged, competent and confident so that they stay in school, they keep learning, and they're equipped to be engaged citizens. Unfortunately, many students are not gaining the skills, the fluency, and the conceptual understanding needed to be successful, and they're essentially leaking out of the pipeline. And even more concern to me is that we're failing to get students into the pipeline in the first place. Back on that last slide I had the pipeline starting way back at pre-K and kindergarten. But often we don't start thinking about helping kids with math or science until later in elementary. So I argue that way back in pre-kindergarten and kindergarten is when we need to start thinking about this. That's when students get their first impressions of STEM topics and careers and when they first judge how interesting those things are. That's when they quickly discover that they know about school stuff or that they don't. In other words, whether they have the basic background knowledge that will enable them to learn alongside their peers. And that's also when students start to fall behind.

So educators need tools and information that can help them help all students master important foundational math skills from day one of school. And as we're doing that, as we're thinking about all students, we really need to be thinking about students' varying math experiences before they ever get to school, students with varied understanding and fluency with math, different attentional needs, different learning styles, and a variety of different disabilities, whether cognitive or physical. One tool doesn't have to magically do all these things, but we wanna make sure that in the toolbox teachers have they have the tools they need to serve all these students' needs. And if one tool can do a lot of it, that's fantastic. So what happens if we don't catch kids early? What we know is that gaps in understanding and performance emerge early and they persist. For example, 70% of students who enter and exit kindergarten below the 10th percentile remain below the 10th percentile in fifth grade. So if we're not getting them out of their kindergarten status, even though they've been in school five years. We need to narrow and ideally close those gaps right away.

Another way of thinking about this, and other studies have shown that only 30% of students who enter below the 10th percentile but exit kindergarten above the 10th percentile are still in the lowest 10th percentile in fifth grade. It's a lot of percentile languages, it's hard to follow, but essentially if we help students early on they continue to show effects of that intervention. So intervening early pays off. And we can think about it the same way we do with reading. Now reading is such an emphasis early on because the goal is to prevent reading difficulties before they fully develop, and that's what we need to do for mathematics as well, prevent the math difficulties before they develop and they impact students' later understanding. So KinderTEK is our attempt at making a solution for some of this. It's a flexible and customizable system of iPad-based math instruction to suit a wide variety of student, parent, teacher, and school needs and learning contexts. Essentially we're creating a tool to meet students wherever they are mathematically, cognitively, and in terms of attention and engagement. So now we're gonna try to play a video. It's been working in our practice sessions here. And the sound will come through the computer or the webinar, so turn on the speakers for your computer if you can. And if you can't, you can always watch the video later, either through our website our through YouTube. So I'm gonna go ahead and activate that now. It's about two minutes long.

- [Girl] We only need...

- [Narrator] Many students struggle with math because they don't have a deep understanding of early math concepts. KinderTEK can help. Here at the Center on Teaching and Learning at the University of Oregon, we use research-based principles of instruction to teach critical math concepts in an engaging iPad environment. Together with teachers and parents, KinderTEK helps all students learn the early math they need to be successful.

- One, two, three, four, five, six, seven, eight.

- [Narrator] KinderTEK truly is a flexible and powerful tool to use in the classroom. Visit the website to learn about the program's multiple instructional modes, student-specific settings, and data reports. During their KinderTEK safari, students learn about numbers, number models, and key problem-solving strategies. They get as much or as little help as they need from the KinderTEK guide each step of the way.

- [Strand] So now when I can see my slides again we'll continue going forward. Again, that video is available on YouTube if you search KinderTEK in YouTube or if you go to our website which is and Okay, so now you've seen the media overview of KTEK. Let me show you how it looks on paper. Essentially the instructional system works with a iPad app, that's the blue set of boxes there, that has both a student area and a teacher area. Students use the app individually. They have their separate log-ons and they encounter whole-number instruction, formative assessment, practice and review, and summative assessment. And as they work there are also different reward options available to them. There are all sorts of customization options for teachers to set differences with rewards, with pacing, with self monitoring. And then there are four modes of instruction.

Originally we started with a sequence mode, and that meant that you go from A to Z and you master each activity in turn. But then we've added the directed mode which allows teachers to select certain categories of Common Core standards that they would like students to work on, as well as an exploration mode which gives students full control over what they work on and when they exit any given activity. That's why we don't necessarily consider that the best use for most instructional situations, but it does have its purposes, for example, in a center, or if a student is helped by an aid and they wanna go specifically to a very particular lesson. That's a great way to go. And then we also have a screen mode which is a one-time use mode just to get a sense of where kids stand with their KinderTek content.

The teacher area has all of the tools a teacher needs to customize the experience for the group of students or each individual student, has report viewing and export options. Again, everything is aligned with the Common Core standards so you can look at things by domains and by standard. There's a note-taking system there, which is a good place for teachers to communicate with each other if they're using one type of KinderTEK or to just keep track of things themselves. And, of course, the class management option. That blue set of boxes is what we considered our Basic implementation model. And I will talk about that more in a minute. But we also have a Pro implementation model, which essentially connects that app to the web and to an online server data recording system and management system for teachers to use. That has much of the same information but in a more efficient and manageable way because then you have bigger screens, you can build more power into it when it's on the web, and we can also allow a lot more features on the web than we can support in the app.

There is also going to be a link to the CTL DBL data system, and that is something we're working on now so that if your school or district is using DBL you will be able to handle some raw string functions and also see some limited data reports showing up in DBL, so you don't necessarily even have to go to the KinderTEK site. The basic version will be available on the App store. The pro version will also be available there, but you'll need to pay a license fee to get the server supports. We also have implementation resources that we are developing pretty intensely this year and coming years to make sure that educators and parents know exactly what KinderTEK can do, why they might wanna use it, how they could use it, and how best to support their implementation and learn from its data. Here I can talk a little bit more about those two different implementation models, the Basic and the Pro. Our aim was to build a system to support all users and contexts, even when wireless is not so great in a lot of schools. And we've run into that a lot with our work, even in schools that promise great wireless.

If often inhibited what we were trying to do. So for purposes of equity and making sure we're able to serve all the students, we made it a priority for all students to get the same quality instructional experience and for teachers to have the same ability to support student learning, even if they didn't have good wireless. So having the Basic and the Pro versions allowed us to do this, because the app instructional experience and reporting is the same and the web allows just more flexibility and efficiency. The differences essentially are related to how many users are supported. The Basic app will support one teacher and 30 students; whether that data is accessible somewhere besides that single iPad device and whether it's backed up and managed elsewhere; and, importantly, whether multiple educators, administrators and aids can access the reports, manage students, and easily run KTEK classrooms.

A lot of you might know that in elementary classrooms there often is grouping and students sometimes get pulled away to do interventions, sometimes teachers combine across classrooms for all or part of math instruction, and having that Pro capability would allow that to happen really flexibly. Otherwise, a particular student's data is on one iPad. So we all know that there are a lot of math apps out there, I bet. You've probably stumbled across them or heard about them out in the wide world. So what differentiates KTEK from other programs and apps? The main thing is that KTEK is grounded in a conceptual framework that takes into consideration a lot of different things. First of all, the research literature is backing up everything that we're doing. We're building on CTL's strong research projects along the way, and we're pulling on what other educational researchers and developmental psychologists and educational psychologists have done with students working in classrooms, especially the struggling students, those who are at risk for math learning disabilities or have already been diagnosed with math learning disabilities. We are trying at all times to address specific student needs and specific contexts. How are different educators and parents using KTEK and KTEK data?

We wanna make sure we're thinking about it and are staying one step ahead of that. We also prioritize ease of access and implementation by all the particular users. And we really want teachers to have an active role in implementing KTEK and in extending KTEK's reach. And we'll talk about that a little bit more in a minute. But essentially it's an individualized program, but the teacher still has an important role to play if she's able to. This is the conceptual framework I'm talking about and thinking about right now, and essentially we have those blue student app boxes on the left that consist of systematic and explicit instruction with critical math content and validated instructional design and delivery behind it. And then motivating instruction-focused technology with individualized delivery features. As a student sits down with that they're going to hopefully receive scaffolded targeted math instruction and practice and stay engaged as they work, and those are going to kinda feed back and forth on each other, and that's gonna help them make progress through KTEK and master KTEK content and lessons.

In turn, we expect that to show up on conceptual understanding and procedural fluency measures. And eventually on larger math achievement measures, like the standardizes tests or end of year grades, or how prepared they are for the next grade of school. We know there's a lot going on here besides what's shown in these boxes. Implementation is critical and key to all of this, so you can have lots of different failure points for implementation. We're trying to anticipate a lot of those and just make it very easy to do. We also are always thinking about what are the students characteristics that might affect how well this process works? What are the teacher characteristics? And what are the contextual characteristics? Jackie is asking how students' progress is tracked. And everything is tracked at the button-click level in our server system and in the app itself. The reports and such are done at the student level and then are can be grouped on the website.

So if you have Basic app accessibility then you're gonna see everything at student level. If you have the Pro version then you're going to be able to group by groups or classes or teachers or schools or even districts. So again, thinking about that first piece of our conceptual framework, the content is absolutely critical. And in kindergarten that comes down to whole-number concepts, specifically counting in cardinality, operations and algebraic thinking, and number and operations in base 10. This is familiar if you're involved in kindergarten instruction because these are coming straight out of the Common Core. Our CTL math programs for the youngest students have always addressed this critical content because we've always gone to the research base for what is most necessary for a strong conceptual foundation in mathematics, and these are the things that consistently pop up. But the Common Core was released just as we were really delving into KTEK content development and thinking about how best to convey student knowledge and progress to teachers. So we were able to fully align our work with those standards, and our partner teachers have really appreciated that our lessons and reports are mapped onto those standards.

Another piece that you saw in that conceptual framework was focused, systematic instruction. First of all, KTEK systematically and explicitly demonstrates and teaches concepts and strategies. It's not a drill and kill. It's not explore and stumble across content. We have sequenced content, we include demonstrations so children know exactly what they're supposed to do and what they can do, we do scaffolding that fades over time as the student becomes more successful, and we include academic feedback. So rather than having on-screen explosions and big, fancy things that catch students' attention whether they're right or wrong, we let them know when they're right, we let them know when they're wrong, but we also let them know why and we give them a chance to correct that. And I should say that we try to make the math salient rather than making the sidebar information salient.

So when students are successful they get more fun things on screen than when they're not being successful. We use a variety of examples and problem types. We use clear math models, for example 10 frames and place value charts. And there are embedded formative assessments throughout the entire experience to facilitate differentiated instruction. Again, we want every student to be working on the content and in the way that is helpful to them. So to give you a sense how this looks, KTEK instructional sequences within activities scaffolds student learning, regardless of which mode you're in. So the activities are pretty similar across all the different modes like sequenced or exploration or directed. Students are using the app individually. So once they log on they see a demonstration so that they know what they're being asked to do. And then they take a pretest.

If they do really well on that pretest they're going to go on to the next section, they won't even work in this activity anymore. If they do horribly on the pretest, the pretest is probably gonna end early and they'll go straight into the next phase. And if they're somewhere in the middle the pretest will last to its full extent to really figure out where they need the most work. Students who enter the activity will again see a demonstration and maybe two or three demonstrations, depending on the type of activity it is. And then they'll enter into a lead phase. There they'll collaborate with an on-screen guide, you'll see a picture in a moment, to learn new skills and to solve problems. The guide will provide prompts and informative feedback every step of the way. The length of the lead phase really vary depending on student performance. And we wanna make sure that students are not getting bogged down too much.

So we give them chances to do a variety of activities each session, but the more they need to work on each skill, the longer they're going to spend in that phase throughout their experience with KTEK. Examples are chosen because they embody a concept and or dispel misconceptions. So we wanna make sure that we're not just randomly choosing examples, but we're choosing ones that have the maximal power to instruct. And then the next phase, once the KTEK system decides the students have a good grasp of the material they're gonna be taken into a test phase. This is done fairly independently, so the guide isn't working with them as much. And again here, the test length varies just like it does in pretest depending on their performance. There are questions here about whether accessibility features KTEK includes. This is something that we don't have a lot of, mostly because KTEK requires no reading, so there really isn't that much on-screen text. That was designed, again, for students who are at risk for math learning disabilities, and initially we weren't really factoring in other types of disabilities.

And now as iPads have gotten more and more accessible and have more of those features built-in, and as we're attempting to really spend our development time trying to make these features better for students with a variety of disabilities, and we're constantly thinking about this more and trying to figure out where, if anywhere, we can make it more accessible. There are characteristics of the instruction itself that we believe are going to help a lot of students with disabilities, but there's not a lot students can do to change how the instruction moves forward, if that makes sense. And that was all to answer a question posed by Terry, sorry about that.

Oh I should say one more thing, is that we do have a KinderTEK panel and we're trying to get more and more feedback from people about what they've found successful with their students who are using online apps, especially if they want to try out KTEK and see which things work and which things don't, and if they have ideas for things we could do. So if you're interested in providing information to us about that or would like to get involved there'll be a chance at the end to let us know that. Okay. Let's see, so the other way that we really think about scaffolding student learning and sequencing instruction is across activities. So within activities we can use it but also across activities. Students progress from concrete to representational to abstract problem types and syncing. So in this example on your screen, this top one is a footprint counting activity where they compare two sets of footprints. The footprints represent the concrete piece. Then the representational, we've now moved that number representation into 10 frames. Again, they could count the dots, but the 10 frame is capturing it in a more representational way, and then finally as they master that type of activity they're gonna be taken on to just looking at the numbers on the leaves on the trees and deciding which one's less or more.

Another important piece of KinderTEK is the math models. We really have chosen those carefully to exemplify concepts and mirror the manipulatives used in schools. So we know that teachers use a lot of different things in a very hands-on way with students and we wanted to try to take the best of those and put them in a virtual environment. We also wanted to make sure that any manipulative we mimicked here would be useful across a lot of activities. We didn't wanna have things just pop up for one lesson and then disappear again. So we have the hundreds chart, the number line, we use base 10 blocks, both on their own and in place value charts, the 10 frame, and a number of others. And then this is our guide that we talked about. She's in various outfits throughout the app. And then different story problems and things will ask students to interact with different types of objects as they're working. I won't go through this flow chart, but I just wanna show it to you to illustrate how performance on embedded assessment items moment by moment and over time dictate the instruction students receive.

So up here we have the pretest for the first activity and down here a pretest for the next activity, and you can see those are all different ways that students can get there. It just shows how quickly two students working next to each other, entering the content at the same place can end up having differentiated instruction very early on. What I've talked about so far gives you an overview of what's driving the content and the lesson flow from an instructional design standpoint, but now let's briefly consider it from a technology standpoint. iPads, believe it or not they're still inherently motivating to kids and exciting to kids. Any of us who get a new iPhone once in a while probably feel the same way. But they have the advantages of being intuitive for children and adults. They facilitate direct interaction, so no mouse or keyboard is needed, which cuts down on a lot of the learning translations that very young children have.

Now, it also introduces some difficulties for students with disabilities who can't do some of the manipulations on the iPad screen, and that's something, again, we're trying to think about and accommodate. But for our initial target audience of students who might be at risk for math-learning disabilities without other disabilities, this is a real bonus. They definitely facilitate and encourage one-to-one interaction, they're portable, fairly affordable in the long run, very durable. We haven't had any break in our project so far and we've been doing this for years. Multi-purpose, easy to maintain, increasingly familiar to students, and, importantly, to teachers. So they're not quite so daunting anymore. And they're very powerful devices, especially when they're connected to the web. They are accessible and can be manipulated by, I won't, this isn't a good thing to say at all, but most people and, as was already mentioned, there are a lot of accessibility features built into the iPad that, depending on the app, you can start taking advantage of them. We're trying to do more of that. And then they're just inherently motivating.

Some other ways KinderTEK motivates is by using verbal praise from the guide and affirmation action on screen. I touched on this a little bit earlier, but we do academic feedback combined with the good jobs, you're doing great, keep going, that kind of stuff, as well as telling them how they can improve and what they did right. And then when they do things well animals jump and run off screen and do things, so it makes it an interesting experience. As they're working they get stickers awarded for persistence and mastery, and those are always playing into the formula, but teachers can also customize whether the persistence is weighting it more or whether the mastery is weighting it more. They also earn pictures and badges for printable scrapbook pages, and you can see that in the middle picture right here. And that is very motivating to the kids. You give the kids real sticker or fake stickers, they still like 'em. We also have progress indicators and on-demand progress tabs that students can view if they need to know that they're making progress in a different way than seeing the sticker pop up or whatever, they can see for themselves that the session is progressing or that they've already worked on three activities and they're gonna be working on one more and then they'll be done.

So it's a good way for them to keep tabs on their own progress and stay engaged if they're starting to think that they're getting a little off task, they can say, "Okay, "only two minutes more, I can do it." Again, those are things that can be said by the teacher as well. And then we have an activity center. That usually is set to happen at the end of a session, but teachers can set it to happen mid-session as well for those students who need it. And that includes a scrapbook and student reports, a puzzle, and a matching game, as you see here. And we've started with the scrapbook and then added the other two features, and students are liking each one that we're building in. Another important piece about KTEK is that it provides information to both students and teachers in real time. So for students we have a student report. We've just now introduced this. And we are still working out exactly what we wanna be showing kids, but it gives you a sense of what we're going to be doing here. They're gonna be able to see which activities they have completed, how far along in the grand scope of KTEK they are, and how long they've been working and how many rewards and things like that.

We also have a teacher report view. This gives you a sense of what the teacher management part of the app looks like. This is the list of all the students in this particular class. If she had more classes, there would be a way to get to those here. This is a student named Aiden, and you can see their password that they're gonna need to log on to the app. In each of the modes you can see that the progress they've made, the time they've spent, and then here is the area where the teacher would go to change detailed information about them, to change their settings and take teacher notes and things like that. If a teacher wants to know, "Okay, I wanna see "how my student's doing in sequence mode," and look at a more specific level, then they're gonna pull up the sequence mode curriculum progress map, and they're gonna see by continent which maps onto Common Core content how they student is doing, how much of that they've mastered, as well as at each lesson and standard level how they've done.

And if they even want more information they can click on these boxes on the right and delve down even further into the lesson to see how different sub groups of members have been performed on and things like that. It looks like Mary has a question about whether students can stop mid-session and whether it'll pick up where they left off next time. Yes, KinderTEK will definitely pick up where you left off last time in terms of the whole instructional sequence. But if you're doing the timed sessions, so we have sessions that can be 10, 15, or 20 minutes long, and that applies to sequence and directed mode, then once the timer ends, with some constraints, that session will end. And the reason we do that is to make sure that the data is logged and protected. We don't want data to get lost because the app is running endlessly. But the next time the student logs on to KTEK then they will pick up relatively close to where they left off. So it might not be the same problem, but it'll be that same activity or an activity adjacent to it.

If they're in exploration mode it will pick up in the middle of the activity as well, but they'll have to find the activity again. So if they were working in activity A and they exit out, they would have to choose activity A again to continue working in it. Otherwise they'll just be given the whole list of activities that they are able to work on. Another important thing to know about KTEK is that it supports teacher needs and is strengthened by teacher expertise. We really, truly believe that educational technology is a wonderful thing, but it's not a silver bullet. It's not the answer, it's part of the answer, and ultimately it's a tool. And so we need teachers to know how to use that tool and why they might wanna use it and how it can inform their own actions in the classroom. Whoops, lost my mouse. We do this by offering Basic and Pro versions, like I've already talked about, by having multiple instructional modes so that in a particular math lesson a teacher might be very interested in using one single activity from KTEK, and so that's going to be fully appropriate, whereas in another set of circumstances they might want a student to just be working through sequence mode on their own at their own pace. We can also have that customization I've mentioned to match student attention, self-monitoring, and other characteristics of students.

And some of the ways that shows up are these on-screen indicators that we have. We introduced the red light, green light, which is accompanied by a thumbs up and stop hand, to help students know when it's their turn to act because sometimes students are a little too anxious and they don't listen to everything the guide's saying, and we wanna give them a chance to really absorb that information before they move on. We also have an audio indicator that does the same thing. Students can choose to see whether they're progressing with a bar graph versus the clock. If they have a timed session they see their rewards accumulating for that session here. And then if they have the progress path option turned on they could press this blue progress path button and see where am I in the plan for the day, and they'd know exactly where they stood. We also have all those reports that I talked about and those are in-app and online, and they are exportable so they can share them, they can print them, they can integrate them with district data. That part we're still working on, close to integrating with district data. And they can use them for database decision-making.

We have a library of implementation resources to again help educators figure out whether and how to use this and help parents figure out whether and how to use this. And I just want to reiterate certain parts of this slide. So the KTEK experience in student learning and the broader in-classroom context can enhance considerably if teachers use their pedagogical expertise and knowledge of their students to implement KTEK and use its data. So I've highlighted the modes, the customization, and the database decision-making here just to bring us all in to the fact that KTEK has data. It's a tool, and we need to be thinking about how best to deploy that with any particular student and with any particular context. Jackie's asking, "So the self-monitoring features "can be customized according "to student preferences and needs?" And the answer is yes. We have a number of different things that can be toggled on and off or adjusted on a slider scale.

So a teacher might know right off the bat that a student maybe needs more rewards than the default is set to and so would set that automatically, or she might find that out over time that the students will be a little antsy as they use KTEK, but if you increase the rewards they do a lot better. Where was I? Yes. Similarly, our development of KTEK continues to be enhanced by bringing in teacher expertise and testing in real classrooms with real teachers. So all through the project there's been integrative design, development, and evaluation work and periods of more formal, summative evaluation. We started with an IES grant from the Special Education Division in the Math and Science Goal 2 which is development work grants. That was three years, and we developed over all three of them 'cause we always found something more we wanted to fix and do and expand. But we really tempted feasibility in years one and two and then did a randomized control study in year three, which was our pilot study. We presented that data elsewhere, so I'll direct you to our website to find out how that went, but the take away point is that the app definitely showed evidence of promise, which was the goal of that project, and that students who successful engaged in KinderTEK had especially good improved outcomes.

So one of the parts of that project was that pacing was a little slow, sometimes it was a little clunky, so students didn't necessarily get as far as we wanted them to. But students who were able to use KinderTEK a lot and got through a lot of the lessons ended up really outperforming their peers. So the content made a difference. We are in the middle of another grant funded by OSEP, and that was a Stepping Up Technology Grant where you take an existing technology, for example, KinderTEK, and you expand it, you make it better, you get it out into the world with a broader set of users, you really provide a lot of supports to help implementation be as good as it can be. And so we're in year three of that.

In years one through three we've really focused on development and small-scale pilot studies. So the objectives having to do with that, we wanted to enhance KTEK with individualized instructional delivery options to better serve students with or at risk for math learning disabilities and, as much as we can, to serve students with other types of disabilities. We want to develop a system of supports to enable high-quality implementation across a variety of contexts, things like using it in a general ed classroom, Title I, special education, variety of contexts, I guess, and home situations as well. And we wanted to make sure that the system could be accessed in the appropriate ways by different educators and administrators, that data could be used with district systems and with the ways teachers generally wanted to, and that it could be linked with the DBL data system just to, again, increase access and reach. Our goal was also to increasingly scale up implementation with our partner districts and scale up dissemination efforts. And that includes offering a menu of professional development tools.

Goal of all that is to help eventually users of KTEK really have high-quality implementations and make it a sustainable program. To do all that we've worked with advisory panels, we conducted user tests and focus groups. In years one and two we did many brief learning trials. And in year three, which we're now doing, those are still continuing, but we're also introducing pilot studies, kinda small-scale pilot studies where they use the app for longer. And we're also starting to introduce asking people across the country to use KinderTEK without being part of a project and just give us feedback about how it works for them, 'cause that super authentic use could also be very informative to our development, and we still have time and resources to do that. So having that feedback early would be a good thing. Next year we'll be entering the larger scale pilot study year, and that is slated to be a randomized control study really examining how those individualized delivery options are working.

And then year five it gets even bigger with scale up and dissemination, and there we'll be asking many different schools to implement KTEK however they think it will work best for them. So some might do it a whole class level for everybody, some might have it be a center, some might do it a pull-up program, some might use Pro, some might use Basic, really trying to see how it works for all those different situations, whether it works, what challenges they encounter, what supports are useful or needed, and what outcomes look like in these different situations. Let's see, it looks like a question. How many districts are currently piloting it? We have two we're working with this year, two districts working with this year, just in a handful of schools still. Part of the requirements of this OSEP project is that we work in different schools for each phase of the project, so we have to be pretty careful about making sure we have access to a lot of different schools. But we have three other districts interested in working with us in future years.

So so far we're all in Oregon, but we've worked in a summer school in a third district already, actually, as well as different preschools and things that aren't really considered part of districts. Now, speaking of, here you go. Don't know my own slides. So to give you a sense of who is involved in our studies and what tasks they include, I jotted down some of that information here. In some studies we screen students in and in other studies everybody in the class participated. But we worked in preschool sites like the YMCA and HeadStart, after-school programs, elementary schools, all different grades and combinations, working in the lab and working in homes. Traditionally, well, in IES grants it was almost always teachers implementing it, just because of the way that grant worked. But with our current grant we've had primarily KTEK staff implementing for all our brief learning trials, and then this year we're handing it off to teachers largely. So they'll be doing those larger class implementations.

Our assessments, kind of a long list, but we pick and choose from these to suit each study's needs. Something I haven't talked about is that everything KTEK does is recorded in a massive data file on the app and transmitted to our server if we have Pro mode going. And there's a wealth of information there that we've dug into a little bit for some studies and looked at a very cursory level for others, but there's a lot of detail there that we can really use to inform our development working forward. We do observations, we do assessments closely linked to the program and farther from it, we do some attention measures to help select students who might need those attentional supports. If you haven't heard of the heads, toes, knees, shoulders tasks, it's a fun one and I dare any adult to try it and do well. Basically you're supposed to do the opposite of what you're told to do, and it gets hard pretty fast. We also have teachers evaluate attention.

We do surveys and interviews to try to get a sense of perceptions and how participants are feeling about it. So there's a lot on this slide, wanted to give you a sense of the type of information we've gleaned from our first two years of this OSEP project. First is that all the changes we made moving from IES to OSEP in terms of architecture, adding rewards, improvements in pacing, customization options, all that, those are working really well and they're welcomed by participants. Having some students use KTEK allows other teachers to do small group instruction with the other students, so that's been really nice. Teachers have loved having KTEK in their classroom because they're able to do more individualized instruction themselves and more differentiated instruction themselves. Also it's great because teachers enjoy using KTEK so much they want us to come back the next year, so that's always a good thing.

Students of all ability levels enjoy using KTEK, though we do, of course, find that the higher-achieving students don't need to spend as long in the program. They whiz through it a lot faster and they might find some of the instruction a little bit slower than they want, but it wasn't aimed at them. So that's okay. The fact that they like it at all is a good thing and we have intentionally been building along the way to make it engaging at some level for those higher-achieving students, just so everything can be fair in the kindergarten classroom. KTEK helps a wide variety of students learn. Learning in KTEK translates to improved performance in external measures. And KTEK is adequately calibrated to the whole number math learning needs of a range of students within the general ed classroom. And it translates to teacher-led contexts. It kind of paths, so what I'll just say is that students who find themselves in a different path than their peers very early on so that differentiation is happening.

Students who can pretest out of earlier activities eventually do land on something they can benefit from, so those story problems at the end of things tend to be worked on by all the kids. And teachers reported improvements in students' conceptual understanding and fluency during their regular classroom instruction. So they were learning stuff in KTEK that then was helping them in their regular classroom instruction, which we found really wonderful. We are shifting away from development and into larger-scale implementation and making supports for educators. And so as we're doing that, again, large-scale studies with the educators leading the way. Doing major expansions of our implementation resources, our web-based recording, and our analyses focus on that game play data. Having people use it for longer, and again, getting feedback from more of the general public about KTEK and factoring that into our work. So how can you learn more? As we've been scaling up people are becoming more aware of the project.

We've been running under the radar intentionally in some ways 'cause we weren't necessarily scaled to support it. But I will tell you which versions of the KTEK are available in which ways. So the first thing to know is that eventually a KTEK app will exist that will function as either Basic or Pro. So you get one app and activate the Pro features if you want. That won't be for a while 'cause we need to make sure we have sustainable support infrastructure in place to support that first. If you have things running on a server you need to have people maintaining that server, checking it, making sure it's all accurate and supporting customers, things like that. So that'll be a little bit of time. But in the meantime, easiest way is to get involved in our research, a little plug for that. We tend to prioritize settings where we can observe and where we can administer pre/post assessments and provide in-person support, just because then we can actually do analyses of how KTEK is working.

But we are also increasingly asking people to beta test the products or to serve as panelists to use KTEK independently and let us know what they think. In those situations either the Basic or the Pro could be provided. And the types of things we're looking at would be, what types of implementations are being successful? What resources are useful or needed and in what format? And then how well does KTEK work with different sets of students? So English learners, different grades of students, students with particular cognitive and physical disabilities, knowing that from a larger scale than we're able to reach within the resources of our project would be really good. We do have a website which has a lot of information in it, so I encourage you to visit that. Overviews of the program, talks about underlying research, and then implementation resources to help people find the right product, research findings, and of course, ways to contact us.

And if you are antsy you can check out KTEK on your own. We do have a version on the App Store right now, but that was posted a long time ago and I would suggest that you wait, at least until next week because we have another submission that will overwrite that one going in today or yesterday. And that has all the lessons rather than just five or six lessons, which is what's up there right now. It has all the instructional modes, it has all the supports and rewards, everything I've shown you today is up there. So I really recommend you wait and get that version rather than getting a stripped-down version that's up there right now. Moving forward all that data will carry forward, but this is a big upgrade. So I'd encourage you to start clean with this one. The Basic version will always support one teacher and 30 students, again, that's to keep it manageable and to make sure that we just have a good student experience along the way. We couldn't have done any of this without our funding from the federal government.

So as I said before, the first KTEK was supported by IES, Institute for Education Sciences and the second one is supported by the Office of Special Education Programs, or OSEP. And we have many people helping us along the way. Concentric Sky here in Eugene, Oregon did all our app development and they were an excellent partner all the way through. We've worked with a lotta different school districts, care centers, the community, parents, educators serving on our panels, and then also the Center for Media and Educational Technology here at the U of O. So this is the chance where you guys can ask more questions than have been coming through the chat so far if you'd like. And this also gives you contact information if you think of questions for later on, you can feel free to email us. This is the website and fill out the contact forms there and things like that. And to just give you a preview, the next two slides have a lot of different resources and references listed. So if you'd like some more of the background research that has provided a foundation for our work, then I encourage you to check out some of those. But for now I will stop talking and let any questions come through that might pop up.

People are thinking, that's good. Is there a cost of the app? Yes, there is, believe it or not. We have a cost set as minimally as we can. The one that's gonna be released within a week or so I believe is gonna be $6.99. But again, if you have that version that will turn into the Basic and Pro version. So once you have it, if your school decides to upgrade and all that, you already have that one for that set of iPads. When we have the Pro services up and running for actual end users, so outside of the grant context, there will be probably either a yearly or semester fee or something to go along with that that would scale to the number of students that you have using it.

And again, that's because there needs to be the infrastructure and the support behind it. We are at the university, so it's non-profit. So the app funds go back into development the entire life of the grant, and then after the grant a lot of those monies return back to CTL to continue doing the development work and expand KTEK or do other types of supports for students. How do families sign up for the beta? I would encourage you to visit our website, and there's a, "Wanna become a participant?" form. You can sign up there, and that's a great way to go. You can also email either me or the KinderTEK address there and we'll get you linked in. It kinda depends on how you're thinking about using it and how that fits into what we're doing right now, but we definitely would like to have those conversations and start figuring it out.

Again, we're just stepping into that side of it, kind of doing the wider support. So something to know up front is that we wouldn't be able to do a lot of hands-on, because our focus really has to be on the planned research studies, but obviously we would like to support you as much as we can, so we'll try to do that. Other questions? Okay, well in that case, you guys feel free to contact our project or me at any time to follow up if you have questions or want to learn more about how to get the app or how to become a participant. Keep an eye on us, we will be updating our website very frequently moving forward. Up till now it's been kind of a holding point, but we will be updating that as soon as we put the update on the App Store, the website'll be updated at the same time. And that will be continually expanding, 'cause a lot of our work this year is looking at implementation resources and making those more robust and increasing our dissemination efforts. So I think I will move it to this next slide which reminds you that CTD put on this webinar. And thank you everybody for coming today. I'll stay on if people have other questions.

- [Woman] Alright, it doesn't look like we have any more questions, so thank you to everyone who joined us and we hope to see you again at future CTD events.

- [Strand] Thank you so much.

- [Woman] Thank you.