Resources

State Performance Plan and Annual Performance Report

MDE: Introduction to Data Use & Action Process

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!

You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###

00:00:01.52  >> You're listening to "A Date With Data" with your host, Amy Bitterman.

00:00:07.34  >> Hey. It's Amy, and I am so excited to be hosting "A Date With Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50  >> "A Date With Data" is brought to you by the IDEA Data Center.

00:00:24.76  >> Hello. Welcome to a new episode of "A Date With Data." I am Amy Bitterman, and today I am joined by Jessica Brady, Julie Trevino and Aaron Darling, who are from the Michigan Department of Education. They are going to be describing a data use and action plan process that they have been implementing with their districts. So let's start off by having each of you say a little bit about yourselves and your role. Jessica, would you like to go first?

00:00:52.60  >> Thank you. My name is Jessica Brady. I work in the Michigan Department of Education Office of Special Education. I work in the performance reporting unit, which is responsible for all things special education data, so the SPP/APR and 618 data, also responsible for all of the monitoring of the Individuals with Disabilities Education Act and the Michigan Administrative Rules for Special Education and then corresponding technical assistance.

00:01:22.72  >> Okay. Thanks, Jessica. That's a lot that you're handling. Julie, what about you?

00:01:29.73  >> Hi, Amy. My name is Julie Trevino. I am the SPP/APR coordinator. I've worked for the Office for 15 years now, kind of worked my way through different positions, but I worked with as a data analyst initially, and so ... but pretty much the entire 15 years all related to anything SPP/APR and significant disproportionality, working in the performance reporting unit under Jessica Brady.

00:01:59.77  >> Great. Aaron?

00:02:01.97  >> Yeah. Thanks. I'm Aaron Darling. I also work with Jessica in the performance reporting unit, and Julie. And I started off about 7 years ago supporting the Special Education Advisory Committee, so SEAC.

00:02:16.07  >> Mm-hmm.

00:02:17.03  >> And then I also assist with [Indistinct] Pro and coordinated early intervening services. And now I'm also taking on the data use and action process, so I coordinate some of that work, as well.

00:02:31.46  >> Great, and that is a great segue into the topic of our conversation. I heard through another TA provider about this really cool process that Michigan was implementing, and I spoke with Jessica, and it just ... It's so fascinating and such a wonderful process that I'm so excited to hear more about it. So if you could just start off by just describing to us what this process is, what it looks like, how it works.

00:03:01.71  >> Yeah. I'm Aaron. I can take that one. So I'm the long-winded one today, but ... so bear with me. But this is a four-phase, eight-step process, and the visual logo is in the shape of a figure eight. And so our prepare phase is the first phase, and it has our first two steps. The first step is to organize our data, so we have folks inventory all the data sources that are at their disposal. We take time to orient to the file structure, which is mostly Excel for us. We walk them through the different data definitions, the purpose of each data source, when it's collected, how timely it is. Then we spend a little time on data visualization with charts and graphs, as well. Then step two is, assess data quality. And so we know that not all data are created equal, so we all subconsciously judge data quality, but this in our process is an intentional step intended to identify quality issues so that we make sure we use quality data as a foundation to drive our decision making moving forward in the process. Then we get to move on to the inquire phase, so step three is conduct data analysis. And we teach five really simple but powerful methods of data analysis that should be conducive to almost any learning level. So we have trends to segregation, gap analysis, comparison and outlier identification. And those five typically get us everything we need in order to move through the process. Then we move on to step four, which is determine actionable causes. And so in this step, we focus on finding our root causes, and we want to be sure that schools are able to influence and control those. In education, there sometimes lies things outside of our control like zip code or socioeconomic status, but we try to direct folks to focus on identifying challenges that are rooted in the data and are truly actionable, meaning that if we can address them properly, student results should get better. Then we finally move to our planning phase, and step five is develop people smart results. So many folks in our field know what it means to write a smart goal, but not everyone takes time to focus on being people smart. And we know as humans we're all kind of resistant to change. Plus, most of us have seen many initiatives come and go over time, and folks often say, "This, too, shall pass." But we do not want data use to be one of those things. So we devote time to engaging with and bringing people along who have boots on the ground that will make a change to improve student results. Then, after we consider our people, we move to step six, so we identify strategic activities and plan. In this step, we take time to consider the context and culture of each school, what it means to be truly evidence based and what interventions are available that fit the specific need identified by the data and then finally how that intervention will be implemented with integrity and fidelity over time. So we intentionally plan action steps for 30, 60 and 90 days including who will do what by when and what the expected outcome will be. Then we move into the act phase, and that starts with step seven, to implement your plan with integrity. We know that context differs and that people often customize the intervention or practice to fit their unique needs. That's why we use the word integrity here, so as long as the core of the intervention, including things like frequency and duration, are intact, we can progress monitor the integrity of the intervention and the student results over time. Then we hit step eight to evaluate progress. So finally, as we collect new progress data, we take time to go back through the entire process as we organize our new data, assess this quality, conduct analysis and so on in order to make needed adjustments over time to ensure student success. So in a nutshell, that is the eight-step figure eight data use in action process.

00:07:18.10  >> Thank you so much. Such a robust, well thought out kind of plan, do, study act process, really interesting and need to hear about. Can you say a little bit more about the reason maybe why districts will be going through this process? Is it for significant disproportionality? Is that one of the reasons? Could it be other reasons, too? Just a little bit about how you're determining which districts are doing this.

00:07:45.89  >> Here, Amy. I'm going to take that question. So this process is intended really to build capacity to improve outcomes for students with IEPs. It can be really utilized, excuse me, at the state ISD or what we call in Michigan the member district level. It's evidence based and is intended to be sustained in each level within our system. And so we're really looking at to improve student outcomes, and it's compatible with other initiatives in Michigan, so our top-10 strategic plan and also our MiCAP. And so really what we're trying to do is look at all of the data because we know that at the state level, we have really broad data, so state performance plan data and other data that is really at that aggregate level ...

00:08:34.97  >> Mm-hmm.

00:08:35.31  >> We're trying to get districts and ISDs to look within their system at the data that they have and really make informed decisions on where root causes are ...

00:08:46.30  >> Mm-hmm.

00:08:46.90  >> ... to inform their systems in order to make actions and sustainable actions to increase student outcomes.

00:08:54.29  >> Great, and why did you decide to kind of develop and implement this process?

00:08:59.64  >> So we decided to develop and implement these processes because as we were looking at the state-level data, we saw that Michigan was doing really well in those areas of compliance.

00:09:11.84  >> Mm-hmm.

00:09:12.67  >> And we wanted to move forward and improve those results areas, so graduation, drop out, assessment, et cetera. We knew that that is more of a problem that we have with not just in special education but also looking at general education, and that's where most of our students are in with educational placement. And so working together, regular education, general education and special education, how can we work together to improve graduation, drop out ...

00:09:48.78  >> Mm-hmm.

00:09:49.35  >> ... assessment, those results indicators that are really hard to improve on a daily basis and working with our gen ed counterparts to improve that in an evidence-based way? We know that when typically we do professional development where people are setting and getting, those activities are not sustainable. This really is a process that the Office of Special Education, it takes a year ...

00:10:16.84  >> Mm-hmm.

00:10:17.20  >> ... four workshops over 2 days, and in between those workshops, the participants have homework to do where the Office of Special Education provides support during those homework sessions to gather data to make sure that these individuals have the support that they need in order to accomplish the goals and come back so that it is sustainable and that the next year, when we are not in those four workshops, the ISD member district can go and do this process again. And so it's sustainable in that we're working on improving those student-level outcomes and reaching the students within our districts.

00:11:01.74  >> Mmm. Well, it's great to hear about that general education component and how they're partners in this work, because we know that that's sometimes often a key to success, when you're kind of trying to make change in special education. It has to be a partnership across education to make that happen, so that's great to hear. I would imagine that this is a pretty big undertaking. Are there challenges that you've encountered along he way, and if so, would have been some ways that you've tackled them?

00:11:33.93  >> Well, I'll jump in on that question, Amy. Yes. [Indistinct] some challenges, definitely. I guess I'll start by saying that scheduling and setting aside ... as Jessica mentioned, setting aside four 2-day work sessions throughout the school year, that's a beast in itself, as you're probably well aware that educators rarely have dedicated time to problem solve and often get swept away by just the whirlwind of the day-to-day challenges.

00:12:05.42  >> Mm-hmm.

00:12:06.19  >> In fact, we actually use a river rafting analogy in ... when we're presenting and through the process to kind of describe how we're kind of feeling because we kind of feel like we're getting in the raft with our intermediate school district and our district partners to navigate our way through this process together. And as you can expect, sometimes the waters get rough and turbulent because we really run into various levels of understanding. As an example, some folks we work with can easily sort and filter through that ... those Excel files like in the early stages that Aaron described about analyzing the data ...

00:12:44.54  >> Mm-hmm.

00:12:44.77  >> ... and sifting through it. And others have no idea what even filtering is.

00:12:50.03  >> Mm-hmm.

00:12:50.57  >> So you have those different, varied levels of skills, and we help them with that because that's okay. We kind of understand that everybody is coming to the table with different skill sets. And then the other thing is that other times, the water really seems to get rough as districts question and resist state-level data.

00:13:09.07  >> Mm-hmm.

00:13:09.78  >> Right?

00:13:10.22  >> Mm-hmm.

00:13:10.54  >> So those are tough conversations to navigate through. But we really get through them by leaning into our common purpose of improving results for students.

00:13:21.22  >> Great, so through all of that, I'm really excited to hear, now that I think that it's been a few years that you've been working with districts and they've been implementing this, what are some of the changes that you have seen that possibly are in part due to going through this process?

00:13:38.34  >> So it's been exciting, so at the end of the year, each in-service distribution data action team that's participated gets the opportunity to report back on their progress monitoring data, and more often than not, we're actually seeing improvement in student outcomes.

00:13:54.87  >> Wow.

00:13:55.04  >> That's like a big win, right?

00:13:56.53  >> Yeah. That's the goal, right ...

00:13:58.30  >> Right.

00:13:58.40  >> ... ultimate goal.

00:13:58.97  >> And it is, and it's kind of rejuvenating. It gets us excited. We work hard all year long, and that's ... At the end of the year when we hear back, that's the most exciting time for us. And we actually had even an in-service distribution who requested that we teach them and support them in the process so that they could work with each of their districts within their ISD.

00:14:22.18  >> Mm-hmm.

00:14:22.50  >> And really, that's the kind of success we want, right?

00:14:25.01  >> Mm-hmm.

00:14:25.30  >> We wanted the capacity to ultimately improve outcomes for students. And I think much of the success has come from really setting aside dedicated time to understanding the data. We collect so much information, so much data, and sometimes we just end up admiring the data, right? I think we've all been there, right?

00:14:44.71  >> Uh-huh.

00:14:46.50  >> But we rarely take the time to really understand the complex root causes and the actions we can take to address them. And this process has built-in time that gives the focus time to look into those root causes and planning to take actions. The process, I think, also helps to build trust between the state, which is us, and ISDs and member districts that we're partnering with, and that's been really super beneficial. Approaching them, I think, from a different angle, not just monitoring and coming down on them ...

00:15:19.22  >> Mm-hmm.

00:15:19.45  >> ... it kind of feels like, probably, to them, but actually working together to help them understand and act on their data. And it's really building that psychological safety that we talk about, that partnership that we're in this together.

00:15:32.85  >> Mm-hmm.

00:15:33.06  >> We're on that rack together. But I guess another two things, too, also to mention is specifically to speak on the strategies we feel have contributed to the success of this whole process ...

00:15:44.73  >> Mm-hmm.

00:15:45.29  >> ... are the two parts that Aaron did a really nice job of describing in the process, which are not typically included in other improvement efforts ...

00:15:53.26  >> Mm-hmm.

00:15:53.51  >> ... and that's ... that, too, which is assessing the data quality ...

00:15:56.62  >> Mm-hmm.

00:15:56.88  >> ... right? Because that's the foundation for all the decisions that are going to be made. And then also being people smart in the developing those people smart results. So I think those are vital, vital in our process in the decision making and sustaining change over time. Those are the things that have been big to contributing to the success of this process.

00:16:19.59  >> Great. So what do you have coming up for this work moving forward? What are the next steps?

00:16:27.11  >> So we are planning on ... We're going to continue building capacity with our ISDs. We see that through this process, we have seen improved student outcomes.

00:16:37.81  >> Mm-hmm.

00:16:38.21  >> So it's worth the effort that the Office of Special Education has put on for this data use and action process. Our ISDs have seen the effort that they put in as worthwhile. We ... At all of our meetings with our ISDs, we have ISDs speak about this, and they see the improved outcomes. They enjoy doing this work. They know that it's hard work. But they do see the benefit. And so the Michigan Department of Education Office of Special Education will continue to build our capacity. So every year, we add more staff to our team in order to build out and add more in-service distributions to the process.

00:17:26.48  >> Mm-hmm.

00:17:26.77  >> So as we're adding more ISDs, we're also supporting the ISDs that we had the previous year.

00:17:32.00  >> Mm-hmm.

00:17:32.57  >> We also now have ISDs that they are in their third year of needs intervention for determinations.

00:17:40.70  >> Mm-hmm.

00:17:41.13  >> This is a process that we tell them that they are going to be participating in because we've seen such growth in ISDs and member districts ...

00:17:51.02  >> Mm-hmm.

00:17:51.27  >> ... and improving student outcomes, so this is an activity that we are having them participate in for the year as a result of being in needs intervention for the third year. We strongly believe in this process, and so with a dedication of staff, we and ... putting in ISDs who are volunteering on a basis that we cannot sustain at this point.

00:18:15.93  >> Mm-hmm.

00:18:16.14  >> And we're saying, "We'll have to get you on a following year" ...

00:18:18.88  >> Mm-hmm.

00:18:19.71  >> ... because how many ISDs really want to join us in this process because of all of the data that we've shown them throughout the 3 years that we have done this? Now, I will say that this has to be done in person.

00:18:34.20  >> Mm-hmm.

00:18:35.23  >> We have done this, and we have ... to do this virtually. It did not work because the collaboration that needs to take place in person, we have also done it just at the ISD level where ISDs have not brought in member districts. It did not work in that instance, as well, so it really is a process that we've had some trials ...

00:18:58.42  >> Mm-hmm.

00:18:58.66  >> ... and then failures because during COVID, we tried that, like I said, virtually. You cannot do this process virtually. You must do it in person.

00:19:07.32  >> Mm-hmm.

00:19:08.02  >> It really is a process where you have to be fully engaged.

00:19:12.27  >> Yeah.

00:19:12.41  >> And we know sometimes when you're virtual, you're not fully engaged. You need to be with the people, hands-on, with the data, wrestling with it and getting your hands dirty. when we did it with just the intermediate school districts, it didn't work because they didn't have enough data to go down deep enough into that student level. So you need where your students really are. And so those are some heads up to, if people are ... want to do this process, you need to have deep enough data in order to make the change because you are picking evidence-based practices to implement with students.

00:19:52.43  >> That's so exciting, that you have more districts that want to participate than you can even really handle right now. So that really shows the level of interest and how it's been demonstrated to really be a successful process. And I'm sure others that might be listening to this episode would love to kind of get more information for ... dig in a little more to see how all this works. Do you have on your website more about the process?

00:20:19.29  >> So, yes, we have information on Michigan Virtual University. We have a little course.

00:20:25.08  >> Mmm.

00:20:25.76  >> And then we do have information on catamaran about a little information about the data use and action process.

00:20:34.06  >> Okay, great. We can put links to those in the description of the episode so folks can check that out. Well, thank you all so much for sharing about this wonderful process. It's so thrilling to hear about something that seems to be working and that districts are interested in doing. So many times, you hear about things that, especially in situations where the state is kind of requiring them to do it, it's their resistance. But the fact that they really want this and it seems to be working is really cool. So thank you all for being on the podcast.

00:21:11.16  >> To access podcast resources, submit questions related to today's episode or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content. Or connect with us via the podcast page on the IDC website at IDEAdata.org.