Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!

You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###
00:00:01.52  >> You're listening to "A Date with Data," with your host, Amy Bitterman.

00:00:07.34  >> Hey. It's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50  >> "A Date with Data" is brought to you by the IDEA Data center.

00:00:24.78  >> Welcome to "A Date with Data." Our guests for today's episode are from the California Department of Education. I am joined by Dr. Jack Brimhall, who is the Interim Associate Director of the Special Education Division, and also Kishaun Thorntona, who is the Education Program Consultant who is also with the Special Education Division, and we're going to be discussing California's significant disproportionality data and systems as well as how the state supports districts that have been identified with significant disproportionality. Welcome to both of you.

00:00:57.53  >> Hey, there. Glad to be here.

00:00:59.47  >> Hello, everyone. Thank you for having us.

00:01:01.65  >> Yes, so just to get things going, if you could each just say a little bit about yourselves and your role in the California Department of Education, Jack, do you want to start us off?

00:01:11.99  >> Sure. Yeah, so I'm Jack Brimhall, and right now I'm the Interim Associate Director with the Special Education Division at the California Department of Education. And one of my roles is the head of the monitoring units, and we have a variety. We have actually five different ... six different monitoring units. Two of those units deal with significant disproportionality, and so we have consultants in those units that work really hard with districts that have been identified as significantly disproportionate. We have another two districts that work with districts ... or two units, excuse me, that work with districts that are disproportionate. And I've been at the California Department of Education for the last 10 years. Before that, I was a high school principal, and before that, I was a high school English teacher, so happy to be here.

00:02:03.47  >> Great. Thank you. Happy to have you. Kishaun, what about you?

00:02:07.26  >> I am Kishaun Thorntona, and I am one of the Focused Monitoring and Technical Assistant Unit Consultants at the California Department of Education, so I'm in one of the two units that Jack described as monitoring districts that are significantly disproportionate. I've been with the department for about 19, 20 years, and worked with districts who were both disproportionate and significantly disproportionate, I would say probably for the last six years, and just excited about the work that we're doing with significant disproportionality and how the Department is monitoring the work to ensure better student outcomes.

00:02:54.06  >> Yeah, awesome. So to get things started, if you could maybe just describe for us how California identifies districts for significant disproportionality, maybe provide some context, a little bit about your methodology, how many districts are typically identified in a year, that sort of thing.

00:03:13.45  >> Well, it's very ... I don't ... We don't do the actual ... "We," meaning the two units, don't calculate, but our data unit calculates ...

00:03:23.42  >> Mm-hmm.

00:03:23.63  >> ... does all the calculations, and it's my understanding it's very ... It's a lot of work. And so they do this with all of our districts, and we have, Jack, probably, what, 1,500 districts?

00:03:35.87  >> Yeah.

00:03:36.17  >> Yeah.

00:03:37.08  >> We have over 2,000 districts in the State of California, so it's big.

00:03:41.81  >> Mm-hmm.

00:03:42.02  >> It's a big job, to crunch all that data for everybody not only so that you can see who's significantly disproportionate, but you have to go backwards and see who's disproportionate first and then track it. So it's a big, big job or our data.

00:03:56.64  >> Yeah, so they do all the risk ratios for every district in the areas of disproportionality. Then, they determine ... After determining the disproportionality for every LEA, those that are disproportionate for 3 consecutive years in the same areas are then identified as significantly disproportionate and then are determined or are placed in our monitoring system. And we have approximately ... Since 2020, we've had approximately ... Well, we started off with probably about 138 ...

00:04:34.27  >> Mm-hmm.

00:04:34.65  >> ... districts that are identified, and it's just been over 110 districts identified every year thereafter.

00:04:40.84  >> Wow.

00:04:41.77  >> Yeah, and right now, to be identified for dispro, to being disproportionate, you would need to have a risk ratio of 3.0.

00:04:51.87  >> Mm-hmm.

00:04:52.51  >> And so that essentially means you're three times more likely to either be that ... have that disability or to be suspended, those types of things. So we use 3.0, and Kishaun can talk about it more, but that's one of the areas in which gets you into significant disproportionality, but then we also have reasonable progress that can help you get out, so ...

00:05:19.45  >> Mm-hmm.

00:05:20.43  >> And our our reasonable progress, we started a couple years ago as a result of our stakeholder, then, we called it stakeholder, Sigda stakeholder group that we've had that we've had that conversation about reasonable progress. So our reasonable progress is a district under a risk ratio of 4.0.

00:05:43.06  >> Mm-hmm.

00:05:43.88  >> And so it has to be an "And." There's an annual 0.5 decrease in the risk ratio, so then they'll be considered for reasonable progress.

00:05:54.98  >> Ah. Would they have to be below 4.0 all 3 years or just the most recent year?

00:06:01.71  >> The most recent 4.0 ...

00:06:03.78  >> Got you.

00:06:04.01  >> ... but have an annual decrease of 0.5.

00:06:07.31  >> Mm-hmm.

00:06:07.58  >> And that's what it is now, but as we continue to build our monitoring system, that might change with the consideration of qualitative data. It's really ... It's a lot of work because there's a lot of districts.

00:06:23.68  >> Yeah.

00:06:24.05  >> And so we want to make sure that we're doing what we are ... we should be doing.

00:06:28.53  >> And one of the things that ... There is a lot of districts, and we spend a lot of time working with those districts because we feel it's important.

00:06:37.35  >> Mm-hmm.

00:06:37.99  >> But it's all rooted in data where our monitoring system is rooted in looking at the data, driving deep into the data to come up with some root causes ...

00:06:49.90  >> Mm-hmm.

00:06:50.26  >> ... for the underlying issue so that ultimately these districts hopefully not only just make progress, but they are no longer disproportionate. And so it's all based in data, and it's also based in implementation science. It's based in best practice. And we have folks that are experts in that area that we have worked with those school districts. So it's exciting. It's hard work but exciting work.

00:07:17.18  >> Yeah. I know. I've heard a bit from Kishaun over the last couple of years about all of the wonderful work you all are doing, and it is. It's thrilling, and I think it's just a model that other states and districts would be curious to hear about and learn from. So once you have your group of districts that you know have been identified with significant disproportionality, how do you then notify them? What data or information are you sharing with them to let them know they've been identified and kind of what those next steps are?

00:07:48.54  >> Every year, we give what we call an annual determination letter.

00:07:52.15  >> Mm-hmm.

00:07:53.16  >> And normally that's released in mid-March, and the annual determination letter is going to indicate what monitoring level the local educational agency is in, and in our state, we have three different levels. We have a universal level of monitoring, which essentially means you've met just about all the indicators, all the targets.

00:08:13.75  >> Mm-hmm.

00:08:14.24  >> Targeted monitoring means you've missed some, so, for example, districts that have been identified as disproportionate ...

00:08:20.73  >> Mm-hmm.

00:08:21.04  >> ... would be in that targeted, and then intensive are the school districts that need the most intensive help and that we provide the most intensive help to. But if you're identified as significantly disproportionate, you would be in that intensive care.

00:08:35.68  >> Mm-hmm.

00:08:36.03  >> It's important to note you can be in the intensive care without being significantly disproportionate, but anyway, that letter explains all that to the school district. It tells them what their care is.

00:08:46.56  >> Mm-hmm.

00:08:47.19  >> If they are significantly disproportionate, if they are disproportionate, it tells them all they need to know and then the next steps. And so once that letter goes out, then we start to actively monitor and work with the school district to come up with a plan. And like I said before, all of our monitoring now is rooted in data.

00:09:07.94  >> Mm-hmm.

00:09:08.35  >> So there's four steps to each plan. There's the looking at the data in step one, and not just one piece of data, by the way. We want to triangulate variety ... a variety of data, so local data, indicator data. California has a dashboard, which is really cool, that breaks down chronic absenteeism data, state performance plan indicator data, so data by The Feds, all of these different pieces of data and qualitative and quantitative data. So it might be that you do case studies and empathy interviews.

00:09:43.18  >> Mm-hmm.

00:09:43.44  >> That's that step one where all that data is looked at, and then step two is where you take that data, and now you start to develop some root causes around the data.

00:09:53.15  >> Mm-hmm.

00:09:53.67  >> And once you get those root causes, then you can start developing a plan that's based on that, and with significant disproportionality, it's difficult because once you're significantly disproportionate, now you have to dedicate 15 percent of your IDEA funds ...

00:10:12.99  >> Mm-hmm.

00:10:13.25  >> ... your special education funds to early-intervening services. And so we need to have those funds tied to activities that are going to support what's in that plan.

00:10:25.69  >> Yeah.

00:10:26.12  >> And Kishaun deals with this all the time, but the plan has to be approved before those funds can be spent, and so we want to have a good plan, but it's also a race to get a good plan so that you can start expanding these funds so that then we can reimburse so they don't lose that 15%.

00:10:43.98  >> And if you have well over 100 of those plans that need to be reviewed and approved, that's a big job, too.

00:10:51.52  >> Yeah, and Kishaun can talk to that because she herself has a number of plans, and then they overlap, right, because you might have some new plans that go 27 months. But then if you're significantly disproportionate again, that's a whole separate set, whole separate pot of money, right?

00:11:09.57  >> Mm-hmm. Mm-hmm.

00:11:10.27  >> So you might have 15 percent of your IDEA funds quarterly intervening services in one plan, but then if you're significantly disproportionate again, now you have a different pot of money. So then Kishaun, if she's your monitor, she's got to track and watch two different plans at once, one plan that's already in the implementation stage while trying to approve a new plan and those new activities toward that new money.

00:11:36.68  >> Yeah, a lot of layers of complexity, for sure.

00:11:41.30  >> And in addition to the ADL letter, after that ADL letter goes out, the annual determination letter goes out, the districts receive that. Then, our contractor and partner, State Performance Plan Technical Assistance Project with the Napa County Office of Education, starts contacting the districts to front load some information through introductory videos and then also around the mandatory webinars that we have. We front load these introductory videos. They're about 10 to 30 minutes in length. And it's for LEAs to understand the why, how and what around significant disproportionality, and the process that they'll encounter as a result of their identification, the front loading of information prior to the actual webinars present or offer information on the federal requirements, identification methodology, technical assistance facilitators and a history of the significant disproportionality work as well as the why the work is important and challenging mindsets. And so it's with these ... with this information is like the start of creating that plan that Jack was just talking about ...

00:13:00.93  >> Mm-hmm.

00:13:01.56  >> ... and really helping them create that foundational piece of knowledge so that they could begin to do all of the required work in creating their plan.

00:13:12.46  >> Great, so kind of what comes next, then, in terms of what the districts need to do and the resources and support that the state and your TA provider are providing the districts?

00:13:29.13  >> So the California Department of Education has created the compliance and implementation monitoring process.

00:13:37.83  >> Mm-hmm.

00:13:38.76  >> And Jack talked about the steps that we have in this process. I think one of the things that's really important to point out in this process is having the right people on the bus to begin with. It's so important to have the right leadership team, so that's normally like the superintendent and their cabinet, across both general and special education. A lot of this work is or all of this work is pre-referral work, and having the partnership and understanding throughout the district is really important. So having the right people on the bus is the start, and then it leads to the creating that ... the Educational Partners Team that is required to then review all the data ...

00:14:32.50  >> Mm-hmm.

00:14:33.36  >> ... that Jack referenced and all the different types of data to come to the root causes. That Ed Partners group is so important because generally they will be the ones that are discussing those root causes or coming down and narrowing down to the root causes that the district feels are contributing factors to significant disproportionality. So I think the biggest thing is having the right people on the bus. One of the other things that are ... We call our monitoring process the CIM, Compliance and Improvement Monitoring.

00:15:07.45  >> Okay.

00:15:07.63  >> Did I say that right, Jack? But that process now requires for LEAs in significant disproportionality to do case study, do a case study. So we look at policies, practices and procedures, and some districts do focus or focus groups or empathy interviews to check their practices ...

00:15:35.32  >> Mm-hmm.

00:15:35.81  >> ... as it relates to policies and procedures, but also that case study is really important and looking at the students that are identified. What is their journey in their educational history? What happened? And to really understand the students that they should be addressing. And so that piece is new in our process this year where it's required, and I'm excited for it because we've seen it done in the past, but it was optional work. But now our ... With our new process, we've made it a requirement.

00:16:11.89  >> Yeah, that's really neat because it seems like the practice pieces is what a lot of states and districts struggle with in terms of how to measure that and how to implement that review, and that's a really innovative way you're talking about being able to, it sounds like, do a really thorough job of kind of checking for those practices and sharing that they are appropriate.

00:16:38.85  >> Yeah, California has really been focused on the qualitative data.

00:16:42.38  >> Mm-hmm.

00:16:43.59  >> Getting that qualitative data in all these different ways is really part of the driving force, as well, as the quantitative data that we've always used in the past.

00:16:54.09  >> Well, you all are very ... sounds like has a data-rich process in place.

00:16:59.50  >> It's ... Not only is it data-rich. Kishaun mentioned it. It's that partner team ...

00:17:06.08  >> Mm-hmm.

00:17:06.52  >> ... and getting, in general, ahead as part of it ...

00:17:09.34  >> Yeah.

00:17:09.47  >> ... because it's a general education plan.

00:17:11.49  >> Mm-hmm.

00:17:11.70  >> These funds and these activities are spending funds towards early-intervening education. It's not special education and what special education funds are going toward general education.

00:17:25.12  >> Mm-hmm.

00:17:25.41  >> So essentially it's a general education plan, and special education students are general education students first, so we feel like it's ... That's a huge part of it, is getting everybody together, the gen ed with the special ed and then using quantitative with the qualitative. It's a process. Its not a sit-and-get. It's a process of improvement.

00:17:50.07  >> I think another piece that I left out, too, is how heavily involved parents are in this process.

00:17:54.98  >> Mm-hmm.

00:17:55.78  >> We require parent input, parent partnership, really, especially on the Educational Partners Team in determining those root causes.

00:18:06.80  >> Mm-hmm.

00:18:07.40  >> It's not districts doing it alone. Parents are involved, parents of the identified group is involved. And so we really are looking at the process in which districts took to get their plans, to create their plans. That's really important in this.

00:18:25.46  >> Yeah, that's a really great strategy that you described, especially with those teams and the composition of them, making sure to include families, and general education has to be at the table. So it sounds like you all have really done some great work in making sure that that is a piece of what the districts are doing. Are there ...

00:18:51.12  >> We're trying.

00:18:52.54  >> I know. Easier said than done, but ...

00:18:54.61  >> Yeah.

00:18:55.17  >> That is the goal.

00:18:56.42  >> It's still definitely a work in progress.

00:18:58.46  >> Mm-hmm.

00:18:58.67  >> We're constantly looking at our process and seeing what works best and ...

00:19:06.37  >> Yeah.

00:19:06.69  >> ... how we could support better, and so we're constantly looking at that, I think. But for the foundation of our system that's in place, but how we go about it, we always want to be reflective of our districts and what we're seeing and what we want to see.

00:19:27.83  >> Mm-hmm. Yeah, and to that end, what do you have coming up next for significant disproportionality in California? Are there any new resources or activities, changes you're thinking about making? What does the future look like?

00:19:46.24  >> We're always doing webinars.

00:19:48.47  >> Mm-hmm.

00:19:49.57  >> And our webinars are determined based on what our districts ... we see the need. MTSS is a huge topic.

00:20:01.03  >> Mm-hmm.

00:20:01.39  >> EL is a huge topic, discipline, so those are things that we focus on as we provide webinars throughout the year for our districts to take part in. We always look at the new articles that come out, the academic articles that come out related to student success, how to support students, how to support districts and staff, how to build out systems. A lot of districts are still in that process of creating their foundation and building on it. So we really look at what districts need, and so we constantly try to provide the webinars, provide the additional resources of reading resources. We partner with other providers that are expertise in certain areas, and so if we see a need of our ... the district needing something that we know there is support available, we offer that support. So that's constantly changing all the time.

00:21:08.80  >> Mm-hmm.

00:21:09.92  >> ... depending on what's new that comes out. I think one of the biggest things that we're doing now is really focusing on which ... the implementation piece.

00:21:19.60  >> Mm-hmm.

00:21:20.26  >> That is huge, and we've talked about that in the peer-to-peers for months now ...

00:21:25.29  >> Yeah.

00:21:25.74  >> ... really looking at implementation and focusing on implementation. How can we support that implementation piece? That's huge, I think, moving forward.

00:21:35.10  >> Yeah, that's definitely, I think, what we're hearing from a lot of states, is they sort of are at a place where they've been identifying districts. They feel like the methodology they have down and notifying the districts, but then how do you address the issue? What are some of those practices that really seem to be effective, and are districts really implementing them with fidelity, so that evaluation and monitoring piece, as well?

00:22:02.47  >> Yeah, that part is so hard because every district is so different.

00:22:05.37  >> Mm-hmm.

00:22:06.31  >> The demographics, the size, the ... and every district is so different, and so trying to see what works, what works best is really difficult.

00:22:17.70  >> Right, because what might work in one context may not ...

00:22:21.09  >> Absolutely.

00:22:22.11  >> ... in another. Well, thank you both so much. You shared so much wonderful, interesting information. I learned some things, and hopefully others that are listening have, as well. So really appreciate you coming on and telling us your story from California and how this has all been going.

00:22:42.17  >> To access podcast resources, submit questions related to today's episode or if you have ideas for future topics, we'd love to hear from you. The links are in the episode context. Or connect with us via the podcast page on the IDC website at IDEAdata.org.