[Navy blue CAI "We power the possible" logo appears on screen with white background, with www.cai.io below in black.] [Title slide. The slide title reads: "Enhancing child welfare: Best practices for measuring and improving outcomes". Above the title in smaller teal text reads: "A NACo Webinar". Near the bottom of the slide, two logos are aligned horizontally. On the left under text that reads "CAI is a sponsor of" features the National Association of Counties logo, and on the right features the navy blue CAI logo with tagline "We power the possible".] [Presentation slide 1. The National Association of Counties and CAI logos are presented at the top of the slide in white text. Above the title is the word "WELCOME" in large white text. The slide title reads: "Enhancing child welfare: best practices for measuring and improving outcomes." In the top right corner, a speaker photo of Melissa Krouse is featured, with her name displayed in white text below the photo. The background of the slide is various shades of teal, creating a cohesive design that looks like a digital sphere."] 00:00:08 - 00:00:32 Melissa Krouse Hi, good afternoon and welcome, everybody. Thank you for joining today's webinar. My name is Melissa Krouse. I'm the corporate relations manager for the National Association of Counties. We are really excited to have the CAI team here today to share an insightful webinar focusing on child welfare, specifically designed for county governments with lessons that can be applied to other behavioral health programs. 00:00:33 - 00:01:12 Melissa A couple of quick notes before we get started. We like to encourage everyone to add your county or where you're from behind your name, and you can do so by clicking the 3 dots, click rename, and then add your county or wherever you're from after your name. It's always helpful for discussions. We also ask that you please keep yourself muted throughout the presentation. And for a final note, we just like to let everybody know that the chat feature is disabled. However, if throughout the presentation you have questions, we ask that you submit them through the Q&A function, and the questions will be answered either throughout the webinar or at the end during the Q&A session. 00:01:13 - 00:01:21 Melissa And now I'd like to introduce Shannon Smith, who is the director for public sector with CAI, who will share more about today's webinar and introduce our presenters. [In the top right corner, a speaker photo of Shannon Smith appears, with her name displayed in white text below the photo.] 00:01:22 - 00:01:30 Shannon Smith Thank you so much, Melissa. So hello, everyone. Really appreciate you joining us. I think this is going to be a very interesting conversation. [Presentation slide 2. The slide title is "Meet the panelists", with smaller teal text above the title that reads: "Enhancing child welfare: best practices or measuring and improving outcomes". Below the title, three sections are lined up horizontally in the middle. The first section includes an image of Hon. Tarryl Clark with text: "Hon. Tarryl Clark, Commissioner, Stearns County, MN". The second section includes an image of William Browning with text: "William Browning, Senior Consultant, Child Welfare, CAI". The third section includes an image of Shannon Smith with text: "Shannon Smith, Director, Public Sector, CAI"." 00:01:31 - 00:01:52 Shannon So, as Melissa said, I am the director for the public sector for CAI. I'm specifically focused on local government and counties. For the last 6 years, I was the IT chief of staff for King County, Washington, which is in the Seattle area. I would like to next move to Commissioner Clark. Would you like to introduce yourself, please? [In the top right corner, a speaker photo of Hon. Tarryl Clark appears, with her name displayed in white text below the photo.] 00:01:53 - 00:02:21 Tarryl Clark Hello, everybody. Glad you're here today. I'm Tarryl Clark. I'm a commissioner in Stearns County, Minnesota. Think Central Minnesota. And for those of you who are not technology gurus, I am part of you, but I have had the honor of chairing NACo's telecommunications and technology. And today's topic is near and dear to every county in Minnesota. So look forward to the conversation. 00:02:22 - 00:02:24 Shannon Thank you. Mr. Browning. [In the top right corner, a speaker photo of William Browning appears, with his name displayed in white text below the photo.] 00:02:25 - 00:02:40 William Browning Hi, I'm Bill Browning. I'm in charge of the Child Welfare Division for CAI. And I've been in child welfare since a very, very long time, which I won't disclose. 00:02:41 - 00:02:43 Shannon Plausible deniability. I like it. [The speaker image that was previously in the top right corner, is enlarged to full screen with the video/image of the respective panelist when they are speaking.] 00:02:44 - 00:03:30 Shannon So thank you both. I think today is really going to be a conversation and dialogue around the measures that are needed for child welfare. There is a lot of different aspects we could cover in this space, but the idea of measuring outcomes for health, human, and social services, we're focusing on child welfare today, but we are aware that there are a lot of wraparound services. Child welfare is never just one person. There's a lot that needs to occur in order to support the families and caregivers for folks. 00:03:31 - 00:04:01 Shannon We're also aware that not every county has county-administered child welfare, but I think that you will find that the conversation that we have around how to measure effective outcomes will be of value no matter what space you're in. So let's start with talking about the federal child welfare reporting requirements, because that has a lot of overhead and oversight, and there are a lot of different spaces where you have to do that. 00:04:02 - 00:04:54 Shannon So, for example, there are so many different reporting requirements, and it feels like alphabet soup sometime, but some of them include AFCARS, which collects case level data on children in foster care, NCANDS, which aggravates jurisdictions for child abuse and neglect data, NYTD, which tracks services and outcomes of youth transitioning into adulthood, which means they're leveraging additional services that counties provide. And then the Family First Prevention Services Act. Family First has really had a huge impact, and we'll probably be talking about that most of all, because it requires reporting of services to prevent abuse and neglect of children, as well as preventing them from entering foster care and preventing long-term congregate care placements. 00:04:55 - 00:05:26 Shannon So when we talk about group homes, that's really what we're speaking to. So it's really important to understand those federal requirements and their impact on county operations. I would like to start with Commissioner Clark, who really has looked at what that federal reporting looks like. So, Commissioner Clark, what are the biggest challenges your county faces in meeting federal child welfare reporting requirements? 00:05:27 - 00:05:56 Tarryl And I'm guessing for many of you, you're going to really be interested in what Bill has to say. So here's the deal in Minnesota and we can get more into it. I doubt we're alone in this. We are one of the 10 states that is county-administered human services, so everything, but there's many more counties that administer human services and other behavioral programs. So that's why this is relevant. 00:05:57 - 00:06:24 Tarryl But 2 of the things for us here are, one, our state does most of the federal reporting, even though we're the ones doing the work, so they do most of the reporting. And our biggest issue, honestly, is they really don't know the work. We're trying to help them, but they don't know the work, so they're doing reviews. They don't know the details. They don't know what questions to ask what to do. So that is a pretty significant challenge. 00:06:25 - 00:07:08 Tarryl And then our county systems, the systems that the state has given us to use, is so old that I am quite sure some of the people on this call, on this webinar, weren't born yet, or you are really young. Think green screens. Think taking a half to a whole day to enter information about a child. Think taking 17 minutes each day for the system just to come up. Think if the system isn't up, you need 17 minutes just to get to it. So whether it's reporting or it's grabbing data or making decisions, it's pretty tricky when it's that old. 00:07:09 - 00:07:54 Tarryl And what I said about the state not understanding, it is reinforced by the fact they haven't put real resources until very interesting after a lot, a lot, a lot of county-driven advocacy. If we want better outcomes, we have to have a better way to do the system. If we want a great workforce, they can't spend 30% of their time just trying to deal with this system. So that, in many ways, is this has become ... It's like we're not even writing about the work versus doing it. We're trying to even get into the system so that we can write about the work. So those are 2 of the biggest challenges in meeting those child welfare reporting requirements. 00:07:55 - 00:08:04 Shannon Mr. Browning, I know you've done a lot of work in this space, particularly in Pennsylvania. Can you speak to this and also to the Family First, how it's affecting those child welfare practices? 00:08:05 - 00:08:31 William Sure. I think one of the things that we have to understand, too, is that when we're responding to federal reports, we're responding to the reports, and they may tangentially be related to the outcomes we're looking at, but not directly, right? Because let's look at the 2 differences. If we're measuring a program, we want to see if it works. That's the ultimate question. Does it work and can we afford it? 00:08:32 - 00:09:13 William The federal government, it's a little reverse. They're saying, "We want you to put these things into the database to justify certain behaviors that we want to incentivize through funding." And so it's a little bit different. And there is somewhat of a disconnect. But ultimately, when you're in child welfare, the federal government has outlined safety, permanency, and well-being as the primary drivers of your system. Are kids safe? Are they maintained safely in their home? And if you have to take them out of their home, are they there for as brief possible time in a natural setting as possible? And well-being, are they being educated? Are they getting health service, et cetera? 00:09:14 - 00:10:14 William So those are the things they're looking at. I think one of the disconnects, too, is when you look at just the quality of the data that's going in the federal system, right? Every county does it differently. Now, I know that you have state systems, but ultimately, the data comes from somewhere. So it's coming from a county worker somewhere. So using the first of the AFCARS. The AFCARS came out in the mid '90s and hasn't changed until it was released. The latest version came out in 2022. So think about that. Nothing's changed. The world's changed, data's changed, practices changed, but because it required an act of Congress, that's another thing that we see with federal reporting requirements versus something that's much more dynamic, like data that you control or that you're seeking. 00:10:15 - 00:11:04 William For AFCARS, you have over 200 fields. I want to think about that for a second. 200 fields of data, over 3,000 counties reporting that data, what could possibly go wrong? I know that when I first started ... I originally started in child welfare in New York and then came to Pennsylvania, and we're trying to figure out where this data was coming from the AFCARS data, because it was reporting the reason every single case was failure to cope, caretakers' inability to cope, which again, goes back to this antiquated system [inaudible 00:10:59]. It didn't reflect the practices because nobody actually knew what failure to cope meant. 00:11:05 - 00:11:29 William And so we discovered that our data was being entered by fiscal tech who just assumed that that's what everything was. Well, they're in care, so it must be failure. So whatever was coming in. And so we had something close to 100% of the cases or failure to cope, but that just illustrates the problem that there isn't that reciprocity in the reporting process that you would expect. 00:11:30 - 00:12:12 William So the other problem ... I'm picking on AFCARS because it's been around like the longest and it's probably the one that's frustrated most counties over the years. And it's also how the federal government pulls the data to determine compliance with many of the measures during the CFSR. And for those of you that don't know the CFSR, it's a roughly every 5-year process where the federal government goes to states and tries to determine if you're in substantial conformity with various measures on safety, permanency, and well-being. So they're pulling out a lot of that data from the AFCARS file. 00:12:13 - 00:12:41 William Problem is AFCARS was never designed to measure outcomes. And so right from the get-go, you're behind the 8 ball here. And if you don't have your own data system, you're really relying on somewhat flawed data. And so I know in numerous jurisdictions that we're in and the jurisdictions I've worked, we've had to have a separate data system to collect the data that was necessary for us to maintain our operations. 00:12:42 - 00:13:28 William And so I think that's one of the largest challenges that the immobility of the federal data reporting, and that's across all the systems, that it's very slow to catch up with what is actually needed and what's actually seen. I think one of the things that was really striking in Pennsylvania when we realized that the data we're reporting to AFCARS, we are inadvertently rewarding jurisdictions that never return kids because we're just looking at the time the child came in, the time the child was discharged. And then we look at, "Oh, here's the average time the child isn't discharged. Isn't that great?" Well, it didn't measure how long they're in. 00:13:29 - 00:14:16 William And so there was a whole data set. The other problem is it's not longitudinal. So you can't really apply it. Now, it's changed in 2022. Some things have changed to make a little bit more, but for instance, you're only allowed one reason for the child going into care. How does that reflect the real world? It generally doesn't. Again, I talked about failure to cope. That didn't really mean anything. Now, in the newer version, they've tried to define that a bit better, but again, it was very slow. So now you had what could have potentially been a great data set from 1995 to present that essentially is, I wouldn't say useless, but severely compromised. 00:14:17 - 00:14:58 William And then, of course, you have the usual mechanical things as far as putting the data in, who does it. It requires training, it requires expertise, requires supervision, and we know that the child welfare turnover rate is such that that's frequently lost. We're frequently called in just to help with the AFCARS file. That's stuff that's ... And I would say this for any consultant. I mean, thanks for the work, but you really shouldn't have to do that. It shouldn't be so complicated that you need us. It should be something that is so user-friendly that you go in, someone from college will look at it, "Oh, here are the fields here. The answer is this is how I do it." 00:14:59 - 00:15:20 William But that's not ... Well, anyone that's been familiar with government technology knows that that's not how it's designed. User experience isn't generally the first thing on their list. It's how do we capture that information so we can report the information for our funding purposes versus how is it usable information for the individual's jurisdiction's practice. I've probably gone on way too long. 00:15:21 - 00:15:56 Tarryl Yeah. Talk about us being able to talk about this for a long time. If you don't mind, I just want to add a couple more things into it. I used to serve in our state legislature, and both the feds and Congress and state legislators also create additional problems. There ends up being additional wrinkles about what you're supposed to put in, et cetera, but not necessarily the resources to make things up to date. And while a few years ago, I thought, "Oh, if our state could just get better software, more modern," our world's changed. As we're standing here today, our world's changed about what's possible. 00:15:57 - 00:16:44 Tarryl So if we're serious about being able to measure child and family outcomes or really be able to document what's happening, we need a whole different platform. We need technology to the extent we can future proof at will, and so that it really isn't just about those fields that you're talking about, Bill. I think the other thing that can be a little tricky to measure, and you've spoken to it, is the culture and change management that we need. There's still an awful lot of even inner county challenges between perhaps county attorneys and human services, or some county attorneys, or the bench, and human services, or others. 00:16:45 - 00:17:37 Tarryl So, for us, with how has Family First Prevention Services Act affected our child welfare practice, our folks are making those changes. They're working with families, we're working in community. I can't say there's a great way for us to capture all of that in this outdated system that you've been just talking about, but in addition, and I think we'll get to more of this in a little bit, a lot of this is about how we're helping move the culture of whether it's the judiciary or attorneys to not just be still thinking, "Hey, getting kids out of the house is the best answer or the right answer when Family First is a very different way of looking at this." And I know we, and I think you have, too, have seen some very good outcomes as we've internally made that mind shift, but we have to keep working on our partners. 00:17:38 - 00:18:16 William Yeah. And I think Family First has the potential to really incentivize practices across the board, positive practices. And it's a start. I wouldn't say it's a finish. It started ... Actually, just a little trivia. I'm wondering if anyone could even pop in, who the first president was that addressed congregate care and felt that kids were best served in their own home, and that the family was the best setting, and if not, something very close to natural setting should be the desired outcome. 00:18:17 - 00:19:02 Shannon I will watch the Q&A to see if anybody puts a response in there. Unfortunately, we have no way to reward people for their answers. But I think you've touched on a couple of really interesting points around the data. And Commissioner Clark, I love this idea of there has to be change appetite before there's ever change readiness. And we talk about that a lot at CAI, that is that movable middle of folks who are willing but not sure how. And once you get them where they have that appetite to change and they can see those outcomes, that starts the groundswell to move more people to understand that this is a good thing. 00:19:03 - 00:20:01 Shannon But in order to do that, it goes back to that, what are the measures? What is the data? Is it accurate? And 2 things come to mind for me when you talk about data collection practices. One ... And we've seen this certainly in other areas, not just in the federal requirements. When you have to collect all of this data, it's not like all of these different systems talk to each other. So you're injuring more than once. And Commissioner Clark, you and I have talked about this, that because your systems are so not modernized, that it's not just 17 minutes to boot it up. It's also how much time is it taking for people to take the data and reenter all of the data, and how many errors occur there along the way? 00:20:02 - 00:20:27 Shannon I think we'll start with Bill on this one, talking about the processes or mechanisms that you use to maintain data quality. How do you ensure that collection of accurate data? Because case managers, they are doing their best. They're very focused on the children and the families, and when they have to reenter these things multiple times, it gets really challenging. 00:20:28 - 00:21:07 William It's very challenging. And, like you said, when you go back to federal reporting standards, every time there is a new thing, whether it's the adoption and safe family or whatever the case is, none of the systems talk to each other. So they develop a new system with new, unique identifiers. So you can't really track kids across all these systems and would be wonderful. To ensure quality data, then you have a way of cross-checking the NCANDS data, the child abuse data. You'd be able to check that data with the placement data in detail and follow that cohort for a long period of time. 00:21:07 - 00:21:52 William So how do you do that? How do you maintain that quality? One is that you have to make sure that you know what you're measuring because you know you have to measure the ... Let me stop there. Not necessarily measure. You know you have to enter the data for the federal reporting system. But you also have to make sure that within the agency, if you're making these decisions, that you have enough data to make those decisions. So it sounds oversimplistic to say like, "What are you measuring?" But you have to ask that question because it's very easy to start saying, "Well, I'm starting a program. I'm doing this," without actually saying, "What is the outcome that you're looking for?" 00:21:53 - 00:22:18 William So what are you measuring? What do you desire? Once you answer those questions, then you can start talking about quality data collection. And then you can start talking about how you manage the quality of that. And the only way you maintain quality is ensure that it reflects the real world. So it's great to have these great aggregate databases, but if it's not reflected in reality, it serves no purpose. 00:22:19 - 00:23:17 William I could give you an example in a state which will remain nameless. I was chairing a committee and it was on congregate care. The board of ed secretary was present and began speaking to me and several judges throughout this jurisdiction, saying that children have quality education and residential facilities. And like, "Excuse me. All right. Okay. Have you read there's a report that came out uneducated and" ... Anyway. And they said, "Well, so [inaudible 00:23:05], tell me how you came to that determination." Well, based on the data that we received from the residential facilities. I said, "So talk to me about that." 00:23:18 - 00:24:01 William Turned out that it was all self-reported data. And then they only did onsite reviews every 6 years, which, again, reflected whatever was given to them. And so you had an entire department in a state making education decisions and licensing decisions based on data that had no connection to the real world because the real-world studies had shown that the credits weren't transferable back in their home districts, that the rate of dropouts were higher, et cetera. That was readily available, but that wasn't ... They were just looking at descriptive data from a flawed data source. 00:24:02 - 00:24:46 William And so that's probably one of the ... It's a large one from a smaller jurisdiction, again, looking at, does it make sense? A jurisdiction we're in looking at the number of kids that were served. They had ... And it was a very small jurisdiction, had over a thousand kids open for service in this very small county. And I asked a simple question, "How many cases do you have open now?" 27. Clearly, that data didn't match, and it was reported probably at least for the last 5 years, because they're required to put in the data for 5 years. 00:24:47 - 00:25:20 William So 5 years of data, funding, et cetera, just because that's a step that I probably most commonly see that's left out from quality data is, does it make sense? It's simple and it requires you to refocus and talk to the practitioners. So that's where the disconnect happens. If you're just doing the data and grabbing data without talking to the practitioners, you're going to miss something over time. And the longer that goes, the greater that error is going to happen. 00:25:21 - 00:26:04 William So I would say that that's probably one of the key things. The other thing is if you're trying to try to start a practice is you have to be data agnostic to start with because then you start playing just to the data. By that, I mean, you just come up with, "This is my end goal. This is what I want." And then you have the brainstorm with the practitioners because they're going to know that they're the subject matter experts. They may not know the data and some of it, you're going to have to deal with the confirmation bias because all teams are at this level of care, et cetera. You can deal with that, but you need to get their input to keep on self-checking and self-correcting the data. 00:26:05 - 00:26:36 William And that's why the federal data that I talked about it is so it doesn't pivot. By the time you realize the data's bad, you've gone a couple decades. So I think that's probably the biggest thing that I would focus on is ensuring not only that the data is entered in a timely and accurate fashion, but it reflects the real world that you're seeing, and you have to check that periodically and set that as part of your practice. When do I actually go? 00:26:37 - 00:27:03 William Now, the feds try to do that with the CFSR. They go out, but look at it. They do a very small population set. So they're seeing the direction, but it's a tiny, tiny subset of the cases that even a county's dealing with when they try to look at the practice to see if the data reflects what they're seeing, but at least it's an effort. When you're local, you could do it more extensively. 00:27:04 - 00:27:08 Shannon Right. Their sampling may not accurately reflect what you're seeing across this space. 00:27:09 - 00:27:10 William Correct. 00:27:10 - 00:27:42 Shannon Yeah. Thank you, Mr. Browning. I think that what I'm hearing from you, which can be challenging for smaller counties, is this ability to ... We treat audit as a dirty word. And really, it's about that quality assurance of the data and the processes when you're able to do that. It can't be every decade, every 6 years. It has to be in a more timely fashion, but that's very tricky when you don't have the resources. 00:27:43 - 00:27:54 Shannon Commissioner Clark, with your permission, I'm going to pivot a little bit about how that influences the data decisions, unless there are some things you wanted to tag into what Mr. Browning said. 00:27:55 - 00:27:56 Tarryl There was a couple things, if it's okay. 00:27:57 - 00:27:58 Shannon Yeah, go for it. 00:27:58 - 00:28:32 Tarryl I go to where you're wanting. I was thinking a couple of things that you were describing, Bill, reinforce the challenge we see. If it's somebody at the state that's entering and they don't actually understand and they haven't talked to any of the practitioners, how useful is it? So if we think about it, we want to have good systems so that we can work together with families, dare I say, work with families. Where are they at? Where are they? And we hoping they're going to go. We want kids to be safe. We've got lots of things we want to have. 00:28:33 - 00:29:23 Tarryl So part of this is very family focused, thus Family First. And then we've got the piece that you just reiterated, which is let's have some accountability. And ironically, if the feds want to find some waste, it's partially how they've set up these systems because they don't have good current systems that help tell the story that can help staff and families and community make good decisions that we can actually figure out what's paying off the most, where are you getting the best benefit, et cetera. And I guess I'd argue that I'm not sure it's ever very helpful that somebody from the feds is saying, "Hey, we're here to help," when we should have the means to be able to have the system help us with those audits. 00:29:24 - 00:30:04 Tarryl So either we all have to figure out how to all talk to each other a heck of a lot more and then nobody's going to get to the work or we need better platforms that can help with that oversight part while we could also pull for the individual families we're talking about, but then have some way to measure so that broader systems can figure out, going back to what you said, Bill, what actually is working or not. I can't say that's what currently is being gathered. So we could do better. And the thing I love about NACo is we're ready to all figure that one out. So I don't know that that helped pivot you enough, but go ahead and keep pivoting. 00:30:05 - 00:30:49 William I'd just add, too, is that the cost of doing it, you brought that up with small counties. The federal government, remember, only gives a ... It's an uncapped entitlement, but it's still 50%. So the 50% match has to come from somewhere and it's competitive in these times. So to develop that system that's robust at the local level and invest in those systems is generally cost-prohibitive, especially when you're talking about county-administered systems. And we've been in jurisdictions where the state has provided very little financial participation or support with the child welfare, and then vice versa, where the majority, even in county driven system, the funding comes to state. 00:30:50 - 00:31:02 William So that's something that needs to be addressed if you want to give the data because 50% is not a lot when you're talking to robust data development system. 00:31:03 - 00:31:06 Tarryl Agree. Well said. 00:31:07 - 00:31:40 Shannon Well, and I think this really speaks to this idea of how can we focus using the data to create those best practices for effective service delivery and management, because too often the data is we speak about measures like timely access to services. Well, are those services effective? It's great that you can do it very quickly, but are they the right set for that particular family or child? Is there the right kind of attention to the effectiveness of the services? 00:31:41 - 00:31:55 Shannon So in your experience, how do we get the right data to ensure timely access to effective services? What are the benefits we see when we're able to use data that way? 00:31:56 - 00:32:00 William I mean, it's mirrored. You could do a whole presentation on that [inaudible 00:32:02]. 00:32:01 - 00:32:06 Shannon I was about to say. I think I just unleashed the facts. 00:32:07 - 00:32:46 William Yeah. I mean, there are so many examples over time where we're just looking at the data effectively and across systems. That's probably ... I think people and departments develop fiefdoms of data. So they assume that once it's not my problem, it's your problem and that they're not connected. An example ... And this is famous in child welfare, right? The child that is in an expensive placement. Eventually, they're going to age out and there'll be somebody else's problem. Well, that problem's not free. It's costing you. It's costing society. 00:32:47 - 00:33:36 William One example that comes to mind is that analysis where we found that residential placements in this one jurisdiction come for nearly 50% of their purchase service budget. And then of those, 60% were limited to just 6 kids. There are high needs kids that are receiving ... And this was just from ... I brought members from behavioral health, et cetera, drug and alcohol together to show, said, "This is just mine." These are the thousands and thousands of dollars we're using to warehouse these children. Because let's be honest ... And I won't go on my soapbox because, again, that's another about congregate care. But we've only known that ... Oh, by the way, the answer to that was Teddy Roosevelt. So we've known it's [inaudible 00:33:37]. 00:33:37 - 00:33:40 Shannon Oh, and someone did answer that. Yes. 00:33:41 - 00:33:42 William Someone answered that, great. 00:33:42 - 00:33:43 Shannon Anonymous attendee. Good job. Yeah. 00:33:44 - 00:34:19 William Good job. Yes. In 1909. So we've known it's a problem. So here, we have 6 kids that were driving a huge, huge amount at a dead end as far as their growth, and they're going to age out, true. But statistics tell us where they're aging out. They're going to be in higher levels of service as adults. They're possibly imprisoned. We know that kids that leave institutions going into adulthood, 70% of them wind up with the criminal system. 00:34:20 - 00:34-21 Shannon Yeah. 00:34:22 - 00:34:57 William So we know there is a cost. And I think that's probably as being a former health and human service director, that would frustrate me is everyone assuming that the problem goes away because it's no longer right in front of your department. And that's ... I know it's like a kumbaya kind of thing, but it's not. It's functional. It's usable. It's something that we'd like to see is that data sharing because ... And I've said this repeatedly throughout my career and I think it's been correct that good practice pays for itself. 00:34:58 - 00:35:39 William When you look at the bad practice that was in the one agency, and I could say it was bad practice because it was my own agency at the time when we did the analysis, it did. It cost us a lot. When we limited that, we're able to maintain our budget by just looking at the data, looking at these high users and what services they're involved in, and what other services could be brought in that were natural supports, that were supports from other services before it was at a crisis level. We were able to maintain the same budget, but actually cut 5 million from our budget while investing it into other services and then maintaining it for 10 years. 00:35:40 - 00:36:24 William Just because we started looking at the data and looking at what do the children, that these high needs children, require for us to provide to them before they just age out of the system. And guess what? They become parents, too. And what I think everyone realizes that's been in child welfare that there may be some transition, but usually, a lot of the higher needs that don't have the resources to move very far, they have children. And then you become the multigeneration of poverty, deprivation. And so you're paying for it for a lifetime. And so I think ... I know it may sound a little bit overdramatic, but the cost of not measuring it is immeasurable. 00:36:25 - 00:36:59 Tarryl And that's where, I think, the kinds of systems that at least the feds are using for reporting are totally ... They're just inadequate. They're woefully ... There's a lot of strong words floating around before and after our board meeting today, a little bit about this talking with our human service administrator. I'm going to give an example that backs up what you're saying in our county. We actually got last year a best-in-class award from NACo for our junior or juvenile community action teams, which was based on one we did for adults. 00:37:00 - 00:37:48 Tarryl So for those of you who are also doing adult work, we looked at who were the biggest utilizers. I'll use the adult part for a second, utilizing, winding up either in detox, the emergency department or in our jail, none of which were the right places, but lots of resources being expended. And frankly, their lives were pretty horrible. Working with internally and breaking down silos as well as with law enforcement, our community mental health center and others, really looking at how, with those folks, could we really adjust how we were working together, what were their services? And we saved a boatload of money, and probably more importantly, those folks' lives got a lot better. 00:37:49 - 00:38:37 Tarryl We started going upstream. And sometimes it's hard. From a budgeting standpoint, if you don't see it within a budget cycle or 2, it doesn't exist. We have seen through good data, but it's hard because it doesn't exist in the systems that we have been given to use. We could show that. So we started doing that with our kids as well because, to your point, Bill, if you can be working on this sooner, if you can be helping parents do a better job, et cetera, working with them, we could start seeing better outcomes. So we know that's a good example of better outcomes, more prevention by resourcing upfront, fewer out-of-home placements, and saving taxpayer dollars. 00:38:38 - 00:39:22 Tarryl Is there a great way to show all that right now within what reports get? No. We have to have different ways of getting that information. And truly, part of what excites me knowing how challenging many of these families' life situations are or the decisions they've made or what's going on in community is if we can get our hands around some of this and learn from each other because we've got better data, then we could start seeing those kinds of outcomes and show what actually works, where the resource ought to be. So that's an example for us. 00:39:23 - 00:40:29 Shannon I'd love to dig in a little bit into this around really thinking about how you're talking about that longitudinal data, which Bill referred to at the very beginning of the conversation. And you've got this really positive success story. And data is used for decision making because it's used for storytelling. And when you can show that thread that says not only are the outcomes better for this particular juvenile community action team impacting these specific folks, it also shows that, as they are getting older, they may have siblings, as they are graduating into additional services that the county might have to provide for them, it is lessening the burden all up, but also they're having much more positive outcomes for their lives because we are talking about people's lives here. 00:40:30 - 00:41:04 Shannon Can you speak a little bit to this idea of data-driven decisions? Was that a hard thing for the JCAT team to get around where you're seeing that they were really able to showcase that idea that you were having fewer out-of-home placements, that you were saving taxpayer dollars? What are some of the things that you felt really showcased that ability to tell that story using those data-driven decisions? 00:41:05 - 00:41:45 Tarryl Well, this won't surprise anybody who's done a lot of work in community or within counties, but in some ways, it really starts with those relationships and just seeing how much it's costing. So it can be hard because right now, there's not great other data out there, but you could see what's happening in your budgets. Some of the highest cost drivers for counties, at least in our state, are things like commitments, out-of-home placements, adults who are vulnerable. And I apologize, somebody decided to come do lawn mowing outside my window. 00:41:46 - 00:42:34 Tarryl Being able for the folks who are really doing the work to be able to say, "Look, we think we can demonstrate and illustrate and tell the stories both from a quantitative and a qualitative standpoint about the differences that are making and getting to your longitudinal piece, we need to be tracking over time." I think, really, it was leading with a positive mindset about what could be, and then being able to use the data to help get more buy-in. Sometimes I think because the feds aren't collecting all of this, it's still hard for us to show it. So if we're still largely about inputs and outputs and how long, it can be harder to tell that story. 00:42:35 - 00:43:08 Tarryl But us figuring out how to get that in so that it does help others to be able to see it, we might not be able to justify the prevention or the more upfront services for what we're going to say is going to happen in a budget 10 years from now, but because we are seeing some of that happening within a couple of years, it means that those who are putting the budgets together and the budget proposals could start showing, "Look, we've been able to bring this piece down." 00:43:09 - 00:44:04 Tarryl I will also say we haven't brought up our workforce yet. My end with all of this is we know that we are saving money when we're retaining great team members, in part because we don't have to train them again, part because we don't have to make somebody go through all the crummy learning of how to use old systems, but it really makes a difference with who they know and how effective they are. So us being actually ... I would say, maybe for those of us who are approving budgets is being able to see that we're having that workforce piece enhanced because of all of this can make a real difference. So as we're doing the technology systems, being able to throw that in the mix, I know, is a real monkey wrench, but I'm just going to say it out loud. 00:44:05 - 00:44:43 Shannon As you should. That's a great point. Workforce retention, particularly in a space that has high levels of burnout, is so critical. And for them to see that positive shift is so important. Mr. Browning, I'm sure you have a lot to say about this idea of ongoing quality improvement and that cross-service collaboration. You have about 4 to 5 minutes to talk about what you've seen in terms of fostering that culture of continuous improvement and collaboration. 00:44:44 - 00:45:16 William It doesn't have to start as something major. It just has to start. And so I recall in one jurisdiction, I started it, and I came in and asked, "How many kids are in placement?" And the managers said, "I don't know, we wait for the AFCARS data to tell us how many kids are in placement." Well, anyone that knows AFCARS data, it's reported quarterly. And so they had no idea point in time even how many kids were in care. There was no computer system, and this goes way back, so I will date myself. 00:45:17 - 00:46:06 William So I went into Lotus. For those of you that ... There used to be a program called Lotus that Microsoft stole and made better, arguably. And so I started my own database, literally having workers go to units and hand counting every kid on the caseload, talking to workers. It was a laborious process, but it started something. And so we started instilling that. That was the culture. We started saying, "Okay, we're measuring kids in place. That's what we're targeting. We're measuring ... All right, now that we know how many kids are in placement, let's drill down a little bit further. What kind of placement are they? Are they in kinship placement? Are they in congregate care placement like group homes or institutions, et cetera?" 00:46:07 - 00:46:40 William And then from there, how long they're in place, because we couldn't necessarily wait for the data to come back to us from the feds because, again, that's not real-time data. There's a significant lag time between the time the data's released and what they're measuring. And plus, again, it's not longitudinal. So at that time, it was just data for kids that were in placement once. The most recent placement. So it didn't take all that historical information. And so we had to continue. 00:46:41 - 00:47:31 William So that would be one of the things I would do is just starting somewhere and then institutionalize in that practice. I started publishing daily data reports. Now, I know point in time data is of limited usefulness in the overall, but it got the entire staff and it was released to all the staff, caseload size, times and everything. And so they would see that. And it was a little controversial in some because I put it by unit. But what happened was instead of it being negative, people started looking and saying, "I think that data's wrong." Before, they didn't know what the data was or how it was important. No, that data's wrong because that average should be because I've done my supervisor reviews, I've done this, I've done that. 00:47:32 - 00:48:09 William Now you expand that to agency wide, and then from the agency to department, to department to county, you start having a really robust environment where data is the expectation. And I think that's something ... And you use your wins. You show them. So we used the data to inform a truancy program because we asked, what were some of the major issues? Now, of course, you have the confirmation bias. So they decided initially, "Well, it was a teams with behavioral health issues was the number one issue." 00:48:10 - 00:48:47 William Well, no, that was the number one issue they were dealing with because they took a lot of time. But when we looked at a referral system, the data coming in, they were truancy. Truancy ones that led to placements because then they became ... They were charged with contempt in jurisdiction that I was in. The judges would incarcerate parents, which does great for truancy, by the way. Not. And so just by using that data, we established a new unit just to deal with a school liaison unit. The filings went down by 90%. 00:48:48 - 00:48:49 Shannon Wow. 00:48:50 - 00:49:15 William And so we used that data to showcase the staff. This is what data does. Here's what the problem was thought. Here's what we identified the problem is. We put in what we thought was a solution, we measured it, and we actually decreased referrals coming to the agency because of truancy, because we developed this liaison unit and then continued tracking it and refining it over time. 00:49:16 - 00:49:43 William That data then led to relationships with the schools. Then the schools then started partnering with the workers and their assigned people to start dealing with some of the social issues that really weren't abuse and neglect, it was poverty, and started having that reciprocity and exchange of information to deal with the issue before it even came to us, kind of the spirit of what Family First is. 00:49:44 - 00:50:07 Shannon So what I'm hearing is you're able to pull those additional levers once you get that first set of data in place and get everybody used to moving forward with almost everything from ... It's like a daily health check, right? And then you're able to move into those more qualitative types of concerns. 00:50:08 - 00:50:18 Shannon Commissioner Clark, I think that you have some thoughts around that qualitative type of improvement on the case manager side as well, if you'd like to touch on that. 00:50:19 - 00:51:03 Tarryl Yeah. I mean, I think it builds well off of what you were just talking about, Bill, to that continuous quality improvement and then the facilitating collaboration. So, for us, a lot of the steps we've taken is things like increased training in regard to collaborative safety, doing community engagement, a lot more community engagement, identifying gaps with each system partners, and trying to strengthen them. Having crucial, those really important conversations with system partners about the challenges and system issues. And a lot of it really has focused on reducing that turnover I brought up, which really supports staff. 00:51:04 - 00:51:33 Tarryl The other thing I wanted to bring up was things like addressing secondary trauma, for example, another subject we could spend a lot of time on, probably altogether. And all of our staff are now trained on motivational interviewing. Whether it's in community corrections, whether it's in our child and family, whether it's an adult, it makes a big difference if you are being able to modify the techniques that you're using, too. 00:51:34 - 00:52:05 Shannon Thank you, Commissioner. So I think it's really interesting that we start with data, and we start with systems, and we start with accuracy, and we come all the way around to material impacts, human improvements, both for the staff as well as for the clients that we serve. It's really ... That balance is something that I think is just so critical and why this is such a topic that we could continue to talk about for quite some time. 00:52:06 - 00:52:21 Shannon So we are getting very close to the end of our time, unfortunately. I did want to pause. I do not see any questions in the Q&A. So I would invite folks ... Cameron, are they able to raise their hand or should they be typing in the Q&A box? 00:52:22 - 00:52:24 Cameron Leonard The Q&A box would be better if that's [unintelligible 00:52:25]. 00:52:25 - 00:52:27 Shannon Okay. 00:52:28 - 00:52:40 Shannon So we will give it just a pause to see if we have any questions. In the meantime, I would like to share that if you're interested in learning more, we do have lots of information. [Presentation slide 3. This slide is divided vertically into two sections. The left section features the "CAI" logo and tagline "We power the possible" in white text. Below is a QR code and text to the right reads, "Thanks for joining us!", followed by contact information: "www.cai.io,", "@CAI," and "@CAI_Insights," each with respective icons. The right section contains an image of a family of four smiling.] 00:52:41 - 00:53:05 Shannon This is a space that CAI really looks at programmatic and fiscal improvements, not just technology, and the child welfare and behavioral health space. We've spent many years. Bill won't confess how long he has worked in this space, but our combined expertise is hundreds of years, and it's just such a great topic [inaudible 00:53:06]. 00:53:06 - 00:53:07 William And it's not just me. 00:53:08 - 00:53:12 Shannon Yeah, exactly. Right. You're not the hundreds of years. 00:53:13 - 00:53:22 Shannon Let's see. So there's a question here that says, "How can EI help this issue?" Is that how can I help this issue? 00:53:23 - 00:53:24 Tarryl I think AI. 00:53:25 - 00:53:33 Shannon AI. Oh, thank you. Okay. So I think there's a ... Yeah, go ahead. 00:53:34 - 00:54:17 Tarryl Yeah. I'm not the expert here, but I'm an and, I can be taught new things. I mean, a lot of the reason why I end up coming into these spaces is how can we apply technology that can make a real difference to better meet our residents, our businesses, and others where they're at, and hopefully not get outdated right away so that the back door could meet multiple possibilities and the front door for those needing it will be much more intuitive. And whether it's more take ups and services or whether it's we don't need this anymore or something else, we could do a lot better. 00:54:18 - 00:54:44 Tarryl And for those of you who are still wondering whether that's the case, AI can help us, and the folks who are on here, not to sound like a pitch for them, but we've got great NACo partners that really can help us figure out how, in a cost-effective way, to better meet our needs. So anything else you 2 want to be saying on that because it's really important. With guardrails, of course. 00:54:45 - 00:54:46 William Sorry. 00:54:46 - 00:55:20 Shannon Right. I think, Mr. Browning, I'll take ... Just to put a bow on that, there is a lot of really interesting work happening in this space that's really looking at how AI can pull data from legacy systems. I think that is probably one of the most critical path items. Unfortunately, we all got it wrong, and Mr. and Ms. Zaluska, I apologize, actually wanted to ask about early intervention. So how can early intervention help the issue of measuring outcomes? 00:55:21 - 00:55:57 William Early intervention, that's one that usually is under the human service department. I think the communication, some systems have that really built in well to their systems. So the child welfare agency will make the referrals to early intervention. They'll use the ASQ to screen kids that are in the system to make sure that they're active referrals. I think the primary collaboration is ensuring that the children are identified prior to coming to child welfare. 00:55:58 - 00:56:33 William I know that there's been a lot of discussion with some funding issues with early intervention. And I think that the data does speak for itself, that children that have received early intervention services are less likely to come to our system or at least be firmly embedded in the system, meaning place, et cetera. Their educational outcomes, at least the short term, are better. You go to the subset of that, they had the head start stuff. So anything that could be done proactively is going to affect the data. 00:56:34 - 00:57:20 William And I think that's something that when we talk about Family First, we're just in the nascent stage. I would have hoped that Family First, that the clearinghouse would have included more early intervention related or those proactive services, and it might, but the services that are allowed to receive the IV-E funding, it's relatively small given the number of services. So just like the California evidence-based clearinghouse that has pretty much the same criteria that the federal clearinghouse has, but it's a much, much broader array of services. It's not just limited to preventing placement and directly limited to abuse and neglect. It could be tangentially related to that. 00:57:21 - 00:58:17 William So early intervention may not directly affect abuse and neglect per se, but it does affect it indirectly. And a good example of an intervention that wouldn't be on the clearinghouse that would be approved for IV-E would be a program that I was involved with that lowered or raised the age that children experimented with drugs. And we know that that's tied to them developing a substance use disorder, which is related to a whole lot of societal ills, such as placement, et cetera. It won't ever show up on the list because it's not directly related to the prevention of child abuse, neglect, and placement, but it does lay the foundation to prevent child abuse, neglect, and placement. 00:58:18 - 00:58:56 William And so I think that ... I'm hoping over time and not in fed time, like in people time, that they're able to look at that again and say, "Look at this, this list is not growing fast enough. It's too small." The fact that you're limiting the states that 50% of your prevention budget has to be invested in the things that are in the clearinghouse, I think that's overly restrictive and several jurisdictions are dealing with it in different ways. You mentioned early intervention, that's just one of the ways, but there are many others. And I won't go on because, again, I could [inaudible 00:58:56]. 00:58:57 - 0059:29 Shannon Yep. And I think early intervention can be in a lot of different places, and it really ties back to that longitudinal measures that we continue to talk about. So we know these aren't quick fixes, and it's all of the things that we've spoken to today really inform that ability to make those more robust decisions when you have that ability to collect data and really think about the decisions you need to make for impacts. 00:59:30 - 00:59:55 Shannon So with that, unfortunately, we have run completely over time, but really good question, and thank you to the person who also spoke to Theodore Roosevelt. And Commissioner Clark, Mr. Browning, thank you so much for your time today, and this will be available as a recording on the NACo website shortly. So thank you all. Have a wonderful day. [Closing slide 1. Blue CAI "We power the possible" logo appears in middle of screen. Company website www.cai.io appears at the bottom center of the screen.]

Enhancing child welfare: Best practices for measuring and improving outcomes

Download transcript (.txt)

Video description

Watch this on-demand webinar, hosted by the National Association of Counties (NACo), featuring experienced county leaders focusing on child welfare specifically designed for county governments, with lessons that can be applied to other behavioral health programs. Speakers review evidence-based practices, informed funding decisions, cost efficiencies, and effective program management. Key areas of discussion include out-of-home placements, prevention services, and reunification services. Also discussed are valuable insights into identifying service gaps, leveraging data for increased service utilization, and supporting fiscal efficiencies

Learn more about our commitment to a principled approach for child welfare and behavioral health services that are people-centric and collaborative with an eye toward stewardship.

Contact a specialist

Transcript

Skip past transcript

Related resources

IT vendor management best practices for the public sector
Thought Leadership

IT vendor management best practices for the public sector

When it comes to external contractors, public agencies have a lot to juggle. Utilizing IV&V in IT vendor management can make things easier.

View this resource →
Project coordination for a global logistics company
Success Story

Project coordination for a global logistics company

Seeing measurable results in a short amount of time, a logistics company looks to CAI as a trusted partner—not just for project coordination, but in future planning and strategizing as well.

View this resource →
7 ways business architects are vital to enterprise transformation
Thought Leadership

7 ways business architects are vital to enterprise transformation

Public agencies can meet the challenges of the future head-on by improving their organizational efficiency with business architects.

View this resource →

Let's talk!

Interested in learning more? We'd love to connect and discuss the impact CAI could have on your organization.

All fields marked with * are required.
Please correct all errors below.
Please agree to our terms and conditions to continue.

For information about our collection and use of your personal information, our privacy and security practices and your data protection rights, please see our privacy policy and corresponding cookie policy.