This is the final report from an actual Six Sigma project example that improved the processing of incident reports. The organization provides behavioral health services in three locations. Incident reports are required by state law, and for accreditation purposes. The old system was averaging 120 days to process reports, mostly because of a few outliers. Even without those outliers, time was 21.5 days. The new process reduced that to 8 days, and even that is expected to decline as the new system gets past its introductory phase.
Whitney: Hi Denton.
Denton: It’s good to have you with us. Tell us about yourself and tell us a little bit about your company if you would.
Whitney: Ok. I work for behavioral health company that has three residential treatment homes that are bit spread out and we also have a business office. I have worked for this company for about seven years and I’ve done a variety of project work and have accumulated different experiences with the company over time, including developing their, you know, the compliance role there and quality improvement and other operational areas within the company.
Denton: Sounds like a perfect place to apply some some Six Sigma. What was the problem that your project set out to solve?
Whitney: I started looking at our incident review process earlier this year and we were kind of doing that in the context of some staff efficiency questions and we had done some tweaks but we wanted to do more and particularly I really wanted to see us get something that was a lot more stable as a process. So we have these incident reports that we’re required to do for different reasons, primarily for state licensing, but over time they had just kind of accumulated other burdens of licensing an accreditation and other organizational goals. So they kind of become a little bit of a behemoth of an organization.
Denton: What might an incident report entail? Tell me about these things.
Whitney: The primary reason is to capture safety concern. When there’s an event that happened that would be out of the norm, that we would want to track so that we can evaluate our safety performance over time.
Denton: Ok, so it’s completely involved in the whole objective of trying to make the facility a safe and good place to be.
Denton: Ok, alright, and plus the state requires you to do that. So I would see that as a pretty important thing so that your licensure is safe and so that your people and your clients are safe and have a good experience and go home in better condition than they came.
Whitney: Right. So a huge part of this is also the data tracking is supposed to inform our attention on to an area of concern for safety so over time we can improve and become more safe.
Denton: Sure, it’s your continuous improvement. Every organization should have something like that. Ok, now I notice you list a cost reduction down here in your charter. I wonder if this isn’t a little bit more like actually what we call a compliance product. Tell us about that.
Whitney: Yeah, I definitely think it is much more of a compliance project. It was actually doing the project charter itself that really brought into light. I kind of started in this initially thinking about, well how much time can I say how much worker time and how I translate that into dollars? But we had already done some work on the on the kind of staffing efficiency side so really the game was, you know that improves compliance and kind of addressing actually the lag-time in how long it took the incident report to get created all the way through to being available for data analysis or getting filed in the right place so that they could be available for example, like a state audit.
Denton: Ok, tell me how you assess your initial condition. You looked at this and there were some data that convinced you that you that happened that you had a problem, tell me about that.
Whitney: Yeah, I had some anecdotal data before I started the Six Sigma project that we had some issues with latency but I didn’t really know how bad it could be, and I also wasn’t really sure that during the time frame that I was collecting this information, that I would get some good examples. I think I did get some good examples of how bad the process can be because as you see, we have three really bad outliers in terms of our latency time from the time of the event to it actually showing up and our business office to be processed and filed correctly. Those we have are two years or longer. That’s pretty long time to get a report through the system.
Denton: Yeah, that’s that’s a long time alright, Whitney. In every project, there comes a decision that you have to make. We have to decide am I going to try to fix and patch up the old process? Or do I need a new process? Now, just looking at this data and interpreting the capability study, this looks to me like this would be a hard project to just fix. What decision did you come to and tell us what happened there.
Whitney: Yeah, you know I think if you take those three data points out you’ll still see that generally we weren’t performing the way we wanted to in terms of getting these reports processed in a timely fashion. So I think the mean in there 21.5 days, and that’s three weeks, that’s just really too long to meet a lot of our requirements for state reporting. If we have an incident that will require reporting. So I think, you know, that might have been a system if we were just looking at that 21.5 days, we might have just said okay, maybe we can just improve upon this. But I think that the three outliers highlight a problem that was in the system. We were using a paper-based system and there was no way of knowing when something had gotten clogged somewhere. Completely impossible to track and I don’t know what the story of these three incidents but I have had stories of people finding a stack of incident reports stuffed in a cabinet somewhere and they bring it to me it’s like six-month-old.
Denton: Oh dear. So this process is not stable and predictable and the average is much too long and so I think you made the decision that you were going to replace the paper process did you not?
Whitney: Yes, I ended up replacing it with an electronic process that is embedded into our electronic health record.
Denton: Cool. Can you explain to me how much better the new processes than the old one?
Whitney: Well it’s a great deal better. You can see from this chart, the mean went down to 8 days. Which is a big improvement. So we’re talking just a little over a week and this was, you know, still trying to fine-tune of a little bit too and still learning through that process of, you know, putting out the initial new system and evaluating it. I think we thought a huge improvement out with just introducing the new system.
Denton: Ok, I really like using the split process behavior charts you’ve done here but I notice that you also gave me a t-test. Now the t-test is usually something we take up in black belt. So it’s kind of an advanced tool. Why don’t you tell us a little bit about that?
Whitney: Yeah, I had recalled using t-tests from my statistics classes and I was kind of familiar with it but it was kind of an opportunity also to reorient myself to it, which is why I wanted to play around with it. So a t-test is used to evaluate if you have, in this case, is the mean from one sample the same or different from the mean from another sample? Here you can see that, at least on the right tail test, that these means are statistically different. The problem here is that there are some assumptions behind t-test. One is normality. Another is homogeneity and I actually don’t recall the third.
Whitney: This is a distinct sample. So this violates the homogeneity and a very big way. You can see that those outliers are really affecting that first sample so it violates that assumption.
Denton: Just from my experience, Whitney, and watching people use the t-test, homogeneity is the big assumption and most people get fastened on normality which they don’t need to worry too much about. Now, also at the end of your project you put in place a control plan to make sure the things don’t drift back to the bad old days. Why don’t you tell us about that a little bit?
Whitney: Yeah, so as I watched the process play out with a new system, I observed that, you know, they’re still need to be some kind of prompt for people to actually complete the process of the incident review, that allows managers to get feedback to staff so they know if they performed correctly in the event of an incident and also enables us to quickly identify safety concerns. So, kind of the primary function there and I didn’t think that we were going to get a kind of across-the-board really well enforced deadline requirement. So what I did was I created a system of automated alerts using an RSS feed to notify managers when they have an instant report that they have to review. I also created an alert system for me that triggers me to pay attention to the types of incidents that may have a reporting requirement. That way I can interface very quickly with a manager after an incident and let them know that there is going to be a reporting requirement, so they are going to have to pay attention to a deadline.
Denton: Good. I’m just going to say it sounds like you’ve really thought about getting people notified and escalating it if it doesn’t get immediate attention.
Whitney: Yes, that was kind of, that was the hope there.
Denton: Good. Go ahead.
Whitney: So the other thing that I worried about was the drift that happens when people are training. I think there’s a lot of training that occurs by, well this is how I do it and not necessarily based on process instruction. Every healthcare company or business that’s accredited through the Joint Commission has a requirement to have a competency assessment and it needs to be done at hire and it also needs to be done at least every three years. We actually do ours annually. So what I did with on the competency assessment for all employees, instead of just saying trained to do incident reports, it now says trained to review the instructions on the incident report that are in the system that we have. So we’re having them look at the embedded instructions that we’ve generated versus having something that’s kind of more evolving as workers start to pass information from one to another.
Denton: Yeah, that’s a fundamentally correct concept. The whole thing of having standard work instructions and not allowing it to function by word of mouth, as you put it, you know, one employee says well, this is the way I do it and the system evolves without your knowledge. Whitney, this is just a splendid project. I really want to congratulate you. This is very well done.
Whitney: Thank you, Denton.
Denton: Thanks for being with us. Whitney.