Don’t Make Perfect the Enemy of Quality

During the Quarterly Data Quality (QDQ) process, agencies check the data of the clients they serve and try to get the best information in the HMIS database possible. The data in the system is scored based on a rubric and a percentage score is assigned to the provider in which the client is enrolled.

QDQ is, in some ways, a snapshot of the data in HMIS. One can think of the initial report that is pulled as the true snapshot. It shows the state of the data at that moment. There may be missing data, inconsistencies, or data entered long after a client has started a program. We can think of these as errors or blemishes but, when assessed against the rubric, they are simply turned into a percentage.

To continue the analogy of the snapshot, imagine a picture of three friends. One friend is kneeling, one sitting in a chair, and one leaning down to fit in the picture frame. One friend has some spinach on his smiling teeth. Another is slightly shaded by an overhanging structure outside of the picture. Think of this snapshot as the initial QDQ report.

The snapshot is handed over to a photo editor who removes the spinach from the picture and diminishes the shading on the one friend’s face. In our analogy, this is where the agency has added missing data that they had and corrected inconsistencies. The data is improved and, some would say, the snapshot of the friends has improved.

Is the snapshot now perfect? That depends on how it is rated. If a rubric were created to rate the photo, it might be perfect if the criteria focused on getting all of the subjects in the picture, clearly seeing the subjects’ faces, and having a clean look. But what if the rubric also stated that if one subject is standing, all of the subjects should be standing? What if the rubric called for four or more people in a picture? What if the rubric only gave full points if no subject is smiling?

The picture could be altered again to make the smiling friends more stoic. But, does this represent the facts of the snapshot? Should a fourth person be edited into the photo? How can a photo of a standing person be made to look like the person is not standing?

A score of 100% from the QDQ Rubric means that the data matches what the rubric describes as full points responses. A score of 100% does not mean that the data represents exactly what a client has told the agency. If enough clients have refused to answer some prompts and a HMIS user has correctly entered Client Refused into the database for these responses, the QDQ score will likely not receive full points and the QDQ score will likely be below 100%. There is no reason to change the data to get a better QDQ score. Quality data comes from correctly representing the responses from clients, not matching the QDQ rubric.

A QDQ score can be a guide to agencies. It can help an agency assess why some data is missing, or how data can be better collected in the future. A narrative added to a QDQ Score submission can give a better picture about what the scores mean.

Minnesota’s data quality has improved with the QDQ process. For individual providers and agencies, the scores might, sometimes, be lower than the agency had hoped for, but the scores do not ever need to be perfect.

For more details on the QDQ Rubric, click here.