QDQ: What Happens with My Scores?
So you’ve taken part in the Quarterly Data Quality process again. Thank you! But what happens after you submit your scores? How are they used? Who sees them and what do they mean?
How is the Quarterly Data Quality Process Going?
As you may or may not know, the Quarterly Data Quality (QDQ) process has been in place for a little over a year now in Minnesota! It was January of 2020 when ICA staff hosted in-person and virtual trainings on how the QDQ process would work. In general, the process has remained the same since then:
Agencies run a special QDQ report that ‘scores’ their providers’ HMIS data quality, based on the extent to which the HUD and MN universal data elements meet criteria for (a.) Accuracy and Consistency, (b.) Completeness, and (c.) Timeliness.
Next, agencies have time to correct DQ issues and then re-run the report, with support from ICA in HMIS User Groups and via the HMIS MN Helpdesk during that time.
Finally, agencies then submit a form indicating they reviewed the QDQ report for their providers and then documenting their scores.
There have been a few adjustments to the scoring rubric for Timeliness, but otherwise the process has remained consistent up until this point.
Click here to see the Scoring Rubric.
What’s New with QDQ This Year?
So, what happens after the scores are submitted? Do they just get filed away somewhere never to be seen again? The short answer is No!
Going forward, Continuum of Care (CoC) leadership and state funding partners will be briefed on agencies’ participation and progress every quarter. These partners monitor agency participation and scores to ensure agencies’ successes are celebrated and that they have the support and resources needed for high-quality HMIS data entry.
CoC Coordinators and state program representatives have agreed to a one-year monitoring pilot in 2021, where responsibility for data quality review, provider communication, and targeted support and recognition is shared among them and supported by ICA. Click here to review that monitoring plan.
Each quarter, monitoring partners will gather to review scores specifically for the following project types (see schedule below):
January – March:
Emergency Shelter and Transitional Housing providers
April – June:
Homelessness Prevention and Rapid Rehousing providers
July – September:
Street Outreach, Services Only, and Day Shelter providers
October – December:
Permanent Supportive Housing, Permanent Housing with Services, and Other Permanent Housing providers
One score category will be chosen each quarter for focus among the categories of (a.) Accuracy and Consistency, (b.) Completeness, and (c.) Timeliness. Additionally, a high score threshold and a low score threshold will be determined for that score category. Providers with a score above the high score threshold will be contacted for recognition of their work. Providers with a score below the low score threshold will also be contacted. They will be invited to complete refresher training, to write an improvement plan, and/or work with their CoC Coordinator or funder on performance improvement strategies.
Can You Give Me an Example?
Let’s say the scores for a provider are as follows:
63% Completeness
89% Consistency & Accuracy
96% Timeliness
The monitoring partners have decided to focus on the Completeness category this quarter. They have also decided that the high score threshold is 95% and the low score threshold is 65%. Based on the Completeness score for the provider above (63%), the monitoring partners would be following up with an email to the provider about their score being below the 65% threshold. The provider would then request either (1) training to help improve the score, (2) the option to write up an improvement plan, and/or (3) request a 1-on-1 meeting with their monitoring partners.
If, in the example above, Timeliness was chosen as the focus for the quarter, and the thresholds were the same, this provider would be getting an email of recognition, since their score is above the 95% threshold.
If Consistency & Accuracy were chosen, the provider above would not be contacted at all - since their score (89%) was neither below nor above the high and low thresholds.
A QDQ Thank You
In this QDQ Monitoring Plan, every quarter the project types of focus will change. Every quarter the category focus and the thresholds may change. Whether your agency is contacted directly or not, every quarter, we at ICA MN are so very appreciative of all your hard work and participation in the quarterly data clean-up process. This DQ work makes a big difference to the quality of service we can all deliver to the clients we serve in Minnesota, and we thank you!