The QDQ Monitoring Plan

CoC Coordinators and state funding program partners in Minnesota have begun monitoring the results of the HMIS Quarterly Data Quality (QDQ) initiative. Each quarterly cycle, the monitoring partners will choose one of the three score areas for focus (Completeness, Consistency and Accuracy, and Timeliness), a category of project types in HMIS for focus (ES, TH, PSH, etc.), and high and low percentage thresholds for the quarter as well.

QDQ Monitoring Supports Options

The monitoring partners want to ensure that all HMIS-participating agencies have the resources they need to maintain a high level of data quality. To that end, there are 3 action options available for providers scoring below the determined threshold for any given quarter:

Option 1: Complete and submit a Data Quality Development Plan to your monitoring partner. This option is meant for projects who have identified their data quality barriers and are making steps to address them.

Option 2: Complete a Data Quality Refresher Training provided by ICA MN.* This is a series of curated content resources, focusing on building an understanding of the QDQ category (Completeness, Consistency and Accuracy, or Timeliness), and offering tips for solving common data entry dilemmas. This option is meant for projects who may be unsure about how to identify or address their data quality barriers and are interested in deeper HMIS training.

Option 3: Schedule a meeting with your CoC Coordinator and project funder. This option is meant to give projects experiencing significant barriers to maintaining data quality the opportunity to work directly with monitoring partners on putting needed supports in place.

*The training content for Option 2 can be found below on this page! If you were contacted by a monitoring partner about your QDQ score and invited to pick one of these options, and you chose Option 1 or Option 3, please refer to that email for next steps.

QDQ Refresher Training Content:

Click on the category title of each section below to expand and to complete the refresher training steps!

Completeness

To complete the refresher training for Completeness, please go through all of the steps in this section:

Step 1 - DEFINE: What is Completeness?

Completeness ensures that all the appropriate and relevant data that agencies or funders need is being fully collected and recorded, and that a community can accurately describe both its clients, and the full scope of services provided to those clients accessing services.

Step 2 - EXAMINE: Identify Your Completeness Scores on the QDQ Monitoring Report

Start by re-visiting your provider's most recent QDQ Monitoring report in ART, which gave you the scores for Completeness that you submitted to the QDQ Data Portal. Open the handout below for instructions on how to run and examine your QDQ report.

HANDOUT - How to Identify Your Completeness Errors - CLICK HERE

Step 3 - LEARN: What To Collect and Record for Complete Data in HMIS

A.) Next, please watch all of the videos listed in the table below that apply to your specific provider's Completeness errors, located on HMIS MN's user training content platform Moodle.* To login to Moodle: CLICK HERE

*If you do not yet have a Moodle account setup to access User Training materials, please email the MN Helpdesk at MNHMIS@icalliances.org to get setup with an account.

B.) Next, open and read the handout linked below, which will help you identify which data elements are required for data collection for all programs using HMIS in Minnesota (Universal Data Elements), and will provide you with a brief explanation of why these data elements are required.

HANDOUT - Data Collection: What to Collect and Why - CLICK HERE

For additional details on data elements, including data collection instructions, click here to view to the Minnesota HMIS Data Standards Guide.

C.) Minnesota's HMIS Knowledge Base provides you with instructional articles on a wide variety of the most common QDQ Data Corrections. The HMIS Knowledge Base is a great page to bookmark so that you can return to it quickly when looking for an answer to your data clean-up questions.

LINK - Take a look now at the Data Corrections Knowledge Base articles - CLICK HERE

Step 4 - REFLECT: Assess What You've Learned About Completeness

Finally, open the handout linked below to check your understanding and to test your knowledge from this Completeness refresher training. You can check your work by using the answer key on the last page of the document.

HANDOUT - Completeness Training: Quiz and Provider Self-Reflection - CLICK HERE

Step 5 - COMPLETION: Communicate to Your Monitoring Partner

IMPORTANT: Please respond to the email that was first sent to your agency by your QDQ monitoring partner (CoC Coordinator, MN State funding program partner, etc.) and confirm to them that you have finished the Completeness refresher training above.

Consistency & Accuracy

To complete the refresher training for Consistency and Accuracy, please go through all of the steps in this section:

Step 1 - DEFINE: What is Consistency and Accuracy?

Consistency and Accuracy work together and are scored together on the QDQ Scoring Rubric. Consistency means that all aspects of a client’s profile and assessment data need to “agree with” each other and should have no contradictions among the data. Accuracy means that any data within the system needs to accurately reflects what is currently true in reality, as provided by the client and as documented in the client’s file.

Step 2 - EXAMINE: Identify Your Consistency & Accuracy Scores on the QDQ Monitoring Report

Start by re-visiting your provider's most recent QDQ Monitoring report in ART, which gave you the scores for Consistency and Accuracy that you submitted to the QDQ Data Portal. Open the handout below for instructions on how to run and examine your QDQ report.

HANDOUT - How to Identify Your Consistency & Accuracy Errors - CLICK HERE

Step 3 - LEARN: How to Record Consistent & Accurate Data in HMIS

A.) Another word we often use to talk about data that is “consistent” is to say it is “congruent.” Congruent data means that it is “in agreement or in harmony” - and if data does not agree, we call it incongruent. The QDQ Scoring Rubric gives us several examples of how data could be incongruent in HMIS. Please study the table below to know the most common examples to watch out for:

B.) Minnesota's HMIS Knowledge Base provides you with instructional articles on a wide variety of the most common QDQ Data Corrections. The HMIS Knowledge Base is a great page to bookmark so that you can return to it quickly when looking for an answer to your data clean-up questions. CLICK HERE

Check out the links below to Knowledge Base articles dealing with the most common inconsistent, or incongruent, data errors that you may see on your QDQ report:

HUD Disability sub-assessments: HERE
Release of Information (ROI) Documentation: HERE
Project Start and End Dates: HERE

C.) Recall that accuracy means that the data within the system needs to accurately reflects what is true in reality, as provided by the client, and as documented in the client’s file. This metric is not as easy to monitor as the others, since it requires checks between what the client has told the intake staff at your agency, and what is written down on paper and/or entered into HMIS.

Brainstorm: Take some time now to think about processes or procedures you could put in place at your agency to regularly monitor that the data being entered by your staff is accurate based on what was reported by your clients.

Step 4 - REFLECT: Assess What You've Learned About Consistency & Accuracy

Finally, open the handout linked below to check your understanding and to test your knowledge from this Consistency and Accuracy refresher training. You can check your work by using the answer key on the last page of the document.

HANDOUT - Consistency & Accuracy Training: Quiz and Provider Self-Reflection - CLICK HERE

Step 5 - COMPLETION: Communicate to Your Monitoring Partner

Timeliness

To complete the refresher training for Timeliness, please go through all of the steps in this section:

Step 1 - DEFINE: What is Timeliness?

The Department of Housing and Urban Development (HUD) states that Timeliness is how long it takes for data to be entered into HMIS once it is collected from the client. It’s easiest to think of Timeliness as a stopwatch. Clients who enter your program provider are given an Entry Date. This is, essentially, the starting of the stopwatch. Timeliness is measured by the time the stopwatch starts ticking to the time a user’s fingers hit the keyboard to enter data. Note that Timeliness is scored only for Entry/Exit records created during the quarter for which you’ve run the QDQ report.

Step 2 - LEARN: The QDQ Scoring Rubric & HUD Timeliness Guidelines

A.) The Timeliness scoring rubric recently went through some adjustment and fine-tuning, when it was reviewed by a group of MN HMIS end users in the summer of 2020. At that time, the changes made to the Timeliness scoring category were:

Timeliness is no longer part of the Total QDQ score.
Timeliness is based on a new formula that considers project type and the general scores from the first two quarters of QDQ.

Please open and view this Timeliness Quick Reference Guide to view the Scoring Rubric on page 2 - CLICK HERE

B.) HUD has compiled requirements and recommended strategies for data collection and reporting in HMIS. Open the following HUD Data Quality Management Program resource and read the sections relating to Data Timeliness on the pages listed below - CLICK HERE

Page 20 - HUD Timeliness Requirements by Project Type

Page 36 - Strategies on How to Address Timeliness Issues

Step 3 - EXAMINE: Questions for Agency Reflection around Data Entry Timeliness

Take a look at your provider's most recent QDQ Monitoring report in ART, which gave you the scores for Timeliness that you submitted to the QDQ Data Portal. Since Timeliness cannot be fixed in the same method as other QDQ scores can be, it is worth examining how the score was earned, and what can be improved upon for the next quarter. Please consider the following questions. Write down your answers and share with your team at your agency:

Is it possible for HMIS users at my agency to do same-day or next day data entry? Can it be done in five days?
What obstacles exist in our daily routines that are preventing timely entry of data? Do we have the right forms? Do we have people trained?
What role can management play in improving a Timeliness score?

Step 4 - QUIZ: Assess What You've Learned About Timeliness

Read the the four questions below and try to pick the correct answers to test your learning about Timeliness:

A.) Which of these statements is true?

I can improve my Timeliness score by using Back Date mode to re-enter my data.
Timeliness is based on the time of the client’s entry in a program to when I enter data in HMIS.
Timeliness is scored the same across all project types.

B.) What are some strategies for improving Timeliness scores? (Choose all that apply)

Same-day entry of data.
Live data entry as one speaks to a client.
Using Back Date mode to enter data.
Altering work schedules to allow for timely data entry.

C.) Which one of these does a low Timeliness score indicate?

It indicates poor data entry.
It indicates data entry that is not done at all.
It indicates data entry that is done slower than the QDQ rubric suggests is important to high data quality.
It indicates a weak comparison with other project types.

D.) Timeliness is scored only for Entry/Exit records created during the quarter for which the QDQ report has been run.