UN Banner
HomeAboutExperiences and LessonsCase StudiesRecommendationsResourcesChecklist
Home
About▶
BackgroundPurposeRationaleStructureTarget Audience
Experiences and Lessons▶
1. Project Planning and Management2. Geospatial Mapping and EA Database3. Enumeration Instruments and Tools4. Data Capture and Management5. Census Testing and Pilots6. Recruitment and Training7. Deployment and Supervision8. Enumeration and Logistics9. Quality Assurance and Risk Management10. Census Analysis and Products11. Partnerships and Collaboration12. Advocacy and Publicity13. Procurement and Financial Management14. Post-Enumeration Survey15. Alternative Approaches16. Planning Checklist
Case Studies▶
1. Project Planning and Management2. Geospatial Mapping3. Enumeration Instruments4. Data Capture5. Census Testing6. Recruitment and Training7. Deployment and Supervision8. Enumeration and Logistics9. Quality Assurance10. Census Analysis11. Partnerships12. Advocacy13. Procurement14. Post-Enumeration Survey15. Alternative Approaches16. Planning Checklist
Recommendations
Resources
Checklist
Skip to main content

Introduction​

Quality assurance is not a separate phase but an ongoing commitment to monitor, evaluate and assure data quality throughout the census lifecycle. In a digital census, technology provides sophisticated tools for real-time monitoring, rigorous evaluation, and proactive risk management. Census processes should show methodological soundness and adherence to professional and internationally agreed standards and efficiency, relevance of data, completeness, accuracy of data, timeliness of information, accessibility, data coherence, and comparisons across space which include data validation against other sources, and basic demographic analysis based on the knowledge of an area.

The UN handbook on census management emphasises integrating risk planning into every phase of the census cycle.

A well-executed digital census combines technology with proven quality assurance methods to ensure accurate, reliable, and secure data collection. This chapter outlines a comprehensive approach to quality assurance, monitoring, evaluation and risk management emphasizing the integration of digital methodologies.

Comparison of quality assurance, monitoring, evaluation and risk management during the non-digital versus the digital era​

As part of a digital census, technology offers powerful tools for enhancing quality assurance and validation throughout the census process. Electronic devices and other technologies impact the census process positively by making the enumeration faster and easier to monitor for quality and allowing for use of multiple data collection approaches to save costs.

Digital systems can be programmed to automatically detect various types of errors or inconsistencies in responses e.g., discrepancy between age and marital status, outliers and unusual values such as extremely high or low income, or duplicate entries i.e., multiple entries for the same person. Data validation rules can be defined within the Computer-Assisted Personal Interviewing (CAPI) application to prevent errors at the point of data collection. Data processing software may be applied to flag errors during data cleaning stages.

Machine learning algorithms can be used to enhance data quality. For example, anomaly detection models can identify complex patterns and anomalies in the data that may not be apparent through traditional methods. Such algorithms can predict potential errors and target quality assurance efforts more effectively. Classification models can be used to predict the likelihood of errors based on respondent characteristics and can also be used to improve the accuracy of imputation for missing data by estimating plausible values.

In a digital census, digital dashboards can be used to provide real-time data quality monitoring. Digital dashboards enable census officials to view data quality indicators and thereby:

  • Track error rates and identify areas where data quality may be compromised.

  • Monitor enumerator performance and identify potential training needs.

  • Assess the impact of interventions and quality improvement efforts.

Furthermore, digital systems automatically generate para-data which was not available in a non-digital era. Para-data is data about the census process itself. Para-data includes timestamps of data collection events e.g., start and end times of interviews, enumerator behavior such as time spent on specific questions and patterns of skipping questions, and device usage patterns such as battery life and connectivity. Analyzing para-data can provide valuable insights into potential quality issues and inform process improvements such as by identifying enumerators who may need additional training, detecting potential fraud or falsification of data, or optimisation of the data collection process for efficiency and accuracy.

Risks and their impact to a digital census are different and can be more costly if not mitigated compared to a non-digital census.

Considerations for quality assurance, monitoring, evaluation and risk management for a digital census​

To ensure comprehensive quality assurance, monitoring, evaluation and risk management, the following key considerations must be in place:  

  • The quality assurance and improvement system should be developed at the planning stage as part of the overall census programme and integrated with all other census phase plans, schedules, and procedures.  

  • There should be good, well-documented internal quality monitoring mechanisms.

  • For monitoring and evaluation in a digital census, real-time dashboards must be adopted. Use electronic dashboards to track data collection progress and identify anomalies.  For evaluation, conduct follow-up surveys (post-enumeration surveys) to assess data accuracy and coverage.

  • Key data quality indicators should be agreed upon at the start. Examples of quality indicators that may be displayed by the dashboard include error rates (e.g., percentage of questionnaires with inconsistencies), inconsistency rates (e.g., percentage of conflicting responses for key variables), the distribution of responses for specific questions, the number of edits/flags raised per enumerator/area, completion rates (e.g., percentage of households enumerated), and time taken per interview.

  • Determine and test the software or technologies to be used at the start. Here are some examples:

    • Statistical software packages (e.g., Stata, R, SPSS) for calculating and visualizing quality indicators.

    • Database management systems (e.g., PostgreSQL) for storing and querying data.

  • Despite digital advancements, physical field supervision remains crucial to address unforeseen issues and must be factored into the quality assurance and improvement plan.

  • There are inherent risks in digital platforms, therefore a comprehensive risk management plan based on assessments done to identify potential risks must be developed in the pre-enumeration phase for each of the census components.

  • In conducting a risk assessment, evaluate the likelihood and impact of each identified potential risk using tools like risk matrices and scenario planning to prioritise them.

  • Put in place a risk monitoring and review system by setting up a governance structure to regularly review and update risk strategies and establish a risk register to track identified risks and mitigation actions.

Detailed guidance on emergency preparedness, risk management and contingency planning in the UN P&R, Rev 4, para 2.125 to 2.157.