Quality and Reliability of PPPs
The ICP publications provide estimates of PPPs and real expenditures at the GDP level and for major components of GDP. However, there is seldom any indication on the quality of the published figures and no measures of reliability or measures of uncertainty associated with PPPs are provided. Currently researchers have no option but to consider all the published results to be of similar quality though intuition suggests that comparisons between some countries (UK and Germany for example) would be intrinsically more reliable than some others (e.g. USA and Mozambique).
Uncertainty associated with published PPPs could be due to product selection, coverage and quality of price surveys, sampling errors, quality of national accounts expenditure data, and also due to the index number methods used in the aggregation process. There have been some attempts to identify a framework to construct measures of reliability. For example, Deaton (2012) and Rao and Hajargasht (2015) consider deviations of observed prices from the predictions generated using the law of one prices as a basis for computing standard errors for PPPs. Needless to say, further research is necessary to establish an analytical framework to construct reliability measures associated with PPPs from ICP.
The FOC report makes a passing reference to reliability measures as it deals with quality assurance.
“The ICP quality assurance framework was derived from the Data Quality Assessment Framework developed by IMF, which focuses on the quality-related features of governance of statistical systems, core statistical processes and statistical products. The ICP quality assurance framework covered six topics: prerequisites of quality; assurance of integrity; methodological soundness; accuracy and reliability; serviceability; and accessibility.” (Paragraph 33, E/CN.3/2016/9, UN, 2016)
Furthermore, one of the FOC recommendations is:
“Quality assurance of resulting PPPs and measures of reliability”. ((g), Paragraph 83, E/CN.3/2016/9, UN, 2016)
The Global ICP Unit identified two strands of research in regards to ICP quality assurance framework.
Component 1. Quality of Data
- Investigate sources of data variability at each level of estimation to assess the overall quality of the results and where data quality can be improved;
- Examine variability in basic heading PPPs before and after linking and identify countries and basic headings appearing as outliers before and after linking;
- Examine variability in basic heading expenditure shares and differences in consumption estimates stemming from household surveys and national accounts; and
- Analyze effect of the countries with high variability in expenditure shares and prices on regional and global results.
Component 2. Reliability Measures for PPPs
- Establish a conceptual framework that underpins reliability measurement;
- Develop a statistical framework for the computation of standard errors or confidence intervals;
- Implement the framework and compute standard errors or confidence intervals for PPPs from the 2005, 2011 and 2017 comparisons as well as for PPP changes over time; and
- Provide guidance to users as to how these measures of reliability can be used in practice.
The Global ICP Unit and regional implementing agencies seek to resolve data quality issues and have processes in place to ensure improved data quality for the 2017 ICP cycle.