In today’s world of swift and rapid change, knowledge is power. Just a few short weeks after the presentation of the preliminary results of the FoQ study at the ARF’s 2009 Annual Re:Think Conference, the first topical paper on Overlap and Duplication in online panels has been completed. This paper, authored by Bob Walker, President of Surveys & Forecasts, LLC and Joel Rubinson, Chief Research Officer and Dr. Raymond Pettit, Senior Vice President at the ARF, sheds definitive light on this critical industry issue.
Following the careful participation and vetting processes established at the outset of this project, this paper is currently under review by the ORQC leadership, committees, and supporters of the project. Following that, it will be released, initially, to ARF members. The paper will then be available as part of an Industry Report later in 2009.
The initial findings on duplication and overlap in online panels introduce a multidimensional model of the influences on survey response quality. The analysis found that although overlap (when a person is a member of more than one online panel) exists, that fact by itself is not a severe problem. But, it does set the stage for duplication (a person taking the same survey more than once) which does have an impact on survey response quality. Since the FoQ study was done across a broad array of panels, the results represent robust empirical evidence.
In short, being a member of multiple panels does not mean that data quality will automatically suffer. Other factors have much more of an impact on the quality of responses to surveys. However, duplication is something that must be avoided. The ORQC’s Industry Solutions and Metrics committees will be working intensely on studying this issue and reviewing the FoQ results to form principles, recommendations and guidelines for the industry.
The first Knowledge Brief will be followed by an extensive paper on Survey Response Quality and one on Inter-Study Comparability and Benchmarking. In total 6 topic-based Knowledge Briefs are planned and will become the core content for the full report, entitled: Foundations and Principles of Online Data Quality.
The next meeting of the ORQC is set for June 9th at the Advertising Research Foundation. More results of this important study will be shared with ARF members at that time.
