Appendix 1: Technical Appendix on Survey Methodology

 

In order to create a nationally representative probability-based sample, GfK's KnowledgePanel ^® has selected respondents based on both random digit dialing and address-based sampling (ABS). Since 2009, new respondents have been recruited using ABS. To recruit respondents, GfK sends out mailings to a random selection of residential postal addresses. Out of 100 mailings, approximately 14 households respond to GfK and express an interest in joining the panel. Of those who express an interest in joining, around 64 percent complete the process and become members of the panel.¹⁸ If the person contacted is interested in participating but does not have a computer or Internet access, GfK provides him or her with a laptop and Internet. Panel respondents are continuously lost to attrition and added to replenish the panel, so the recruitment rate and enrollment rate may vary over time.

For the 2014 mobile survey, a total of 6,892 KnowledgePanel ^® members received e-mail invitations to complete the survey, including both the primary sample and a rural oversample. The primary sample included a random selection of 2,308 out of the 2,657 KnowledgePanel ^® respondents who participated in the Board's 2013 mobile survey and an additional 2,657 randomly selected KnowledgePanel ^® respondents who did not participate in the Board's previous survey. (See table 1 in main text.) From these two components of the primary sample, a total of 2,925 people (excluding breakoffs) responded to the e-mail request to participate and completed the survey, yielding a final stage completion rate of 58.9 percent. The recruitment rate for the primary sample, reported by GfK, was 14.6 percent and the profile rate was 64.0 percent, for a cumulative response rate of 5.5 percent. Answers from these respondents were used to compute statistics presented in the main body of the report, as well as in the tables in appendix 3.

The 2014 survey also included an oversample of respondents residing in rural areas as defined by Rural Urban Commuting Area (RUCA) codes.¹⁹ Respondents were selected for inclusion in the rural oversample if the ZIP code for their residence was classified as being in RUCA codes 7.0-10.3. Because RUCA codes are assigned at the Census tract level, ZIP codes meeting this criteria were identified based on the crosswalk available from the Center for Rural Health at the University of North Dakota. (See ruralhealth.und.edu/ruca .) All members of KnowledgePanel ^® residing in rural areas based on this definition, but not already included in one of the other two samples, received an invitation to respond to the survey. Of these additional 1,927 KnowledgePanel ^® members who received invitations as a part of the rural oversample, 1,298 people (excluding breakoffs) responded to the e-mail request to participate and completed the survey, yielding a final stage completion rate of 67.4 percent for the oversample. The recruitment rate for the rural oversample, reported by GfK, was 14.4 percent and the profile rate was 63.3 percent, for a cumulative response rate of 6.1 percent. Answers from these respondents were combined with answers from the other two samples and used to compute statistics presented in box 1 of the report.

To enhance the completion rate, GfK sent e-mail reminders to non-responders on days three and ten of the field period. GfK maintains an ongoing modest incentive program to encourage KnowledgePanel ^® members to participate. Incentives take the form of raffles and lotteries with cash and other prizes. KnowlegePanel ^® members who were a part of the rural oversample in the 2014 survey were offered an additional $5 incentive for completion of the survey.

Significant resources and infrastructure are devoted to the recruitment process for the KnowledgePanel ^® so that the resulting panel can properly represent the adult population of the United States. Consequently, the raw distribution of KnowledgePanel ^® mirrors that of the U.S. adults fairly closely, baring occasional disparities that may emerge for certain subgroups due to differential attrition rates among recruited panel members.

The selection methodology for general population samples from the KnowledgePanel ^® ensures that the resulting samples behave as an equal probability of selection method (EPSEM) samples. This methodology starts by weighting the entire KnowledgePanel ^® to the benchmarks secured from the latest March supplement of the Current Population Survey (CPS) along several dimensions. This way, the weighted distribution of the KnowledgePanel ^® matches that of the U.S. adults. Typically, the geo-demographic dimensions used for weighting the entire KnowledgePanel ^® include gender, age, race/ethnicity, education, Census region, household income, home ownership status, metropolitan area status, and Internet access.

Using the above weights as the measure of size (MOS) for each panel member, in the next step a probability proportional to size (PPS) procedure is used to select study specific samples. Since this survey includes a rural oversample, the departure caused by this oversample from an EPSEM design are corrected by adjusting the corresponding design weights accordingly with the CPS benchmarks serving as reference points.

Once the sample has been selected and fielded, and all the study data are collected and made final, a post-stratification process is used to adjust for any survey non-response as well as any non-coverage or under- and over-sampling resulting from the study-specific sample design. The following variables were used for the adjustment of weights for this study: gender, age, race/ethnicity, education, Census region, residence in a metropolitan area, access to the Internet, and residence in a rural area according to the definition used for the rural oversample. Demographic and geographic distributions for the non-institutionalized, civilian population ages 18 and over from the March 2014 CPS are used as benchmarks in this adjustment. For the geographic distribution of residence in a rural setting, the full set of members of KnowledgePanel ^® was used to generate the benchmark since the CPS does not provide statistics on rural status according to the criteria used to select the oversample.

Although weights allow the sample population to match the U.S. population based on observable characteristics, similar to all survey methods, it remains possible that non-coverage or non-response results in differences between the sample population and the U.S. population that are not corrected using weights.

There are several reasons that a probability-based Internet panel was selected as the method for this survey rather than an alternative survey method. The first reason is that these types of Internet surveys have been found to be representative of the population.²⁰ The second reason is that the ABS Internet panel allows the same respondents to be re-interviewed in subsequent surveys with relative ease, as they remain in the panel for several years. The third reason is that Internet panel surveys have numerous existing data points on respondents from previously administered surveys, including detailed demographic and economic information. This allows for the inclusion of additional information on respondents without increasing respondent burden. Lastly, collecting data through an ABS Internet panel survey is cost effective, and can be done relatively quickly.

There are possible questions about the extent to which results from an online survey of technology use can be interpreted as being representative of the technology use of the U.S. population. As with any survey method, Internet panels can be subject to biases resulting from undercoverage or nonresponse and, in this case, potential underrepresentation of adults who are physically or cognitively impaired or who may prefer not to use some forms of technology. Not everyone in the United States has access to the Internet and there are demographic (income, education, age) and geographic (urban and rural) differences between those who do have access and those who do not. These concerns are partially corrected by GfK providing Internet access to respondents who do not have it in order to include the portion of the population that does not have Internet access in KnowledgePanel ^®. They are further corrected by the use of post-stratification weights to ensure that the Internet usage and key demographics of the weighted sample population matches the entire U.S. population. That said, participation in this type of survey may require a certain level of skill and interest in responding online, which could limit coverage of some groups, particularly among those in the population who are less likely to use computers or the Internet. As a result, to the extent that these differences cannot be incorporated into the sample weights, technology usage among survey respondents may differ along key dimensions from that of the overall U.S. population.

 

References

Back to Top

• Other Formats

• Full Report (PDF)

Skip stay connected section

• Stay Connected

• Twitter
• YouTube
• Flickr
• RSS Feeds
• Subscribe