Description of the Survey
The Survey of Household Economics and Decisionmaking was fielded from October 11 through October 24 of 2019. This was the seventh year of the survey, conducted annually in the fourth quarter of each year since 2013.52 Staff of the Federal Reserve Board wrote the survey questions in consultation with other Federal Reserve System staff, outside academics, and professional survey experts.
Ipsos, a private consumer research firm, administered the survey using its KnowledgePanel, a nationally representative probability-based online panel. Ipsos selected respondents for KnowledgePanel based on address-based sampling (ABS).53 SHED respondents were then selected from this panel.
Survey Participation
Participation in the 2019 SHED depended on several separate decisions made by respondents. First, they agreed to participate in Ipsos' KnowledgePanel and then they completed an initial demographic profile survey. According to Ipsos, 12.2 percent of individuals contacted to join KnowledgePanel agreed to join (study-specific recruitment rate), and 62.1 percent of recruited participants completed the initial profile survey and became a panel member (study-specific profile rate). Finally, selected panel members agreed to complete the 2019 SHED.
Of the 19,994 panel members contacted to take the 2019 SHED, 12,238 (excluding breakoffs) participated, yielding a final-stage completion rate of 61.2 percent. All the stages taken together, the cumulative response rate was 4.6 percent. The final sample used in the report included 12,173 respondents.54
Targeted Outreach and Incentives
To increase survey participation and completion among hard-to-reach demographic groups, Board staff and Ipsos utilized a targeted communication plan with monetary incentives. The target groups—young adults ages 18 to 29, adults with less than a high school degree, and those who are a race or ethnicity other than white and non-Hispanic—received additional email reminders and text messages during the field period, as well as additional monetary incentives.
All respondents to the survey received a $5 incentive payment after survey completion. Respondents in the target groups received a higher $15 incentive. These targeted individuals also received follow-up emails during the field period to encourage completion. Additionally, the incentives offered to some targeted individuals increased modestly to $20 during the field period to increase the incentive for completion.55
Final-stage completion rates for all population groups increased in 2019 relative to 2018. As was the case in 2018 when the targeted incentive plan was first introduced, response rates in 2019 were similar between the targeted and non-targeted groups. Hence, this targeted incentive system reduced the differences in response rates across subpopulations and improved the quality of the final data.
Survey Questionnaire
On average, the survey in 2019 took respondents 19 minutes (median time) to complete, two minutes shorter than the previous survey. The shorter interview length reflected a continued effort to lessen respondent burden. The number of questions was reduced and the length of the questionnaire was shortened.
Because one motivation for the survey was to understand where there may be vulnerabilities or weaknesses in the economy, one priority in selecting questions was to provide information on the financial experiences and challenges among low- and moderate-income populations. The questions were intended to complement and augment the base of knowledge from other data sources, including the Board's Survey of Consumer Finances. In addition, some questions from other surveys were included to allow direct comparisons across datasets.56 The full survey questionnaire can be found in appendix A of the appendixes to this report.
Survey Mode
While the sample was drawn using probability-based sampling methods, the SHED was administered to respondents entirely online. Online interviews are less costly than telephone or in-person interviewing, and can still be an effective way to interview a representative population.57 Ipsos' online panel offers some additional benefits. Their panel allows the same respondents to be re-interviewed in subsequent surveys with relative ease, as they can be easily contacted for several years.
Furthermore, internet panel surveys have numerous existing data points on respondents from previously administered surveys, including detailed demographic and economic information. This allows for the inclusion of additional information on respondents without increasing respondent burden. The respondent burdens are further reduced by automatically skipping irrelevant questions based on responses to previous answers.
The "digital divide" and other differences in internet usage could bias participation in online surveys, so recruited panel members who did not have a computer or internet access were provided with a laptop and access to the internet to complete the surveys. Even so, individuals who complete an online survey may have greater comfort or familiarity with the internet and technology than the overall adult population.
Sampling and Weighting
The SHED sample was designed to be representative of adults age 18 and older living in the United States. Because the sample size was large enough to obtain sufficient coverage of low-income populations without an oversample, unlike previous years the 2019 survey did not include a low-income oversample. This change improved the overall representativeness of the sample and reduced the importance of survey weights in analyses of the results.
The Ipsos methodology for selecting a general population sample from KnowledgePanel ensured that the resulting sample behaved as an equal probability of selection method (EPSEM) sample. This methodology started by weighting the entire KnowledgePanel to the benchmarks in the latest March supplement of the Current Population Survey along several geo-demographic dimensions. This way, the weighted distribution of the KnowledgePanel matched that of U.S. adults. The geo-demographic dimensions used for weighting the entire KnowledgePanel included gender, age, race, ethnicity, education, census region, household income, homeownership status, and metropolitan area status.
Using the above weights as the measure of size (MOS) for each panel member, in the next step a probability proportional to size (PPS) procedure was used to select study specific samples. This methodology was designed to produce a sample with weights close to one, thereby reducing the reliance on post-stratification weights for obtaining a representative sample.
After the survey collection was complete, statisticians at Ipsos adjusted weights in a post-strati-
fication process that corrected for any survey non-response as well as any non-coverage or under- and over-sampling in the study design. The following variables were used for the adjustment of weights for this study: age, gender, race, ethnicity, census region, residence in a metropolitan area, education, and household income. Demographic and geographic distributions for the noninstitutionalized, civilian population age 18 and older from the March Current Population Survey were the benchmarks in this adjustment.
Although weights allow the sample population to match the U.S. population (not in the military or in institutions, such as prisons or nursing homes) based on observable characteristics, similar to all survey methods, it remains possible that non-coverage, non-response, or occasional disparities among recruited panel members result in differences between the sample population and the U.S. population. For example, address-based sampling likely misses homeless populations, and non-English speakers may not participate in surveys conducted in English.58
Despite an effort to select the 2019 SHED sample such that the unweighted distribution of the sample more closely mirrored that of the U.S. adult population, the results indicate that weights remain necessary to accurately reflect the composition of the U.S. population. Consequently, all results presented in this report utilize the post-stratification weights produced by Ipsos for use with the survey.
April 2020 Supplemental Survey
As discussed in box 1 and in the "Financial Repercussions from COVID-19" section of this report, the Federal Reserve Board also conducted a survey in April 2020 to understand recent financial conditions and how circumstances for families have changed since the main survey was fielded in the fall. This survey was also fielded using the Ipsos KnowledgePanel. As such, the same sampling and weighting approaches apply to this survey as applied for the main SHED survey conducted in the fall. However, the sample size, survey length, and incentive structure differs from the main SHED survey. The respondents to the April supplement are drawn from the entire KnowledgePanel and typically did not also complete the fall survey.
The April 2020 survey included 13 questions and was fielded from April 3 through April 6. Respondents to the survey receive a small incentive from Ipsos for their participation, but because of the short length of the survey respondents are not offered additional incentives for completion.
Of the 2,853 panel members contacted to take the survey, 1,030 participated, yielding a final-stage completion rate of 36.1 percent. All the stages taken together, the cumulative response rate is 2.5 percent. The final stage completion rate is lower than it was for the fall 2019 SHED because this survey was conducted over a single weekend in order to obtain timely results. As is the case for the SHED survey that was fielded in fall 2019, after the survey collection is complete, statisticians at Ipsos adjust weights in a post-stratification process that corrects for any survey non-response as well as any non-coverage or under- and over-sampling in the study design.
References
52. Data and reports of survey findings from all past years
are available at https://www.federalreserve.gov/consumerscommunities/shed.htm. Return to text
53. Prior to 2009, respondents were also recruited using random-digit dialing. Return to text
54. Of the 12,238 respondents who completed the survey, 65 were excluded from the analysis in this report due to either leaving responses to a large number of questions missing, completing the survey too quickly, or both. Return to text
55. All targeted adults received an email encouraging completion between three and six days into the field period. Targeted adults under age 60 received a follow-up email between 5 and 8 days into the field period mentioning the increased incentive payment, as well as a third reminder between 8 and 10 days into the field period. Return to text
56. For a comparison of results to select overlapping questions from the SHED and Census Bureau surveys, see Jeff Larrimore, Maximilian Schmeiser, and Sebastian Devlin-Foltz, "Should You Trust Things You Hear Online? Comparing SHED and Census Bureau Survey Results," Finance and Economics Discussion Series Notes (Washington: Board of Governors of the Federal Reserve System, October 15, 2015). Return to text
57. David S. Yeager et al., "Comparing the Accuracy of RDD Telephone Surveys and Internet Surveys Conducted with Probability and Non-Probability Samples," Public Opinion Quarterly 75, no. 4 (2011): 709–47. Return to text
58. For example, while the survey was weighted to match the race and ethnicity of the entire U.S. adult population, there is evidence that the Hispanic population in the survey were somewhat more likely to speak English at home than the overall Hispanic population in the United States. Sixty-four percent of Hispanic adults who responded to the SHED speak Spanish at home, versus 71 percent of the overall Hispanic population who did so based on the 2018 American Community Survey. See table B16006 at https://data.census.gov. See the Report on the Economic Well-Being of U.S. Households in 2017 for a comparison of results to select questions administered in Spanish and English. Return to text