July 16, 2024

The Challenges Facing Economic Measurement and Creative Solutions

Governor Adriana D. Kugler

At the 21st Annual Economic Measurement Seminar, National Association for Business Economics Foundation, Washington, D.C.

Thank you for your generous introduction, Ellen. I am delighted to be here with the National Association for Business Economics (NABE), and, in particular, I am pleased to be speaking at a conference covering an issue that is close to my heart and on which I have spent many years working: economic measurement.

When Federal Reserve officials tell audiences that their judgments are data dependent, some skeptics perhaps presume that monetary policy is already on a path set in stone. But most in this room likely know what I mean when I talk about data dependence. I am a member of the Federal Open Market Committee (FOMC), which, of course, pursues a dual mandate of maximum employment and stable prices.1 When I say I am data dependent, that means I am considering the totality of the data—the full range of economic indicators that provide a sense of where the labor market, economic activity, financial conditions, and inflation have been and where they might be going. Policymakers must have high-quality and accurate data to understand the economy and set the correct policy.

The truth is that it is not just the Fed that needs data. Consumers, businesses, investors, and others have access to more information than ever before when making decisions. It is incumbent on economists, private- and public-sector data collectors, and others to ensure that available data are carefully collected, accurately measured, and clearly presented, and that data collection and measurement efforts are further enhanced and continue to improve.

To be sure, data collection and economic measurement can be challenging, and different types of data face pros and cons, which is why I take an expansive approach to using data, as I will explain a bit later. But I will begin by highlighting a few challenges to economic measurement. I will then provide some examples of how those challenges might be addressed. I will also provide examples of how nontraditional data generated by the private sector can help provide additional angles from which to view aspects of the economy that may not be clear in data produced by government statistical agencies. Finally, I will offer some examples of how, in recent years, our statistical agencies have adapted and innovated, sometimes by incorporating private-sector data to address specific measurement challenges I have in mind. To be clear, my interest in nontraditional data is not a critique of the statistical agencies. To the contrary, official data are critically important to policymakers, researchers, and the public. Rather, I view public and private data as being complementary and helping to provide a more complete picture of the economy.

Importance of Data
Economic measurement, the task some of you contribute to every day, is at the core of real-time policy analysis and forecasting. We at the Board of Governors rely on a broad array of data produced by both government and the private sector. And we carefully scrutinize every important economic release and are very familiar with the methodological details of those reports. Suffice it to say, we care a lot about economic measurement at the Fed.

And the Fed itself produces many important data series, including two principal federal economic indicators: the Industrial Production and Capacity Utilization report and the Consumer Credit statistical releases.2 In addition, our website features numerous other data products that look at bank assets, liabilities, and structure; monetary aggregates; international finance; business finance; and household finance—including data on the level, share, and composition of wealth for households at different points in the wealth and income distribution, as well as for different demographic groups.3 The Board also conducts separate surveys of consumers and household economic decision making and surveys of loan officers and credit officers, well known as the SLOOS and the SCOOS. Beyond the Federal Reserve Board, each of the Federal Reserve Banks engages in measurement and data production of various kinds, including closely watched surveys of firms and households. For example, the New York Fed conducts a survey of consumer expectations, which was just released last week, the Atlanta Fed tracks wage data, the Richmond Fed surveys firms on price-setting behavior, among other things, and several Reserve Banks publish alternative measures of inflation and reports on factory and business activity in their regions.

Challenges with Measurement
Of course, as one reviews various sources of data, one can see that there are some common measurement challenges. I will mention a few. These are not new, nor is my list exhaustive. My intent here is to focus on challenges that highlight the tradeoffs in using public and private data and to show how these sources of data complement each other.

The first challenge I will identify is that traditional measurement approaches sometimes struggle to track rapidly changing economic developments. This was the case early in the pandemic and is often true at turning points in the business cycle. Of course, it is at those very moments when policymakers, and the public, are most critically interested in how the economy is faring. The reasons for this lag are well known. Statistical agencies can only survey households and businesses every so often, and it takes time to compile and publish high-quality statistics. However, financial markets and business decisions move quickly. Reports produced on a monthly or quarterly basis, often with a multi-week lag, may not be available with sufficient frequency to inform real-time decisions. Also, some indicators, by design, take on new market information slowly. A recent and relevant example is housing services inflation, as measured as part of the consumer price index (CPI) and the personal consumption expenditures price index. Both rely heavily on slowly changing leasing agreements that only adjust to market conditions over many months. Business entries and exits, sometimes called births and deaths, are similar in that they take time to appear in surveys that underlie important statistical products. I will touch on that a bit more later.

A second challenge is that many statistical reports were created decades ago and may not be focused on newer or growing sectors of the economy. For example, there are monthly reports on factory orders, shipments, inventories, and output among the principal economic indicators. Because developments in the goods sector can matter importantly for economic fluctuations, that level of detail is indeed helpful to have. But we would benefit from the same depth of information on domestically produced services, too. While service-sector output constitutes a larger share of U.S. gross domestic product (GDP), the category has only one single dedicated quarterly report, even though much of the post-pandemic recovery requires us to understand how services consumption and employment have evolved. For example, did you know that quarterly hog and pig counts are a principal economic indicator? Yet, there is not a regular government report specifically dedicated to the gig economy. I am certainly glad we collect data for important agricultural commodities, but I think we should probably be examining the large and diverse gig economy as much as we do the pig economy. I believe the statistical agencies grapple with these issues, and it takes time to lay the groundwork and develop new data series, but this challenge is a real one. If reports are built for the past, that may mean they struggle to capture big developments today and in the future, such as the changing nature of where and how people work or the rising use of artificial intelligence (AI) technology.

Finally, there is an impediment that everyone here is acutely aware of: These challenges are made even more difficult because official surveys have seen declining survey response rates. That results in lower precision of statistical estimates and can lead to the need for further, costly surveys to collect the necessary amount of information, as one may need to conduct several rounds of data collection before having sufficiently large samples that can provide reliable estimates. Declining response rates may also be selective and pose a challenge to representing the broader population. Innovative methods for data collection and behavioral interventions to encourage survey responses can be used to address concerns and are being tried in some cases by statistical agencies.

A Data Explosion
While these challenges in traditional data that I have described may take some time to be addressed, I am encouraged by the explosion in data produced by the private sector over the past decade or so that can greatly enhance our understanding of the economy. Such data give an opportunity to measure economic developments with greater timeliness, at a higher frequency, and with more granularity. That said, those data often face their own challenges, including issues with representativeness, the lack of methodological consistency, and a short time-series history. Nevertheless, such nontraditional data can be helpful, especially when used jointly with official statistics to which these new sources can be benchmarked.

A prominent example of valuable private-sector data is employment statistics from payroll providers. Such weekly data allowed economists at the Federal Reserve Board to understand, essentially in real time, employment losses when the pandemic first took hold in the United States in March 2020. In comparison, it took until early May to get similar information from the Bureau of Labor Statistics' (BLS) employment report.4 Indeed, in trying to understand the rapidly changing economy during the pandemic, Fed economists closely followed a variety of privately produced higher-frequency data. These included reports on restaurant reservations, hotel occupancy, and airport passengers. Economists even used anonymized phone-tracking data to estimate business shutdown rates and trends in retail spending.5

Credit and debit card transaction data are another example of a helpful, private-sector tool that economists use to understand consumer behavior in various sectors or regions in close to real time. These figures provided what proved to be a pretty reliable picture of economic developments during the pandemic, well before traditional statistics—such as quarterly GDP and monthly retail sales—became available.6 The need for timely, high-frequency information is not limited to pandemics and turns in the national business cycle. For example, Federal Reserve Board economists have used credit and debit card transaction data to estimate the effects of local natural disasters almost in real time when official statistics are often not available.7

When we look at economic turning points, it is also important to consider reports on expectations and anticipated outcomes from nongovernment sources. Those include surveys of expectations of future inflation, anticipated hiring or layoffs, and consumer and business sentiment on the economy or the path of the economy. I pay close attention to these surveys because they are forward looking and help inform where behaviors by businesses and households may be trending.

Tracking supply chains is another area where private-sector sources have proved extremely valuable, especially during the COVID-19 pandemic. One important source of such data is the Institute for Supply Management's surveys of purchasing managers. While those series are well established and closely followed, Fed economists found them particularly helpful in recent years to observe supplier delivery times, order backlogs, items in short supply, and measures of inventory satisfaction. Other private-sector data give economists insight into supply chains as well, including measures of air, sea and overland freight costs; the number of waiting container ships; and the volume of railroad traffic. These data series offered additional details on the level of constraint in supply chains following the pandemic shock, alongside reports from official statistical agencies.

I will offer two additional examples of how I use nontraditional data at the Fed—one from each side of our dual mandate. To better understand the labor market, I have been watching job vacancy, quit, and layoff data closely for many years. The official government source for this information is the Job Openings and Labor Turnover Survey (JOLTS). While extremely important, JOLTS data are released with a lag of more than a month. And this survey has seen a particularly large decline in response rates.8 This instance turned out to be another example of where private data could enhance my understanding. Job search sites' data on postings are updated more frequently than the federal report and generally confirm that JOLTS measurement appears to be accurate. Similarly, JOLTS data on layoffs are lagged. That is why I also look at other figures, including unemployment insurance claims; Worker Adjustment and Retraining Notification, or WARN, notices; employment reductions from the Institute for Supply Management; anticipated layoffs from outplacement firm Challenger, Gray and Christmas; and mentions of layoffs in earnings reports and the Beige Book.

With inflation, an important component to track recently has been housing services costs, which is typically the single largest expense for U.S. households. Housing services are a big reason why the overall inflation rate remains above our 2 percent target. The official measures of housing services inflation are intended to capture overall growth of housing costs—that is, costs incurred by owners and renters—drawing from a survey of rental lease terms. But rental leases tend to change only gradually, so the official measures can significantly lag current market conditions. That is why policymakers can also rely on current market rent data, showing what landlords charge new tenants, information that is available from multiple private-sector sources. Those data can provide some early signal of where official housing inflation series are likely headed.

Here I have mentioned just a handful of private-sector data examples. But each of these helps address measurement challenges I mentioned earlier. From these data, we can gain timeliness and higher frequency, with a better read on underlying economic dynamics like market pricing. And with often low survey response rates in official data, these private-sector sources may provide another perspective on underlying economic developments.

Official Innovations
While I see great value in considering data produced outside of statistical agencies, the private sector is not the only place where innovation in economic measurement is occurring. Statistical agencies have long made use of private-sector data, so there is nothing "nontraditional" about using private-sector data to shed more light on economic measurement; in fact, it is a long tradition. I can think of several examples.

The Bureau of Economic Analysis (BEA) taps many private-sector sources in its compilation of GDP statistics, covering topics ranging from oil and gas drilling to insurance premiums.9 At the Board, the Industrial Production and Capacity Utilization report features a long list of private data sources for the output of products ranging from semiconductors to lumber.10 And our Consumer Credit statistics rely in part on data from a large credit bureau and an association of credit unions.11 I could go on, but the broader point is that our country's statistical agencies do not ignore valuable sources of economic measurement that the private sector has to offer—far from it. They make significant use of private information in combination with official surveys and administrative data. And they are actively advancing that practice further.

Economic developments in recent years have been met by a flurry of innovation and adaptation from statistical agencies, relying not only on private-sector sources but also on novel uses or production of government data. One notable product that I have mentioned in past speeches is the Census Bureau's Business Formation Statistics (BFS). First published in 2018, the BFS series was developed in collaboration with the Board's staff and relies on new business applications for Employer Identification Numbers, or EINs.12 During the early months of the pandemic, weekly data from the BFS provided a timely indicator of the initial decline in economic activity, and then later the figures likewise documented the rebound.13

Another measurement invention was created during the early months of the pandemic: the Census Bureau's high-frequency "Pulse" surveys—one each for households and businesses. Those surveys leveraged existing Census Bureau resources to provide quick-turnaround information about the rapidly evolving health and economic situations.14

The Household Pulse Survey was rapidly developed shortly after the pandemic struck the U.S., with participation from a broad set of government agencies.15 The survey provides a range of economic information about Americans, including the pandemic-induced jump in remote work. The survey has adapted over time as the health and economic situations have evolved, incorporating questions about vaccination, access to infant formula during a product shortage, inflation, and other issues. The Small Business Pulse Survey provided timely information about employment, revenue, financial conditions, and expectations for future growth, survival, and needs in the highly uncertain pandemic environment. Later iterations of the survey shed light on supply chains, as the United States grappled with input shortages and disrupted freight networks. The Small Business Pulse Survey was discontinued, but it planted the seeds for the current Business Trends and Outlook Survey, which provides a similar wealth of information about business conditions and is already the gold standard for understanding trends such as business-level implementation of AI technologies.16

The national accountants at the BEA have also shown a flair for innovation. For example, the bureau has introduced many "satellite accounts" and other special series over the years—some still in experimental form—with national account–style information about special topics, including health care, income distribution, global value chains, small business, the digital economy, and the space economy, among others.17 Thinking of special-topic national accounts is a way the statistical agencies can keep up with an ever-evolving economy. In fact, the treatment of research and development investment in current GDP methodology originated as a satellite account.18

Another example comes from the BLS and, specifically, how the agency tracks market rents as part of its inflation data series, an issue I mentioned earlier. While the official statistics on housing services inflation do not break out new tenant market rents from the rest of the index, BLS staff recently began exploiting the housing survey data that underlie the CPI to construct a New Tenant Rent Index (NTRI).19 The NTRI looks at quarterly changes in the terms offered in new leases. Data users are still learning how to best combine the signal from the NTRI with other measures of market rents, but this research effort shows the agility of the statistical agencies in responding to current needs of understanding inflation dynamics.

These are but a few of the many innovations that have arisen from statistical agencies. Regarding the measurement challenges I mentioned earlier, the examples I just listed have helped observers better track the economy during periods of rapid change or when there are shifts in the structure of the economy, such as increased remote work or the rise of AI, changes to immigration patterns, and supply bottlenecks. A harder nut to crack is declining response rates, but it may be that more integration of private-sector data with official data provides an avenue for addressing this problem as well.20

Seeing Where the Economy Stands
I hope my discussion today highlights the tremendous innovation in economic measurement in recent years, supplied by both inventive private-sector data generation and nimble statistical agencies. It is incumbent on economists, researchers, and officials from statistical agencies around the world—many of you who are here today—to be open to new and diverse ways to measure the economy so that this healthy pace of innovation can continue.

I want to again stress two things. First, the United States has world-class statistical agencies that have both a long history of rigorously constructing government data sources and adapting and combining information from private-sector with official sources. And, second, private-sector data will continue to be useful for providing granularity, timeliness, and frequency advantages that can complement official statistics, so long as data users are appropriately cautious. Despite the many challenges, the future of economic measurement is bright. The statistical agencies have already proven their ability to innovate and adapt, even under tight resource constraints. And the wealth of private-sector data sources will only expand in the future. When I form my economic outlook and policy assessments, my approach is to watch a wide range of indicators, both official and unofficial, with a focus on the strengths and weaknesses of each.

All the data I carefully examine in my current role allow me to better understand where the economy stands. My colleagues on the FOMC and I make determinations on the policy actions that will be most appropriate for achieving our dual mandate, and so I would like to briefly share my views on how I see the economy evolving and how I see appropriate monetary policy.

Despite a few bumps at the beginning of the year, inflation has continued to trend down in all price categories. But inflation remains above our target. I do believe that supply and demand are gradually coming into better balance. Supply-side bottlenecks continue to heal, and demand has moderated amid high interest rates and as households' excess savings have depleted. The labor market likewise has seen substantial rebalancing and nominal wage growth moderating as a result—even while keeping up with inflation. Job vacancies and the quits rate have come down from their historically high levels from a couple years ago, and the vacancy-to-unemployment ratio is now back at its pre-pandemic level. On the labor supply side, the increased entry of prime-age workers and immigration have both helped to expand the labor force and compensate for excess retirements we saw during the pandemic. This continued rebalancing suggests that inflation will continue to move down toward our 2 percent target. As I have discussed in recent remarks, if economic conditions continue to evolve in this favorable manner with more rapid disinflation, as evidenced in the inflation data of the past three months, and employment softening but remaining resilient as seen in the past few jobs reports, I anticipate that it will be appropriate to begin easing monetary policy later this year. But my approach to this policy decision will continue to be data dependent and to rely on multiple and diverse sources of data to form my view of how the economy is evolving, especially as upside risks to inflation and downside risks to employment have become much more balanced. If the labor market cools too much and unemployment continues to increase and is driven by layoffs, I would see it as appropriate to cut rates sooner rather than later. Alternatively, if incoming data do not provide confidence that inflation is moving sustainably toward 2 percent, it may be appropriate to hold rates steady for a little longer.

As I conclude, I want to thank those of you in this room who do the hard work each day to create and analyze the economic data that allow not only policymakers like me, but also consumers and businesses, to gain a better understanding of ongoing developments in the U.S. economy. And let me thank NABE again for having me. It was a pleasure to speak with you today.


1. The views expressed here are my own and not necessarily those of my colleagues on the Board of Governors and the Federal Open Market Committee. Return to text

2. See Executive Office of the President, Office of Management and Budget, Office of Information and Regulatory Affairs, and Office of the Chief Statistician of the United States (n.d.), "Schedule of Release Dates for Principal Federal Economic Indicators for 2024 (PDF)" (Washington: OMB). Return to text

3. Board of Governors of the Federal Reserve System (2024), "Data," webpage. Return to text

4. Both the private-sector data-based measure and the official payroll survey are eventually benchmarked to official data from the BLS. See Tomaz Cajner, Leland D. Crane, Ryan A. Decker, Adrian Hamins-Puertolas, and Christopher Kurz (2023), "Payroll Employment at the Weekly Frequency," AEA Papers and Proceedings, vol. 113 (May), pp. 145–50; and Tomaz Cajner, Leland D. Crane, Ryan A. Decker, John Grigsby, Adrian Hamins-Puertolas, Erik Hurst, Christopher Kurz, and Ahu Yildirmaz (2020), "The U.S. Labor Market during the Beginning of the Pandemic Recession (PDF)," BPEA Conference Drafts, Brookings Papers on Economic Activity, June 25, pp. 1–46. Return to text

5. See Leland D. Crane, Ryan A. Decker, Aaron Flaaen, Adrian Hamins-Puertolas, and Christopher Kurz (2022), "Business Exit during the COVID-19 Pandemic: Non-traditional Measures in Historical Context," Journal of Macroeconomics, vol. 72 (June), 103419. Return to text

6. See Tomaz Cajner, Laura J. Feiveson, Christopher J. Kurz, and Stacey Tevlin (2022), "Lessons Learned from the Use of Nontraditional Data during COVID-19 (PDF)," in Wendy Edelberg, Louise Sheiner, and David Wessel, eds., Recession Remedies: Lessons Learned from the U.S. Economic Policy Response to COVID-19 (Washington: Hamilton Project and Hutchins Center on Fiscal and Monetary Policy at Brookings), pp. 315–346. Return to text

7. See, for example, Aditya Aladangady, Shifrah Aron-Dine, Wendy Dunn, Laura Feiveson, Paul Lengermann, and Claudia Sahm (2016), "The Effect of Hurricane Matthew on Consumer Spending," FEDS Notes (Washington: Board of Governors of the Federal Reserve System, December 2). Return to text

8. See Bureau of Labor Statistics (2024), "Household and Establishment Survey Response Rates," webpage. Return to text

9. National income and product accounts (NIPA) data sources are listed in detail in the NIPA handbook; see Bureau of Economic Analysis (2023), Concepts and Methods of the U.S. National Income and Product Accounts (PDF) (Washington: BEA, December). Some private-sector data are retrieved by the BEA itself, while others are already incorporated into statistical products the BEA obtains from other agencies. For example, the BEA uses Census Bureau data on construction value put in place to construct estimates of nonresidential structures investment; the Census Bureau, in turn, uses private-sector data on construction starts to build the sampling frame for its construction survey. Return to text

10. The data sources for Industrial Production and Capacity Utilization can be found at Board of Governors of the Federal Reserve System (2024), "Industrial Production and Capacity Utilization - G.17," webpage. Return to text

11. The data sources for Consumer Credit can be found at Board of Governors of the Federal Reserve System (2022), "Consumer Credit - G.19," webpage. Return to text

12. See Kimberly Bayard, Emin Dinlersoz, Timothy Dunne, John Haltiwanger, Javier Miranda, and John Stevens (2018), "Early-Stage Business Formation: An Analysis of Applications for Employer Identification Numbers," NBER Working Paper Series 24364 (Cambridge, Mass.: National Bureau of Economic Research, March). Return to text

13. See the box "Small Businesses during the COVID-19 Crisis" in Board of Governors of the Federal Reserve System (2020), Monetary Policy Report (PDF) (Washington: Board of Governors, June), pp. 24–26. Return to text

14. See Census Bureau (2022), "Small Business Pulse Survey: Tracking Changes during the Coronavirus Pandemic," webpage, February 11. Return to text

15. See documentation at Census Bureau (2024), "Household Pulse Survey Technical Documentation: Methodology," webpage. Return to text

16. See Kathryn Bonney, Cory Breaux, Cathy Buffington, Emin Dinlersoz, Lucia S. Foster, Nathan Goldschlag, John C. Haltiwanger, Zachary Kroff, and Keith Savage (2024), "Tracking Firm Use of AI in Real Time: A Snapshot from the Business Trends and Outlook Survey," NBER Working Paper Series 32319 (Cambridge, Mass.: National Bureau of Economic Research, April). Return to text

17. Other topics with satellite accounts or similar products include arts and culture, outdoor recreation, travel and tourism, household production, marine economy, and coastal areas. Related special topics are covered by the integrated macroeconomic accounts and the integrated industry-level production account (KLEMS). For descriptions, see Bureau of Economic Analysis (2024), "Special Topics," webpage. Return to text

18. See Carol E. Moylan and Sumiye Okubo (2020), "The Evolving Treatment of R&D in the U.S. National Economic Accounts (PDF)" (Washington: Bureau of Economic Analysis, March). Return to text

19. See Bureau of Labor Statistics (n.d.), "New Tenant Rent Index," webpage. Return to text

20. For helpful discussions of traditional and nontraditional data and their potential relationship, see Katharine G. Abraham (2022), "Big Data and Official Statistics," Review of Income and Wealth, vol. 68 (December), pp. 835–61; and Katharine G. Abraham, Ron S. Jarmin, Brian C. Moyer, and Matthew D. Shapiro (2022), "Introduction: Big Data for Twenty-First-Century Economic Statistics: The Future Is Now (PDF)," in Big Data for Twenty-First-Century Economic Statistics, National Bureau of Economic Research, Studies in Income and Wealth (Chicago: University of Chicago Press), pp. 1–22. Return to text

Last Update: July 16, 2024