April 04, 2024
Enriching Data and Analysis in Economics with Real Life Experiences
Governor Adriana D. Kugler
At Women in Economics Symposium, Federal Reserve Bank of St. Louis, St. Louis, Missouri
Thank you, President Musalem, and many congratulations on the start of your new position here at the St. Louis Fed. We are all so happy to have you as part of the Federal Reserve System, and I am grateful for the opportunity to speak here today.1
I will start with the issue that has brought us here—why it is so important for women to be a full and equitably represented part of the economics profession. There are, of course, moral reasons for economics to reflect the values of our democratic society—it is the right thing to do. But there are eminently practical reasons as well. Economics favors the productive utilization of resources, and bringing more women into the profession obviously makes better use of the talents and efforts of more than half of the population. Greater diversity in a workforce yields better ideas and greater productivity, and the field of economics has certainly been enriched by these contributions.
But today, I would like to focus on the benefits of greater representation of women that accrue to economic research and policymaking—the way that you can make economic analysis more constructive and relevant. In a broad sense, different experiences and perspectives among those engaged in any social science should provide better insights about how people think and act. But I would like to take this point a step further and connect this issue of representation to the ways in which we approach economic research. In particular, I will describe how my own personal experience has been an important tool for me to use, in combination with the more familiar and rigorous tools of the economist, in ways that I believe have strengthened my research. In doing so, I also hope to show that personal experience can be used by you and others in our profession to provide deeper insights about the questions we grapple with. If this is true, it would constitute another way in which greater representation of women—and others with underrepresented backgrounds—will increase and enhance the valuable contributions that economics makes to society.
In my discussion, I will refer to some of my own past research to illustrate my general approach to economics. Importantly, though, my comments today are only illustrative of a general approach and are not commentary on any specific policy questions.
In explaining how I let personal experience and rigorous economic methodology inform each other, I am going to employ, in a somewhat casual way, two words that you may be familiar with. I think of a "deductive" approach as beginning with or relying on theory or general observations and then using data or qualitative evidence to verify those ideas in specific settings. For me, an "inductive" approach means beginning with personal experience and specific evidence from a particular context and then moving to more general lessons that we can draw on as economists about given issues or topics. In my career, I have found it fruitful to employ both of these approaches at different times, and often in succession or in alternation, to move from the specific to the general, developing a theory and then sometimes using specifics to test that theory.
Let me start with a little personal background, since that helped form my approach to research. I was born in the U.S. but grew up in Colombia. While there, my parents and I traveled around the country, where I observed a broad range of social problems: poverty, homelessness, child labor, and a lack of access to health care and other basic necessities. As a child, when you see suffering to this extent, you wonder about why and how it is occurring. Later, as an undergraduate, I found that economics offered tools that could help society understand and, I hoped, address some of these problems. What I started to see, and then saw more over time, was that in many ways the different social problems I observed were closely related to the opportunities people have to work and earn an income that reflects their effort and productivity as workers—in short, issues at the heart of labor economics. It also turned out that the tools of economics—for example, math and statistical analysis—were well suited to studying these questions and helped ground my work in objective reality. So I developed these skills by pursuing a Ph.D. in economics. During my career, I have conducted research on a range of topics, including labor markets, productivity, and social safety nets. And, as I will show, I have employed both the inductive and the deductive approaches that I have described here to formulate economic questions, rigorously examine data and evidence, and then test my conclusions.
The first example I want to share with you is a paper I wrote with Scott Imberman and Bruce Sacerdote on the relocation of families displaced by Hurricanes Katrina and Rita.2 In 2005, I was a professor of economics at the University of Houston, in a city that was a major destination for people forced to evacuate from New Orleans and the surrounding area because of the hurricanes that year. Of the million people displaced by the hurricanes in 2005, Houston received 75,000 evacuees, many of them school-age children.
Houston was a fast-growing city with schools that already faced challenges, so people had questions about what effect this influx of new students might have on our local schools—now stretched for resources—and the students already attending them. It mattered to me as a member of the community with my own children attending those schools. Fortunately, economics provided tools to explore this question.
The question we considered was about "peer effects"—does the performance of your peers affect your own performance in school? How will students coming from other schools affect the outcomes for those already in Houston? The research literature showed some evidence of an effect, but the results were mixed. It is easy to imagine some reasons why peers could matter. A key challenge was that, even if we could find a relationship between the outcomes of the existing Houston students and their peers from Louisiana, that would not prove that the new students "caused" the outcomes of the existing Houston students. Classmates in the same classes with similar backgrounds could have similar outcomes for lots of reasons. But in this case, these new peers did not choose to attend certain schools based on pre-existing trends but instead were assigned to schools based on their shelter location, which is basically a random assignment—highly conducive to estimating causal effects. And the magnitude of this particular event was large enough to provide lots of variation in the composition of peers. I consider the approach we took to designing this study a good example of the power of deductive reasoning—developing a hypothesis and then finding the data by which to test it.
But we didn't just think about the data we started with. We had lots of discussions with school administrators and teachers. What they had to say about their experiences led us to explore ways in which the new students were affecting the existing ones, beyond grades and test scores—for example, behavior in schools and absenteeism. As any parent of school-age kids can tell you, these are obvious and significant factors in how students learn, and these discussions resonated for me as a parent with kids in school. My co-authors and I knew we had to make this part of our research.
Our basic approach in this paper was to relate changes in student outcomes with the prevalence of evacuee children in their district, controlling for a rich set of factors. Our conclusions were that in Houston, a large influx of evacuees did not generally have a large effect on test performance of local students. We did detect some negative effects in some subgroups, particularly in math. Motivated by discussions with teachers and administrators, we also found some small negative effects on attendance and behavior. Using traditional methods in economics, we conducted a number of what economists call "robustness checks," making sure that our assumption that the relocating families weren't picking school districts based on pre-existing trends held up, trying different ways of measuring outcomes or specifying models, and attempting to exclude alternative explanations, such as changes in teacher quality or educational resources.
Using these familiar methods, we had found that Houston schools were able to absorb the new students with little aggregate effect on local students. But that finding sent us back to think about the theories we had developed at the start of the project and the different reasons that make peers relevant to a student. In a sense, now that we had some results, what had started as a deductive analysis turned into an inductive one. And by examining the data more thoroughly, we discovered that a crucial factor in determining the impact of peers on academic outcomes was the extent to which students are grouped and interact with classmates of a similar academic ability level. Adding this dimension to our research, we found that peer effects were, in fact, stronger among cohorts of students with similar academic ability. It is a nuance to our overall finding of little significant effect and a result that I suspect would not have been attainable without the personal experience that I brought to the project as a parent—one of the factors that had motivated the work in the first place.
Let me offer a second example of this "inductive" approach, based on personal experience, and the interaction with a "deductive" approach that resulted in research that I am particularly proud of. As I said earlier, I taught at the University of Houston, a great school that has a very diverse student body, with people from all walks of life and at different stages of their lives. There were a lot of commuters, many still living at home with parents. There were also a lot of actual parents in my classes and people who had to work full-time jobs while completing their education. I once had a student who needed to reschedule a final exam because she had to go to court to fight for custody of her child. I had students who struggled to secure childcare to attend class, others who struggled with access to transportation to and from campus. The hurdles they often had to overcome were daunting. And while many obtained loans and other tuition assistance, the challenges they faced were multidimensional, and I often found myself thinking that their education would be much more successful if they had support in other areas of their lives.
I had written two papers on a training program in Colombia that included a stipend for workers.3 The stipend was intended to subsidize transportation for workers, with an extra amount for women that was intended to supplement their costs for childcare. The results found a significant boost to wages and employment from the training program, especially for women, and we found that the gains stayed with those workers for over a decade after the program. Moreover, we also found that the program encouraged workers and even their relatives to pursue more formal schooling—high school and even college. But these papers didn't allow me to explore the separate contribution of the stipend to the wage, employment, and educational gains we found. My own experience with students at the University of Houston suggested that this more comprehensive support for the workers in training was important, so I decided to follow up on my research.
So I wrote another paper—jointly with Felipe Barrera-Osorio and Mikko Silliman—based on an experiment we conducted in collaboration with a training program.4 In this experiment, we randomly offered stipends to some but not all workers. Students might use the stipend to pay for transportation to and from the program, or perhaps to help cover meals during the program, depending on what they saw as their most pressing needs. I like to call this a form of "wraparound" support; the stipend wraps around the main program to make it easier for people to participate. Some workers accepted to the training program were put in a class that focused on technical skills, and others in a class emphasizing social skills—again using random assignment. We found that participants saw an increase in employment, hours worked, and wages relative to workers who did not take the program, and the effects were greatest for women. Labor market outcomes improved more for those who received technical skills training, though this difference diminished after less than a year. And, lastly, we found that the effects of the program were indeed magnified by the stipend program. The "wraparound" support enhanced the value of the training program and likely helped mitigate the other labor market challenges faced by participants.
This paper again shows how insights gained from personal experience, combined with the deductive methods and tools of economics, can be the basis for good research. As I suspected from my experience with my University of Houston students, providing education is more effective, and the returns are higher, when students have some "wraparound" support that makes their lives a bit easier. We took this inductive approach and demonstrated it using standard tools of economics—randomization in a carefully designed experiment.
And this approach continued in my past work as a policymaker. At the U.S. Department of Labor and at the World Bank, I confronted questions that resonated for me with experiences I had had and earlier applied in my work as a researcher. At the Labor Department, I worked on workforce development issues, and we were able to include access to more comprehensive "wraparound" services to promote strong take-up and retention in these programs. When I was advocating for this approach, I thought about my students in Houston and those workers in Colombia, all of them trying to overcome big challenges and build a better life. And I was grateful then, and I'm grateful now, for my personal experiences, which didn't come from a textbook or a lecture but helped open my eyes to the answers to some of the important questions that economics strives to answer.
The research and policy work I have just described predate my time as a member of the Federal Reserve Board, and the policies I studied then are outside the purview of my current role. But I continue to apply the general approach of bringing my own experience, and the experiences I hear from others, together with the rigorous tools of economics. I listen to people from a range of communities and backgrounds to better understand the economic data and the statistical analysis we perform while working to make monetary policy decisions within the framework assigned to us by Congress that benefit all Americans.
So I will close my remarks by urging all of you to find ways to bring your unique experiences into your work in economics. I am a woman, a Latina, an immigrant, and a mother. As a person with this particular background, I have found that I have a unique perspective I can bring to the economics profession. By all means, build the technical skills you will need to prove your ideas; that is, indeed, critical in the economics profession. But don't be afraid to bring up your own perspective, based on your own experiences, when discussing economic issues. I think this is especially important for women, whose perspectives are still underrepresented in the profession.
Whether you decide to pursue a path in economics focused on research, policy, business, or some combination of these, you will make the greatest contribution by becoming a true expert on the issues you work on. You will accomplish this by refining your technical skills and, importantly, by bringing your unique perspective to each problem you seek to solve.
1. The views expressed here are my own and are not necessarily those of my colleagues on the Federal Reserve Board or the Federal Open Market Committee. Return to text
2. See Scott A. Imberman, Adriana D. Kugler, and Bruce I. Sacerdote (2012), "Katrina's Children: Evidence on the Structure of Peer Effects from Hurricane Evacuees," American Economic Review, vol. 102 (August), pp. 2048–82. Return to text
3. See Adriana Kugler, Orazio Attanasio, and Costas Meghir (2011), "Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial," American Economic Journal: Applied Economics, vol. 3 (July), pp.188–220; and Adriana Kugler, Maurice Kugler, Juan E. Saavedra, and Luis Omar Herrera-Prada (2022), "Long-Term Educational Consequences of Vocational Training in Colombia," Journal of Human Resources, vol. 57 (January), pp. 178–216. Return to text
4. Felipe Barrera-Osorio, Adriana Kugler, and Mikko Silliman (2023), "Hard and Soft Skills in Vocational Training: Experimental Evidence from Colombia," World Bank Economic Review, vol. 37 (August), pp. 409–36. Return to text