It is a pleasure to join this group of dedicated researchers, bankers, community leaders, and policymakers interested in the worthwhile and challenging process of program assessment. Meaningful program review can be achieved only through measurement and critical analysis. Systematic research of community economic development programs has been limited. Accordingly, your challenge is to vastly expand the information base. Measuring the Impact of Community Economic Development Programs The overarching objective of community economic development and empowerment is to help underserved populations accumulate assets and improve their economic well-being. Measuring the results of programs dedicated to such goals is essential to maximizing the impact of these programs and managing scarce resources. Meeting the goals, particularly in areas and among populations where biases and negative perceptions may have contributed to market failures, helps people improve their financial standing, regardless of their current economic status. For nearly four decades, numerous policies and programs have been implemented with the intent of increasing economic opportunity. They have used a variety of management and funding strategies, ranging from federal government appropriations to debt and equity financing from private sources. Nevertheless, despite the broad spectrum of programs, the length of time they have been in place, and the array of funding participants, empirical research quantifying their impact is rare, regardless of whether government agencies, nonprofit organizations, or private entities sponsor the programs. The lack of measurement is particularly regrettable for government-sponsored programs, because quantifying their impact is crucial to the legislative process. When a bill is proposed in Congress, the nature of the problem and the factors presumed to be contributing to that problem typically are explicitly stated. And generally there is a projection of the outcomes that would indicate success. This process of problem diagnosis, program justification, and projection of results, if fully embraced, provides a cost-benefit structure for assessing a program's value. The program can be judged worthwhile when the data demonstrate that the benefits exceed the costs, including the opportunity costs of any investment. Even with such a framework, conducting research on community development and economic empowerment programs can be challenging, in part because the effects these programs intend to achieve are often quite difficult to measure and may not become apparent for relatively long periods of time. Initiatives aimed at complex economic and social challenges that were decades in the making require, more than likely, many years to achieve their goals. Unlike the standards for macroeconomic performance, virtually no specifically defined standards exist for monitoring the value of social and economic improvement programs. For community development researchers, the challenge is to develop parameters that can be used to objectively assess the value of their programs. For example, the measures that affordable housing organizations use could illustrate the extent to which their programs have, or have not, increased homeownership rates and property values, reduced crime, improved school performance, or spurred new private-sector investment in a disadvantaged neighborhood. Effective research must isolate the variables that best convey the impact of a program, define the specific data that must be collected, and develop a system for maintaining and retrieving the data over time. In other words, the challenge is to quantify the marginal effect of a program. The value of such a system is clear. So too, however, is the complexity of creating it. Consider, for example, the difficulty of measuring the marginal impact of a financial education program. It requires unique data collection techniques and unconventional tracking systems to gauge the benefit to an individual derived from making informed financial decisions that resulted from that educational program. Socioeconomic Trends in Central Cities The relative paucity of data and research on community development programs has limited the ability to fully demonstrate their impact and credibly differentiate those that are successful from those that are ineffective. Undeniably, impressive local community development initiatives have been undertaken, and individual testimonials reveal advances in the economic well-being of many of the beneficiaries. However, the absence of formal data collection and research for the numerous neighborhood revitalization efforts over the past several decades has resulted in a reliance on mostly anecdotal reporting at a neighborhood or individual level. Anecdotal information is not without value. It offers clues to the construction of a more formal statistical analysis. But, as I am sure all of you have experienced, anecdotes can be selective and can convey a false message of the success or failure of a community development program. Given the lack of data demonstrating outcomes from new initiatives, the inclination is to examine existing data to identify trends in areas where community development organizations have been a consistent presence for some time. One hopes that broad positive trends that cannot be understood fully from conventional market forces will suggest the possibility of community development being at least a partial explanation of these trends. Since most community development initiatives focus on urban areas, data on socioeconomic trends in central cities may offer some insight into the influence of local economic and social programs. For example, Census statistics compiled for the State of the Cities database of the Department of Housing and Urban Development (HUD) show that increases in the rates of change in homeownership in central cities slightly exceeded the increases in suburban communities between 1970 and 2000. The Federal Reserve Board's 2001 Survey of Consumer Finances concluded that between 1998 and 2001, families in the lowest quintile of the income distribution increased their rate of homeownership nearly 5 percent, saw their median income grow more than 14 percent, and realized a 25 percent gain in their median net worth. Although these gains in homeownership rates in central cities and the economic progress of lower-income families are encouraging, other data covering a longer time frame are less sanguine. In particular, HUD's State of the Cities database indicates that residents in central cities barely increased their real median family income between 1969 and 1999, while families in suburban communities did appreciably better. In addition, the poverty rate in central cities, according to this database, increased 23 percent during this thirty-year period, while it decreased 7 percent in suburban areas. These seemingly contradictory data undermine efforts to plan an appropriate course of action. The absence of credible data clearly renders researchers unable to attribute the gains to a particular program or the continued challenges to a particular failure. In weighing the implications of recent trends in data, it is important to factor in the presence of changes in external market influences. For example, advances in mortgage underwriting and delivery systems have resulted in increased availability of funding for homeownership. Community development investors' funding strategies have also changed considerably over this thirty-year span. At the Federal Reserve, economists strive to identify the appropriate variables for assessing the impact of regulations, in particular the Community Reinvestment Act (CRA). In addition to the research presented at this conference, Federal Reserve economists have undertaken studies to assess whether the CRA causes banks to provide a mortgage subsidy and to determine the performance and profitability of CRA-eligible loans. But the lack of broad data management systems to identify and track the performance and profitability of most CRA-eligible loans presents researchers with a challenge, as does the need to focus on changes that can truly be attributed to the CRA, rather than to changing market forces. Information Gains in Community Development While empirical research on specific community development programs is limited, insights nonetheless have been gained from experience over the past several decades. Many community development corporations (CDCs) have modified their strategies and their structures accordingly. Most notably, CDCs have realized the necessity of diversifying their funding sources and reducing their reliance on government funding, which is vulnerable to the vagaries of shifting political priorities. In seeking to ensure continued financial support for their programs, community development leaders have expanded their range of financing and, in the process, have gained a better understanding of the risk tolerance and return requirements of their various capital providers. In addition to diversifying funding sources, community developers have also sought to broaden their financing strategies. They once viewed debt as the primary, if not the sole, vehicle available for capitalizing community development efforts, but now recognize the vital role of equity investment in helping communities withstand economic downturns. New sources of equity--community development venture capital funds and secondary markets that securitize community development loan pools--have become available to energize market forces in economically distressed neighborhoods. Advances in technology have significantly improved the identification and development of new financing strategies. With increased information-processing capacity, loan portfolio managers can better assess risk and monitor credit performance. Additionally, the ever-increasing availability of data facilitates the development of neighborhood profiles that can be useful in understanding and tracking community socioeconomic trends. For example, the cross-referencing of data sets on mortgage lending patterns, business start-ups, and employment figures against crime statistics and property values can provide a valuable perspective. Benefits of Research Many valuable lessons have been learned in community development over the years. And the dissemination and application of such lessons as they emerge are essential to improving program effectiveness. Formal research can accelerate the rate of such learning. Only through a comprehensive understanding of the outcomes of a program can success be emulated and failures reduced. By consistently and reliably measuring outcomes, and thus helping current and prospective investors better assess their risks and predict their returns, community development organizations can attract more funding. Such accountability is crucial for any organization, regardless of its size. In addition to increasing funding options, research can also increase the scope and scale of programs. As effective strategies are identified, they can be replicated and incorporated into efforts in other communities, as well as by organizations seeking to develop programs to address related issues. The Process of Measuring Outcomes As I noted earlier, it is important to establish formal procedures for program assessment. At the start of a program, the nature of the problem should be identified, as well as the presumptions of the various causes of that problem. With a clearer understanding of the issues, policymakers and community leaders are better able to devise a strategy for overcoming the problem. Finally, a well-constructed program must include a projection of its benefits to serve as a benchmark for later evaluation. In conclusion, I want to emphasize the importance of the role of the interpreter of the research. Analysts must be scrupulously honest in characterizing research results, or their work becomes advocacy and is no longer research. This objectivity is paramount because research findings from previous efforts become the basis for subsequently targeting scarce resources to their highest and best use. Such objectivity requires great discipline and integrity on the part of the researcher; it requires that researchers resist any innate desire to characterize results in the most, or least, favorable light possible. An understanding of the findings, positive or negative, is the greatest contribution of research. The failure of a program is not a research failure; it is a source of information. And acknowledgement of unadulterated research findings, regardless of how disappointing, contributes to a foundation of knowledge upon which future successes can be built. I often say at the end of a day that I learned a great deal. Unfortunately, most of what I learned was that what I thought I knew at the beginning of the day was false. In the quest to do good for our society's most vulnerable populations and communities--the objective compelling the work of this group--we must embrace the challenge to develop objective and quantifiable standards to assess community development programs. Ultimately, such research is the only means for determining whether we are making advances in overcoming failures in distressed neighborhoods and improving access to economic opportunities for traditionally underserved populations. I applaud your efforts and look forward to learning of your future progress.

Return to top 2003 Speeches