Report to the Congress on Credit Scoring and Its Effects on the Availability and Affordability of Credit
Submitted to the Congress pursuant to section 215 of the Fair and Accurate Credit Transactions Act of 2003
August 2007
Print (1.84 MB PDF)
Scope of the Report
This report provides the results of the assessment of the effects of credit scoring on credit and was prepared by the Federal Reserve Board. The report draws on secondary sources of information, such as public comments and previous studies or analyses as well as on an analysis of a credit-scoring model constructed specifically for this report.
Public Comment on the Study
The Federal Reserve Board and the Federal Trade Commission sought comments and suggestions from government agencies, members of the public, industry groups, and other interested parties, including community organizations and fair lending and fair housing organizations. Comments and suggestions came largely in three ways: First, in response to two Federal Register notices seeking answers to a wide range of specific questions about the use of credit scoring in credit and insurance and about ways to conduct the study; second, in meetings with interested parties to gain further insight on how to conduct the study, to learn about available data and analytic approaches, and to hear concerns regarding the agencies' plans for the study; and third, through detailed discussions with leading builders of credit-scoring models to learn about the techniques for building such models.4
Approaches Considered in Conducting the Study
Section 215 of the Fact Act essentially asks for a review of three related concerns regarding credit scoring. The first is the effect of credit scoring on the availability and affordability of financial products to consumers in general. The second is whether the relationship between credit scores, on the one hand, and credit performance, availability, and affordability, on the other, vary across demographic groups. The third is whether credit scores in general, and the particular factors included in credit-scoring models, may result in negative or differential treatment of specific subpopulations and, if so, whether that treatment could be mitigated by changes in the model development process.
Regarding the first concern--the effect on the availability and affordability of credit--commenters provided only limited information. Some described in general terms how credit scoring has affected credit availability and affordability but gave little specific information or direct evidence. The relative paucity of specific evidence provided by commenters is not surprising, as much of the data, particularly regarding the effectiveness of risk evaluations based on judgment versus credit scoring, is proprietary and often based on evaluations conducted many years ago. Nonetheless, the present study reviews the information provided by commenters, and by other reports in the public domain, regarding the effects of credit scoring on credit availability and affordability. The study also analyzes data gathered over the years by the Federal Reserve Board in its Survey of Consumer Finances. These data provide indirect evidence of the effects of credit scoring on credit availability and affordability over time.
Commenters suggested ways in which the study could address the second and third issues in the section 215 requirement: whether the relationship between credit scores and credit performance, availability, and affordability varies across populations and whether credit scoring, in general, as well as particular factors included in credit-scoring models may disadvantage specific subpopulations and whether any improvements could be found in changes to the models. The suggestions fell into two broad types of inquiry. The first type was a series of "disparate impact audits" of existing major credit-scoring models. The audits would focus on the appropriateness of the factors used in model development and of the weights attached to those factors and on the relationships between credit scores and loan performance. The second type--the "model building" option--would address the potential for creating disparate impact in the process of developing a credit-scoring model. This approach would evaluate the creation and use of a generic credit-scoring model rather than of any model that already exists. The information collected to develop this model could also be used to empirically evaluate the relationship between credit scores and credit performance, availability, and affordability.
The audit approach would either be restricted to an evaluation of analyses conducted by the model builders themselves or would require the auditing entity to have access to the actual samples used to estimate each model and all of the model weights and components. In contrast, the model-building approach requires original work to create a credit-scoring model that corresponds to the process followed by the industry and to collect data against which to test and evaluate it; it therefore offers the potential for a much wider scope of analysis and can address issues and methods not considered in the self-assessments of the industry's model builders. However, the second approach--model building--is limited in that it cannot offer a definitive conclusion about any particular model; rather, its results are only suggestive of the issues that arise in the process of model development. The issue of representativeness is important to both approaches. The audit approach requires that the models reviewed be representative of those used by the industry.5 The model-building option requires that the process of creating and estimating the model be representative of industry practice.
There was little choice in deciding which of the two approaches to use. A strict audit approach was not feasible because necessary data on the personal demographic characteristics of credit applicants and borrowers generally is not available, except for data identifying the sex, race, and ethnicity of home mortgage applicants. Although a few suggestive studies have been conducted by relying on the racial composition of neighborhoods to represent the race or ethnicity of individuals, they do not satisfy the requirements of section 215.
A modified audit approach was considered. It would entail gathering information on the racial, ethnic, and other personal characteristics required and appending them to the actual samples of data on individuals used by model developers for a representative sample of industry credit-scoring models. This would allow each model to be evaluated for the issues identified in section 215. However, even the modified approach was not feasible. Model developers generally use estimation samples stripped of personal identifying information such as name and Social Security number. Obtaining this information would have required going back to the original data sources and attempting to gather this information with appropriate legal safeguards. The logistics of such an undertaking were sufficiently complex and daunting that this approach could have been at best used for one or two models. Narrowing the scope strips the audit approach of one of its principal strengths, namely, coverage of a large number of models in use today. Moreover, unless this approach relied on the original sample of observations used for the actual model development, it could no longer be represented as an audit of the actual credit-scoring model being evaluated.
These limitations led the Board to adopt the second approach for conducting this study--creating a model from scratch and assembling a data set with which to evaluate issues related to the relationship between credit scores and credit performance, availability, and affordability. Having made a choice of approach, other issues needed to be addressed, the most important of which was to choose the types of credit-scoring models to assess.
Types of Credit-Scoring Models
Credit-scoring models differ from each other along three distinct dimensions: (1) the factors used to form the prediction, (2) the type of borrower performance the model is designed to predict, and (3) the population used to estimate the model (that is, the population used to empirically derive the model's predictions).
The narrowest set of factors used to form predictions is drawn from information included in the credit records maintained by credit-reporting agencies. Models that limit the factors to that set are the most widely used and are commonly referred to as credit history scoring models.6 Other credit-scoring models derive their predictions from a broader or different set of data, such as the information recorded on applications for credit (much of which does not appear in records of the credit-reporting agencies) or a creditor's own data on experiences with their customers.
Among other things, the models seek to predict borrower performance for a specific credit product, such as home mortgages, automobile loans, and credit cards or performance for any type of credit account. (A later section of this study provides a more extensive discussion of what credit-scoring models seek to predict and how they are used.)
When the populations used for estimation include a creditor's current or prospective customers, the model is typically referred to as a custom credit-scoring model. When the population is based on a representative sample of all individuals in credit-reporting agency records, the resulting model is typically referred to as a generic credit-scoring model. A generic credit-scoring model in which the predictive factors are limited to the information contained in credit records is generally referred to as a generic credit history scoring model. For reasons described below, the model developed specifically for this study and those used to evaluate the relationship between credit scores and credit performance, availability, and affordability are generic credit history scoring models.
Reasons for Focusing on Generic Credit History Scoring Models
Thousands of custom credit-scoring models are in use today by lenders to support their underwriting, account management, and marketing, whereas generic credit history scoring models are relatively few in number and made available by only a small number of firms.7 However, although they are few in number, generic credit history scoring models are central to the operations of the credit industry.
As a summary of the credit histories of individuals, generic credit history scores are widely used by lenders to supplement and support various aspects of the lending process. For example, (1) lenders use them even when they also draw on a broader range of information such as data from applications for credit, (2) a generic credit history score alone often provides the credit history component of lending evaluations that are conducted manually, (3) lenders that have developed their own (that is, custom) credit history scoring models often use generic scores to facilitate loan sales and to enhance portfolio management, and (4) lenders commonly use generic credit history scores as a criterion, often the sole criterion, in deciding who should receive so-called "prescreened" solicitations for new accounts. It is this central role played by generic credit history scoring models that placed them, rather than some custom model or a model looking at factors other than credit history, at the center of this study.
The choice to focus on generic credit history scoring models has limitations. Decisions about loan pricing and assessments of credit risk are likely to be based on credit-scoring models that include a broader set of information than those used to estimate credit history scoring models. Thus, empirical assessments of the relationship between generic credit history scores and credit performance, availability, and affordability may not be fully reflective of the relationships that would be observed between the credit scores actually used to underwrite credit and subsequent credit outcomes.
Further, because the factors evaluated in this study are restricted to items included in credit-reporting agency files, the results related to assessments of possible differential effect will not be applicable to other types of information considered in credit underwriting or other uses. And even for the credit-record items reviewed here, the assessment of differential effect may not necessarily be consistent with an analysis that would simultaneously consider other types of information often included in credit evaluations. Despite these limitations, the approach taken here is likely to be suggestive of results for other existing models, whether they are generic history scores or are based on other types of information.