Board of Governors of the Federal Reserve System

Supervisory Stress Test Framework and Model Methodology

Analytical Framework

The Federal Reserve estimated the effect of the supervisory scenarios on the regulatory capital ratios of the 35 firms participating in DFAST 2018 by projecting the balance sheet, RWAs, net income, and resulting capital for each firm over a nine-quarter planning horizon, which for DFAST 2018 begins in the first quarter of 2018 and ends in the first quarter of 2020. Projected net income, adjusted for the effect of taxes, is combined with capital action assumptions to project changes in equity capital. The approach followed U.S. generally accepted accounting principles (GAAP) and regulatory revised capital framework.²¹ Figure 8 illustrates the framework used to calculate changes in net income and regulatory capital.

Projected net income for the 35 firms is generated from projections of revenue, expenses, and various types of losses and provisions that flow into pre-tax net income, including

• pre-provision net revenue (PPNR);
• loan losses and changes in the allowance for loan and lease losses (ALLL);
• losses on loans held for sale (HFS) or for investment and measured under the fair-value option (FVO);
• other-than-temporary impairment (OTTI) losses on investment securities in the available-for-sale (AFS) and held-to-maturity (HTM) portfolios;
• losses on exposures resulting from a global market shock for firms with large trading and private equity exposures; and
• losses from the default of the largest counterparty of firms with substantial trading, processing, or custodial operations.

PPNR equals net interest income plus noninterest income minus noninterest expense. Consistent with U.S. GAAP, the projection of noninterest expense includes projected losses due to operational-risk events such as fraud, computer system or other operating disruptions, and litigation-related costs and expenses related to the disposition of foreclosed properties (other real estate owned (OREO) expenses).

Provisions for loan and lease losses equal projected loan losses for the quarter plus the amount needed for the ending ALLL to be at an appropriate level to account for projected future loan losses. The amount of provisions over and above loan losses may be negative, representing a drawdown of the ALLL (an ALLL release, increasing net income), or positive, representing a need to build the ALLL (an additional provision, decreasing net income).

Because the loss projections follow U.S. GAAP and the Board's capital adequacy rules, they incorporate any differences in the way these guidelines recognize income and losses based on where assets are held on the firms' balance sheets. As a result, losses projected for similar or identical assets held in different portfolios can sometimes differ. For example, losses on loans held in the accrual portfolio equal credit losses due to failure to pay obligations (cash flow losses resulting in net charge-offs). For similar loans that are held for sale or held for investment and classified as fair value loans, projected losses represent the change in fair value of the underlying assets in the supervisory scenario.

Following this approach, changes in the fair value of AFS securities and OTTI losses on securities are separately projected over the nine-quarter planning horizon. Under U.S. GAAP, changes in the fair value of AFS securities are reflected in changes in accumulated other comprehensive income (AOCI) but do not flow through net income. In addition, if a security becomes subject to OTTI, all or a portion of the difference between the fair value and amortized cost of the security must be recognized in earnings.²² Consistent with U.S. GAAP, OTTI projections incorporate other-than-temporary differences between book value and fair value due to credit impairment but generally do not incorporate differences reflecting changes in liquidity or market conditions.

For the six firms subject to the full global market shock, the losses on trading and private equity positions as well as the credit valuation adjustment are projected assuming an instantaneous re-pricing of these positions under the global market shock (see Global Market Shock and Counterparty Default Components). Losses from the global market shock are assumed to occur in the first quarter of the planning horizon. No subsequent recoveries on these positions are assumed, nor are there offsetting changes such as reductions in compensation or other expenses in reaction to the global market shock. In addition, incremental losses from potential defaults of obligors underlying firms' trading positions are projected over the planning horizon.

For the eight firms subject to the counterparty default component, the losses associated with the instantaneous and unexpected default of the largest counterparty across derivatives and securities financing transaction (SFT) activities are projected. These losses are assumed to occur in the first quarter of the planning horizon.

For the six IHCs subject to the supervisory market risk component in 2018, losses associated with the supervisory market risk component are treated as an add-on to losses associated with the macroeconomic scenarios and are assumed to occur in the first quarter of the planning horizon.

Over the planning horizon, the Federal Reserve projects quarter-end amounts for the components of the balance sheet. These projections are made under the assumption that firms maintain their willingness to lend while demand for credit changes in response to conditions in the scenario. Firms are assumed to use lending standards in line with their long-run behavior. Any new balances implied by these projections are assumed to have the same risk characteristics as those held by the firm at the start of the planning horizon except for loan age. Where applicable, new loans are assumed to be current, and firms are assumed not to originate types of loans that are no longer allowed under various regulations. The Federal Reserve also incorporates material changes in a firm's business plan, such as a planned merger, acquisition, consolidation, or divestiture.²³ Only divestitures that had been completed or contractually agreed to prior to April 5, 2018, are incorporated. Once adjusted, assets are assumed to grow at the same rate as the pre-adjusted balance sheet.

Model Methodology

The Federal Reserve's projections of revenue, expenses, and various types of losses and provisions that flow into pre-tax net income are based on data provided by the 35 firms participating in DFAST 2018 and on models developed or selected by Federal Reserve staff and evaluated by an independent team of Federal Reserve model reviewers. The models are intended to capture how the balance sheet, RWAs, and net income of each firm would be affected by the macroeconomic and financial conditions described in the supervisory scenarios, given the characteristics of the firms' loans and securities portfolios; trading, private equity, and counterparty exposures from derivatives and SFTs; business activities; and other relevant factors.²⁴

Detail of model-specific methodology is provided in appendix B. Changes to supervisory models used in DFAST 2018 are described in box 1.

Models were developed using multiple data sources, including pooled historical data from financial institutions. An industrywide approach was generally adhered to, in which the estimated model parameters are the same for all firms and reflect the industrywide, portfolio-specific, instrument-specific response to variation in the macroeconomic and financial market variables. This approach reflects both the challenge in estimating separate statistically robust models for each of the 35 firms and the desire of the Federal Reserve not to assume that historical firm-specific results will prevail in the future. This means that the projections made by the Federal Reserve will not necessarily match similar projections made by individual firms.

The Federal Reserve deviated from the industrywide modeling approach when the historical data used to estimate the model were not sufficiently granular to capture the impact of firm-specific risk factors, and firm-specific indicator variables (fixed effects) representing the firm's average longer-term history were more predictive of the firm's future performance than industry variables. For example, the models to project components of PPNR feature firm-specific indicator variables because available data are not sufficiently granular and a firm's own history, after controlling for structural changes over time, is proven to be more predictive of the firm's revenues and expenses under stress than industry-level history. In some other cases, such as the projections of trading and counterparty losses, sensitivities to risk factors and other information generated by the firms from their internal pricing models are used due to the lack of position-level data and modeling complexity.

Loan losses are estimated separately for different categories of loans, based on the type of obligor (e.g., consumer or commercial and industrial), collateral (e.g., residential real estate, commercial real estate), loan structure (e.g., revolving credit lines), and accounting treatment (accrual or fair value). These categories generally follow the classifications of the Consolidated Financial Statements for Holding Companies (FR Y-9C) regulatory report, though some loss projections are made for more granular loan categories.

Two general approaches are taken to model losses on the accrual loan portfolio. In the first approach, the models estimate expected losses under the macroeconomic scenario. These models generally involve projections of the probability of default, loss given default, and exposure at default for each loan or segment of loans in the portfolio, given conditions in the scenario. In the second approach, the models capture the historical behavior of net charge-offs relative to changes in macroeconomic and financial market variables.

Accrual loan losses are projected using detailed loan information, including borrower characteristics, collateral characteristics, characteristics of the loans or credit facilities, amounts outstanding and yet to be drawn down (for credit lines), payment history, and current payment status.

Data are collected on individual loans or credit facilities for wholesale loan, domestic retail credit card, and residential mortgage portfolios. For other domestic and international retail loans, the data are collected based on segments of the portfolio (e.g., segments defined by borrower credit score, geographic location, and loan-to-value (LTV) ratio).

Losses on retail loans for which a firm chose the fair-value option accounting treatment and loans carried at the lower of cost or market value (i.e., loans held for sale and held for investment) are estimated over the nine quarters of the planning horizon using a duration-based approach. Losses on wholesale loans held for sale or measured under the fair-value option are estimated by revaluing each loan or commitment each quarter of the planning horizon.

Losses on securities held in the AFS and HTM portfolios are estimated using models that address other-than-temporary differences between amortized cost and fair market value due to credit impairment but generally do not intend to reflect temporary differences reflecting changes in liquidity or market conditions. Some securities, including U.S. Treasury and U.S. government agency obligations and U.S. government agency mortgage-backed securities, are assumed not to be at risk for the kind of credit impairment that results in OTTI charges. For all other debt securities, OTTI charges are projected using the statistical relationship between historically observed OTTI write-downs and measures of the fair value of the securities. The models use securities data collected at the individual security level, including the amortized cost, market value, and any OTTI taken on the security to date.

Losses related to the global market shock and the counterparty default components are estimated based on firm-estimated sensitivities to various market risk factors, market values, and revaluations of counterparty exposures and credit valuation adjustment under the global market shock.

PPNR is generally projected using a series of models that relate the components of a firm's revenues and non-credit-related expenses, expressed as a share of relevant asset or liability balances, to firm characteristics and to macroeconomic variables. Most components are projected using data on historical revenues and operating and other non-credit-related expenses reported on the FR Y-9C report. Separate data are collected about firms' historical losses related to operational-risk events, which are modeled separately from other components of PPNR.

The balance sheet projections are derived using a common framework for determining the effect of the scenarios on balance sheet growth, and, as noted, incorporate assumptions about credit supply that limit aggregate credit contraction. These sets of projections are based on historical data from the Federal Reserve's Financial Accounts of the United States (Z.1) statistical release, which is a quarterly publication by the Federal Reserve of national flow of funds, consolidated balance sheet information for each firm, and additional data collected by the Federal Reserve.²⁵

Once pre-tax net income is determined using the above components, a consistent tax rate is applied to calculate after-tax net income. After-tax net income also includes other tax effects, such as changes in the valuation allowance applied to deferred tax assets (DTAs) and income attributable to minority interests. See box 2 for an explanation of modifications to the calculation of projected capital to account for the passage of the TCJA in December 2017.

Model Risk Management, Governance, and Validation

The Federal Reserve places great emphasis on the credibility of its supervisory stress testing process, which is supported by a rigorous program of supervisory model risk management. The Federal Reserve's supervisory model risk management program includes effective oversight of model development to ensure adherence to consistent development principles; rigorous and independent model validation; a strong supervisory model governance structure; and annual communication of the state of model risk in the overall program to the Board of Governors. Several aspects of the Federal Reserve's supervisory stress testing program, including its model risk management framework, have been reviewed by external parties.

Most of the models used for supervisory stress testing were developed by Federal Reserve staff, although certain models were developed by third parties.²⁶ In developing the supervisory models, Federal Reserve staff draws on economic research as well as industry practice in modeling the effects of borrower, instrument, collateral characteristics, and macroeconomic factors on revenues, expenses, and losses. Three groups are collectively responsible for managing and validating the Federal Reserve's supervisory stress testing models: the Model Oversight Group (MOG), the System Model Validation unit, and the Supervisory Stress Test Model Governance Committee.

Supervisory model development, implementation, and use is overseen by the MOG, a national committee of senior staff drawn from across the Federal Reserve System. The MOG strives to produce supervisory stress test results that reflect likely outcomes under the supervisory scenarios and ensures that model design across the system of supervisory stress testing models result in projections that are

• from an independent supervisory perspective;
• forward-looking and may incorporate outcomes outside of historical experience, where appropriate;
• based on the same set of models and assumptions across firms;
• generated from simpler and more transparent approaches, where appropriate;
• stable such that changes in model projections over time reflect underlying risk factors, scenarios, and model enhancements, rather than transitory factors;
• appropriately conservative; and
• consistent with the purpose of a stress testing exercise.

In overseeing the development of supervisory models, the MOG considers whether modeling choices and structures adhere to the above principles, reviews the results of common model risk management tools,²⁷ and assesses potential model limitations and sources of uncertainty surrounding final outputs. Assisting the MOG in these efforts is the Model Risk Management Group, which reviews, assesses, and implements industry standards and best practices for model risk management in stress testing operations. This group is composed of Federal Reserve staff and helps set internal policies, procedures, and standards related to the management of model risk stemming from individual models as well as the system of supervisory models used to project post-stress capital ratios. In this way, the Federal Reserve's approach reflects the same standards for model risk management to which banking organizations are expected to adhere.

Each year, the supervisory stress testing models are validated by an independent System Model Validation unit comprised of dedicated full-time staff members not involved in supervisory modeling, supplemented by subject matter experts from across the Federal Reserve System. This group's model validation process includes reviews of model performance and conceptual soundness and reviews of the processes, procedures, and controls used in model development, implementation, and the production of results. For each model, the group assesses, on an annual basis, the model's reliability, based on its underlying assumptions, theory, and methods, and determines whether there are any issues requiring remediation as a result of that assessment. The Model Validation Council, a group of academic experts not affiliated with the Federal Reserve, provides advice to the Federal Reserve on the validation program and activities.²⁸

The MOG and the System Model Validation unit are overseen by the Director of the Federal Reserve Board's Division of Supervision and Regulation. The Supervisory Stress Test Model Governance Committee--a committee of senior Federal Reserve staff that includes representatives from model development, implementation, validation, and scenario design--advises the Director on matters related to the governance of supervisory stress test models and facilitates the Director's oversight role by providing a regular forum to present and discuss relevant issues. This committee also identifies key model risk issues in the supervisory stress testing program and elevates these issues to the Director and the Board of Governors. In 2016, the committee initiated an annual formal communication to the Board of Governors on the structure of the supervisory stress test model risk management program and the state of model risk as determined by each year's model validation process.

The development and validation of the supervisory stress testing models have been subject to rigorous review by both internal and external parties. In 2015, the Federal Reserve Office of the Inspector General (OIG) reviewed supervisory stress testing model validation activities and recommended improvements in staffing, model inventories, and communication with management.²⁹ Each of the suggested improvements recommended by the OIG has been implemented, and the OIG has formally closed its findings. In 2016, the Government Accountability Office (GAO) issued a report on the Federal Reserve's stress testing and capital planning programs.³⁰ The GAO's report recognized that the Federal Reserve's stress testing program has played a key role in evaluating and maintaining the stability of the U.S. financial system during and since the most recent financial crisis. The GAO report included five recommendations as to how the Federal Reserve could improve its management of model risk and ensure that decisions based on supervisory stress test results are informed by an understanding of model risk. The Federal Reserve has already addressed a number of recommendations and continues to enhance the program consistent with other GAO recommendations.

To further enhance the credibility of the supervisory stress test, the Federal Reserve invited comment on a proposal to increase the transparency of the stress testing program in December 2017 (see box 3).

Data Inputs

The models are developed and implemented with data collected by the Federal Reserve on regulatory reports as well as proprietary third-party industry data.

Certain projections rely on aggregate information from the Financial Accounts of the United States (Z.1) statistical release. Others rely on the FR Y-9C report, which contains consolidated income statement and balance sheet information for each firm. Additionally, FR Y-9C includes off-balance sheet items and other supporting schedules, such as the components of RWAs and regulatory capital.

Most of the data used in the Federal Reserve's stress test projections are collected through the Capital Assessments and Stress Testing (FR Y-14A/Q/M) information collection, which include a set of annual, quarterly, or monthly schedules.³¹ These reports collect detailed data on PPNR, loans, securities, trading and counterparty risk, losses related to operational-risk events, and business plan changes. Each of the 35 firms participating in DFAST 2018 submitted data as of December 31, 2017, through the FR Y-14M and FR Y-14Q reports in February, March, and April 2018. The same firms submitted the FR Y-14A reports, which also include projected data, on April 5, 2018.

Firms were required to submit detailed loan and securities information for all material portfolios, where the portfolio is deemed to be material if the size of the portfolio exceeds either 5 percent of the firm's tier 1 capital or $5 billion for LISCC and large and complex firms. Portfolios are deemed to be material for large and noncomplex firms if the size of the portfolio exceeds either 10 percent of the firm's tier 1 capital or $5 billion.³² The portfolio categories are defined in the FR Y-14M and Y-14Q instructions. Each firm has the option to either submit or not submit the relevant data schedule for a given portfolio that does not meet the materiality threshold (as defined in the FR Y-14Q and FR Y-14M instructions). If the firm does not submit data on its immaterial portfolio(s), the Federal Reserve will assign the median loss rate estimated across the set of firms with material portfolios.

While firms are responsible for ensuring the completeness and accuracy of data reported in the FR Y-14 information collection, the Federal Reserve made considerable efforts to validate firm-reported data and requested resubmissions of data where errors were identified. If data quality remained deficient after resubmissions, conservative assumptions were applied to a particular portfolio or specific data, depending on the severity of deficiencies. If the quality of a firm's submitted data was deemed too deficient to produce a supervisory model estimate for a particular portfolio, the Federal Reserve assigned a high loss rate (e.g., 90th percentile) or a conservative PPNR rate (e.g., 10th percentile) to the portfolio balances based on supervisory projections of portfolio losses or PPNR estimated for other firms. If data that are direct inputs to supervisory models were missing or reported erroneously but the problem was isolated in such a way that the existing supervisory framework could still be used, a conservative value (e.g., 10th percentile PPNR rate or 90th percentile loss rate) based on all available data reported by firms was assigned to the specific data. These assumptions are intended to reflect a conservative view of the risk characteristics of the portfolios given insufficient information to make more risk-sensitive projections.

Table 1. Applicable capital ratios and calculations for firms in the 2018 Dodd-Frank Act stress tests

Capital Action Assumptions and Regulatory Capital Ratios

After-tax net income and AOCI are combined with prescribed capital actions to estimate components of regulatory capital. Changes in the regulatory capital components are the primary drivers in changes in capital levels and ratios over the planning horizon. In addition to the regulatory capital components, the calculation of regulatory capital ratios accounts for taxes and items subject to adjustment or deduction in regulatory capital, limits the recognition of certain assets that are less loss-absorbing, and imposes other restrictions as specified in the Board's regulatory revised capital framework.

To project post-stress capital ratios for the Dodd-Frank Act supervisory stress tests, the Federal Reserve uses a standardized set of capital action assumptions that are specified in the Dodd-Frank Act stress test rules. Generally, common stock dividend payments are assumed to continue at the same level as the previous year. Scheduled dividend, interest, or principal payments on any other capital instrument eligible for inclusion in the numerator of a regulatory capital ratio are assumed to be paid, and repurchases of such capital instruments are assumed to be zero.

The capital action assumptions do not include issuances of new common stock or preferred stock, except for issuances related to expensed employee compensation or in connection with a planned merger or acquisition, the extent that the merger or acquisition is reflected in the firm's pro forma balance sheet estimates.³³ The projection of post-stress capital ratios includes capital actions and other changes in the balance sheet associated with any business plan changes under a given scenario.

For the first quarter of the planning horizon, capital actions for each firm are assumed to be the actual actions taken by the firm during that quarter. Over the remaining eight quarters, common stock dividend payments are generally assumed to be the average of the first quarter of the planning horizon and the three preceding calendar quarters.³⁴ Also, firms are assumed to pay scheduled dividend, interest, or principal payments on any other capital instrument eligible for inclusion in the numerator of a regulatory capital ratio. However, repurchases of such capital instruments and issuance of stock are assumed to be zero, except for issuance of common or preferred stock associated with expensed employee compensation or in connection with a planned merger or acquisition.

The five regulatory capital measures in DFAST 2018 are the common equity tier 1, tier 1 risk-based capital, total risk-based capital, tier 1 leverage, and supplementary leverage ratios. A firm's regulatory capital ratios are calculated in accordance with the Board's regulatory capital rules using Federal Reserve projections of assets, RWAs, and off-balance sheet exposures.

The denominator of each firm's regulatory capital ratios, other than the leverage ratios, was calculated using the standardized approach for calculating RWAs for each quarter of the planning horizon in accordance with the transition arrangements in the Board's capital rules.³⁵

 

References

 

 21. CFR part 217. Return to text

 22. A security is considered impaired when the fair value of the security falls below its amortized cost. Return to text

 23. The inclusion of the effects of such expected changes to a firm's business plan does not--and is not intended to--express a view on the merits of such proposals and is not an approval or non-objection to such plans. Return to text

 24. In some cases, the loss models estimated the effect of local-level macroeconomic data, which were projected based on their historical covariance with national variables included in the supervisory scenarios. Return to text

 25. Financial Accounts of the United States (Z.1) is available on the Federal Reserve website at www.federalreserve.gov/releases/z1/. Return to text

 26. A list of providers of the proprietary models and data used by the Federal Reserve in connection with DFAST 2018 is available in appendix B. In some instances, the Federal Reserve relies on firm-provided estimates in place of model output. Return to text

 27. Those tools include the use of benchmark models, where applicable, performance testing, and sensitivity analysis, which isolates the effect of a change in one model input on the eventual model output. Return to text

 28. See "Federal Reserve Board announces the formation of the Model Validation Council," April 20, 2012, https://www.federalreserve.gov/newsevents/pressreleases/bcreg20120420a.htm. Return to text

 29. See The Board Identified Areas of Improvement for Its Supervisory Stress Testing Model Validation Activities, and Opportunities Exist for Further Enhancement, October 29, 2015, https://oig.federalreserve.gov/reports/board-supervisory-stress-testing-model-validation-reissue-oct2015.pdf. Return to text

 30. See Additional Actions Could Help Ensure the Achievement of Stress Test Goals, GAO-17-48, November 2016, https://www.gao.gov/assets/690/681020.pdf. Return to text

 31. The FR Y-14 reports are available on the Federal Reserve website at www.federalreserve.gov/apps/reportforms/default.aspx. Return to text

 32. The Federal Reserve raised the immateriality threshold for large and noncomplex firms from 5 percent of tier 1 capital or $5 billion to 10 percent of tier 1 capital or $5 billion. See Amendments to the Capital Plan and Stress Test Rules, 82 Fed. Reg. 9308 (February 3, 2017), https://www.gpo.gov/fdsys/pkg/FR-2017-02-03/pdf/2017-02257.pdf. Return to text

 33. See 12 CFR 252.56(b). Return to text

 34. Additionally, common stock dividends attributable to issuances related to expensed employee compensation or in connection with a planned merger or acquisition are included to the extent that they are reflected in the firm's pro forma balance sheet estimates. This assumption provides consistency with assumptions regarding issuance of common stock. Return to text

 35. See 12 CFR 252.42(m); 80 Fed. Reg. 75419 (Dec. 2, 2015); 12 CFR part 217, subpart G. Return to text