Qualitative Assessment Framework, Process, and Summary of Results
Overview of Qualitative Assessment Framework
In addition to the quantitative assessment of each firm's capital adequacy discussed above, the Federal Reserve reviewed capital plans submitted by the LISCC and large and complex firms to assess the strength of each firm's capital planning practices.
In the qualitative assessment, supervisors focus on the firms' analyses and practices used to determine the amount and composition of capital needed to continue to lend to households and businesses throughout a period of severe stress. In doing so, the Federal Reserve evaluates the comprehensiveness and reasonableness of a firm's capital plan; the reasonableness of the assumptions and analysis underlying the plan, including the extent to which it captures and addresses potential risks stemming from firmwide activities; and the robustness of the firm's capital planning process.19 Where applicable, the assessment leverages existing supervisory information about each firm, such as supervisory findings and information from examinations conducted throughout the year. Effective capital planning appropriately accounts for firmwide risks and is subject to effective oversight. The Federal Reserve's qualitative assessment of capital plans focuses on the extent to which each firm's analyses supporting its capital plan appropriately capture the specific risks and vulnerabilities faced by the firm under stress. Specifically, the Federal Reserve evaluates how each firm identifies, measures, and determines capital needs for its material risks under both expected and stressful conditions and whether the analyses and practices used provide a reasonable basis for its board of directors to make sound capital planning decisions.
Guidance published in December 2015 provides further supervisory expectations for capital planning for firms that are subject to the CCAR qualitative assessment.20 The letter explains that the Federal Reserve's expectations for capital planning processes are tailored based on the size, scope of operations, activities, and systemic importance of the firm. In particular, the Federal Reserve has heightened expectations for LISCC firms and expects them to have the most sophisticated, comprehensive, and robust capital planning processes.
Box 3. Scope of CCAR 2018 Qualitative Assessment
The CCAR 2018 qualitative assessment was focused more narrowly than in past years. The 2018 assessment focused on identification of material risks, firms' internal scenario design and consideration of idiosyncratic risk, and certain stress loss and revenue estimation practices, summarized in table A. Firms were only required to submit documentation for those elements in the in-scope areas. The more targeted scope of review reduced the burden of submitting supporting documentation and required fewer meetings between examiners and firm management than in previous years. In addition, the narrower focus of the assessment allowed examiners to spend more time testing the assumptions and analysis underlying firms' capital plans.
Table A. Scope of CCAR 2018 qualitative assessment
| Areas | Exposure types | |||||
|---|---|---|---|---|---|---|
| Retail credit risk | Wholesale credit risk |
Counterparty credit risk |
Trading risk | Securities and net interest income | Non-interest income and expense | |
| Risk management | All | All | All | All | All | All |
| Internal controls--model validation and independent review of estimation approaches | (1) Credit cards; and (2) automobile loans |
(1) Commercial & industrial loans and other commercial loans and leases held for investment; and (2) commercial & industrial loans, other commercial loans and leases, and commercial real estate loans held-for-sale |
All | (1) Interest rate products; (2) foreign exchange products; (3) equities; and (4) commercial mortgage-backed securities |
(1) Credit cards; (2) commercial & industrial loans; (3) all deposit products; and (4) securities loss estimation |
Sales and trading |
| Incorporating stressful conditions and events | All | All | All | All | All | All |
| Estimating impact on capital positions | (1) Credit cards; and (2) automobile loans |
(1) Commercial & industrial loans and other commercial loans and leases held for investment; and (2) commercial & industrial loans, other commercial loans and leases, and commercial real estate loans held-for-sale |
All | (1) Interest rate products; (2) foreign exchange products; (3) equities; and (4) commercial mortgage-backed securities |
(1) Credit cards; (2) commercial & industrial loans; (3) all deposit products; and (4) securities loss estimation |
Sales and trading |
The Qualitative Assessment Process
For LISCC and large and complex firms, CCAR's qualitative assessment is the culmination of three supervisory activities that evaluate whether firms have sound practices and analyses for determining their capital needs on a forward-looking basis:
- assessment of the underlying analyses and support for firms' annual capital plan submissions,
- monitoring of firms' remediation of outstanding supervisory findings related to capital planning, and
- execution of targeted horizontal exams pertaining to capital planning undertaken throughout the year.21
As explained in more detail below, these three evaluations are conducted at different times throughout a given year and together allow the Federal Reserve to gain a comprehensive view into six areas critical to sound capital planning: (1) governance, (2) risk management, (3) internal controls, (4) capital policies, (5) scenario design, and (6) projection methodologies.22 See box 4 for explanations of these areas and examples of past deficiencies.
Box 4. The Importance of Capital Planning and Examples of Historical Deficiencies
Capital is central to a firm's ability to absorb unexpected losses and continue to lend to creditworthy businesses and consumers in times of stress. Firms must have in place sound capital planning practices that allow them to reliably determine their expected capital needs under stress on a forward-looking basis. This allows firms' boards of directors to make informed decisions about capital actions. The practices that are important for sound capital planning are also foundational to a firm's broader risk identification, measurement, and management frameworks.
The emphasis on strong capital planning practices is a direct response to many of the critical shortcomings that were exposed by the financial crisis and hindered firms' ability to effectively manage risk in the face of financial stress. For example, during and immediately following the crisis, a number of firms had significant problems identifying and measuring their risks, which undermined their ability to determine their capital needs. Some of the firms were unable to aggregate their total exposure to their major counterparties and lacked ready access to basic information about the location and value of the collateral they held.
As noted earlier, the Federal Reserve focuses on six key areas for capital planning when assessing a firm's capital planning processes: governance, risk management, internal controls, capital policies, scenario design, and projection methodologies. This box discusses why each area is essential to capital planning and gives examples of historical deficiencies at firms. The deficiencies described in these examples, standing alone, did not result in a qualitative objection. Firms that received qualitative objections in past CCAR cycles generally had multiple deficiencies in one or more areas of capital planning.
1. Governance
Strong governance in capital planning requires a firm's senior management to design and oversee its capital planning process and its board of directors to periodically review and approve that process. In doing so, senior management should make informed recommendations to the board of directors regarding a firm's capital planning and capital adequacy. These recommendations should have sound analytical support and take into account the expectations of key stakeholders, including shareholders, rating agencies, counterparties, depositors, creditors, and supervisors. In order to make these recommendations, senior management should design and oversee the firm's capital planning process--including its use of models and other estimation approaches--as well as an independent review framework that identifies weaknesses within the capital planning process.
It is the responsibility of the board of directors to ensure that a firm's capital plan is consistent with the firm's strategic direction and its risk appetite. A common element of deficient capital plans has been the failure of management to ensure that the analyses underlying the firm's capital plan were reliable or to accurately communicate the firm's full capital planning practices--including weaknesses therein--to the firm's board of directors.
Example: A firm was found to have deficient governance over capital planning because its senior management presented and its board of directors approved a capital plan that did not accurately represent the firm's expected financial condition or account for the material risks it faced. This resulted from the firm's management team failing to effectively account in the capital plan for increased risks stemming from weakened credit underwriting standards in connection with its most material portfolios, despite learning of those risks and related underestimation of their reserves shortly before their capital plan was submitted to the Federal Reserve. This occurrence raised significant concerns about management's oversight of the firm's capital planning process and, in turn, the reliability of the grounds upon which the firm's board of directors made capital decisions.
2. Risk Management
A firm's risk management infrastructure should identify, measure, and assess its material risks, including specifically how they may evolve under stress, and should provide a strong foundation for capital planning. A firm's risk identification process should include a comprehensive assessment of risks stemming from its unique business activities and associated exposures. The risk identification process should be dynamic and comprehensive, and drive the firm's capital adequacy analysis. Sound risk measurement processes inform a firm's senior management and board of directors about the size and risk characteristics of exposures faced by the firm under both normal and stressful operating conditions, thereby allowing the firm's leadership to make well-supported decisions about capital needs under stress.
Example: A firm's risk identification process was found to be inadequate for capital planning purposes because it was not integrated with the process used to develop the firm's capital plan. While the firm had a process to identify its material risks, these risks were not included consistently in the firm's stress scenarios or represented in its revenue and loss estimation approaches. As a result, material risks identified by the firm were not factored into the determination of its capital needs under stress.
3. Internal Controls
A firm's internal control framework supports its entire capital planning process. A sound internal controlframework should have (a) policies and procedures that support consistent and repeatable processes, (b) validation of estimation methods for suitability, (c) reliable data and information systems, and (d) an internal audit function that independently evaluates the efficacy of the capital planning process. A sound internal control framework helps ensure that all aspects of the capital planning process are functioning as designed and result in sound assessments of the firm's capital needs.
Example: A firm's internal controls were found to be inadequate because the process for estimating total losses was highly manual, without appropriate controls. This made it difficult to compile and verify final results, and led to fundamental errors in the firm's capital plan. This weak control environment rendered the firm's capital plan unreliable and led to its board of directors making capital distribution decisions based on incorrect information.
4. Capital Policy
A capital policy is a firm's written description of the principles and guidelines used for capital planning, issuance, usage, and distributions. The capital policy should reflect a number of factors, including the firm's business strategy, risk appetite, organizational structure, governance structure, post-stress capital goals, and real-time targeted capital levels. It should also establish the actions the firm will take in the event of breaching a post-stress capital goal, real-time targeted capital level, or early warning metric. A sound capital policy underpins the creation of post-stress capital goals that are aligned with a firm's risk appetite and risk profile. It is also critical to a firm's ability to appropriately manage its capital adequacy under normal circumstances and continue to be able to lend during times of stress. Prior to the crisis, most firms did not have forward-looking capital policies to guide their response to deteriorating financial conditions.
Example:A firm was found to have a deficient capital policy because the policy lacked detail in critical areas. The policy did not establish capital limits that were supported by forward-looking analysis of the firm's risks or considered the capital the firm needed to maintain the confidence of its counterparties. The capital policy also did not set forth the actions the firm could take to improve its capital position. These weaknesses inhibited the firm's senior management and board of directors from proactively addressing capital shortfalls.
5. Scenario Design
Scenario design entails creating a hypothetical economic environment over a specific period of time, including both a narrative of the situation and paths of economic variables that relate to the scenario. Well-designed scenarios should incorporate appropriately stressful conditions and events that could adversely affect a firm's capital adequacy. Firm-specific scenarios should reflect the specific vulnerabilities of the firm and directly link to the firm's risk-identification process and associated risk assessment. Scenario design is essential to testing the range of potential outcomes a firm could face in stress and contributes to informed capital planning processes.
Example: A firm's scenario design process was found to be inadequate because it did not incorporate its unique risks and business activities into its stress scenario design. The firm was overly reliant upon events from the financial crisis in designing its stress scenarios, despite material changes in its risk profile and business mix since that time. As a result, this process resulted in a stress scenario that was not particularly stressful or applicable to the firm in its current state and, therefore, did not provide a useful means of determining capital adequacy.
6. Projection Methodologies
Forward-looking capital planning requires a firm to make projections of its future capital needs. In doing so, a firm should estimate losses, revenues, expenses, and capital using a sound method that relates macroeconomic and other risk factors to its projections. The firm should be able to identify the manner in which key variables, factors, and events in a scenario affect losses, revenue, expenses, and capital over the planning horizon. Sound projection methodologies allow a firm's senior management and board of directors to make appropriate, informed decisions regarding the firm's capitalization. Deficient projection methodologies may also be evidence of weak internal controls, such as model risk management.
Example: A firm's capital plan was found to be deficient because the models used to estimate losses for one of the firm's most material portfolios did not sufficiently capture relevant risk drivers, were based on unsupported assumptions, and used very limited data. The resulting models were not sensitive to the firm's risk characteristics and scenario conditions. These weaknesses raised significant concerns about the reliability of these methodologies and the loss estimates resulting from them. As a result, management of the firm was unable to provide reliable loss projections on a major portfolio to its board of directors, and the board of directors was unable to make informed decisions about capital adequacy at the firm.
Assessment of Capital Plan Submissions
In April of each year as a part of the CCAR exercise, firms submit to the Federal Reserve capital plans that include detailed descriptions of the firms' capital planning practices and underlying analyses, including descriptions of their internal processes for assessing capital adequacy and their policies governing capital actions. Those plans are then assessed by subject matter experts from across the Federal Reserve System over a three-month period. The assessment is also informed by related supervisory work conducted throughout the year and remediation of outstanding capital planning-related supervisory findings.
Two groups of supervisors--dedicated supervisory teams (DSTs) and horizontal evaluation teams (HETs)--conduct an initial assessment of each firm's capital plan submission. DSTs, which are composed of Federal Reserve staff that focus on a single firm, assess the adequacy of firms' capital planning practices related to governance, risk management, internal controls, and scenario design. HETs are composed of Federal Reserve staff that are not assigned to a specific financial institution for purposes of the CCAR annual exercise but instead focus on the examination of practices across multiple firms. Some HETs assess the reasonableness of firms' stressed loss, revenue, and expense estimation approaches and the governance and controls around those approaches. Other HETs, such as the capital planning review team, work closely with DSTs to provide a horizontal assessment across the DSTs' areas of focus.
The DST and HET assessments consider whether a firm's capital planning practices allow it to reliably estimate its capital needs on a forward-looking basis, given dynamic changes that can occur to a firm's risk profile. These assessments are based on previously articulated supervisory guidance and expectations. The horizontal element of the exercise assists the Federal Reserve in consistently applying its supervisory expectations to its assessment of each firm's capital planning practices.
After this initial assessment, the DSTs and HETs rate each firm's practices in each of the six areas noted above. These ratings, which indicate the extent to which a firm's capital planning practices meet previously communicated supervisory expectations, are used to determine the nature and severity of supervisory feedback. The initial supervisory assessments are subject to review by a national committee comprising senior staff from across the Federal Reserve System that seek to confirm that
- evaluations are aligned with the supervisory expectations communicated to the industry;
- evaluations are well supported and are consistently applied across firms accounting for their size and complexity; and
- assessments, as reflected in the ratings, are appropriately calibrated to the materiality of the supervisory concern.
This committee also groups firms based on the ratings for each assessment area, with consideration of the firms' individual risk profiles. The groupings assist the Federal Reserve in facilitating the consistent application of supervisory guidance across firms. However, the qualitative assessment of a firm's capital plan is based on an absolute assessment of an individual firm's capital planning practices relative to the Federal Reserve's expectations as set forth in SR Letter 15-18 and not on comparative groupings. As such, a low grouping is not, in and of itself, a reason for an objection to a capital plan.
The DSTs formulate a recommendation to object or not object to a firm's capital plan based on the combined assessment, after extensive review by the national committee. The LISCC's Operating Committee, which comprises senior staff from across the Federal Reserve System, then reviews and presents its own recommendation for each LISCC firm to the director of the Board's Division of Supervision and Regulation.23 Reserve Banks responsible for the supervision of large and complex firms that are not LISCC firms make recommendations with regard to those firms, after review by a separate committee of senior staff. The director makes the final recommendations, with supervisory findings, to the Board of Governors, which makes the final decision whether to object to a firm's capital plan.
Objections on qualitative grounds can arise for reasons including, but not limited to
- unresolved material supervisory issues;
- inappropriate assumptions and analyses underlying a firm's capital plan; or
- inadequate governance and internal controls, risk management and risk identification in support of a firm's capital planning practices.24
Communication of Feedback
Soon after the completion of the CCAR exercise, whether a firm's capital plan is objected to or not, the Federal Reserve sends a letter to each firm, noting areas where the firm's capital planning analyses and processes meet supervisory expectations, or exhibit weaknesses, and actions the firms must take to address any weaknesses. Each firm is required to submit a plan detailing how it will address any identified weakness, and supervisors then assess whether those plans are likely to address the cited weaknesses in a reasonable period of time. The Federal Reserve then communicates its evaluation of the action plans to the firm. In this way, the feedback letters serve as a guide for firms and supervisors to develop a common understanding of how supervisory concerns will be remediated.
Monitoring Outstanding Findings
DSTs and HETs monitor each firm's progress in remediating outstanding supervisory findings consistent with the firm's remediation plan. Any resulting concerns are communicated to firms on an ongoing basis so that changes, if needed, can be made by the firm before the next CCAR exercise. The annual process is meant to give firms regular feedback so they know the issues they face--before, during, and after the CCAR qualitative assessment--and can make improvements throughout the year.
Horizontal Examinations
Horizontal examinations are assessments of a common area or practice (such as internal audit) across multiple firms by a coordinated team of examiners. Throughout the year, the Federal Reserve conducts horizontal examinations aimed at assessing whether firms have sound capital planning practices in place to enable them to reliably determine their capital needs under expected and stressful conditions. The focus of a given year's capital planning horizontal examinations are determined in the fall of each year, and findings from the exams serve as key inputs for the annual CCAR qualitative assessment.
Qualitative Assessment Results
The Federal Reserve objected to the capital plan of DB USA Corporation on qualitative grounds based on material weaknesses in capital planning.
Qualitative Assessment Results
The qualitative assessment conducted as part of CCAR 2018 found that most firms either meet or are close to meeting the Federal Reserve's supervisory expectations for capital planning. In particular, most firms' revenue and loss estimation approaches have matured and generally result in credible estimates that inform capital adequacy assessments. These advances have resulted from those firms improving the methods they use to identify their unique risks, using sound practices for identifying and addressing model deficiencies, and appropriately relying upon the results of capital stress testing to evaluate their capital positions on a forward-looking basis.
This year's qualitative assessment revealed a number of trends in capital planning practices, including the following:
- Certain factors, including changing market dynamics, lack of access to relevant data, and weak judgement-based estimation approaches, make it difficult for firms to reliably forecast stressed losses or revenues in certain areas, such as credit cards, auto loans, and revenues from certain business lines. Some firms are able to use appropriate techniques to address these deficiencies, while others struggle to do so.
- Several firms with trading portfolios purchased large trading positions to offset the losses arising from the instantaneous market shock. This practice can raise concerns if the risks arising from these strategies, such as counterparties' willingness to make such positions available during periods of market stress, are not sufficiently analyzed, represented in the firms' capital plans, or conveyed to their boards of directors.
- In many instances, firms' internal controls around capital planning continue to fall below supervisory expectations for various reasons, including insufficient investment in information systems and data management efforts, poorly constructed and/or executed capital planning audits, and ineffective model risk management functions.
Reasons for Qualitative Objection
The Board of Governors objected to the capital plan of DB USA Corporation because of widespread and critical deficiencies across the firm's capital planning practices. Material weaknesses were identified in data capabilities and controls supporting the firm's capital planning process; in approaches and assumptions used to forecast revenues and losses arising from many of its key business lines and exposures under stress; and in the firm's risk management functions, including model risk management and internal audit. Together, these weaknesses raise concerns about DB USA's ability to effectively determine its capital needs on a forward-looking basis.
References
19. CFR 225.8(f)(1). Return to text
20. See SR Letter 15-18, "Federal Reserve Assessment of Capital Planning and Positions for LISCC Firms and Large and Complex Firms," December 18, 2015, www.federalreserve.gov/supervisionreg/srletters/sr1518.htm. Return to text
21. Horizontal examinations are assessments of a common area or practice (such as internal audit) across multiple firms by a coordinated team of examiners. Return to text
22. Ibid. Return to text
23. See SR Letter 15-7. "Governance Structure of the Large Institution Supervision Coordinating Committee (LISCC) Supervisory Program," April 17, 2015, www.federalreserve.gov/supervisionreg/srletters/sr1507.htm. Return to text
24. For further information on the qualitative grounds upon which capital plans may be objected, see "Box 2. Considerations for Capital Plan Qualitative Assessments" of Board of Governors of the Federal Reserve System, Comprehensive Capital Analysis and Review 2016: Assessment Framework and Results (Washington: Board of Governors, June 2016), 9, www.federalreserve.gov/newsevents/pressreleases/files/bcreg20160629a1.pdf. Return to text