Qualitative Assessment Framework, Process, and Summary of Results
Overview of Qualitative Assessment Framework
In addition to the quantitative assessment of each firm's capital adequacy discussed above, the Federal Reserve conducted a full review of the capital plans submitted by the LISCC and large and complex firms to assess the strength of each firm's capital planning practices.
In the qualitative assessment, supervisors focus on the firms' analyses and practices used to determine the amount and composition of capital needed to continue to lend to households and businesses throughout a period of severe stress. In doing so, the Federal Reserve evaluates the comprehensiveness and reasonableness of a firm's capital plan; the reasonableness of the assumptions and analysis underlying the plan, including the extent to which it captures and addresses potential risks stemming from firm-wide activities; and the robustness of the firm's capital planning process.19 Where applicable, the assessment leverages existing supervisory information about each firm, such as supervisory findings and information from examinations conducted throughout the year.
Effective capital planning appropriately accounts for firmwide risks and is subject to effective oversight. The Federal Reserve's qualitative assessment of capital plans focuses on the extent to which each firm's analyses supporting its capital plan appropriately captures the specific risks and vulnerabilities faced by the firm under stress. Specifically, the Federal Reserve evaluates how each firm identifies, measures, and determines capital needs for its material risks under both expected and stressful conditions and whether the analyses and practices used provide a reasonable basis for its board of directors to make sound capital planning decisions.
Guidance published in December 2015 provides supervisory expectations for capital planning for firms that are subject to the CCAR qualitative assessment.20 The letter explains that the Federal Reserve's expectations for capital planning processes are tailored based on the size, scope of operations, activities, and systemic importance of the firm. In particular, the Federal Reserve has significantly heightened expectations for LISCC firms and expects them to have the most sophisticated, comprehensive, and robust capital planning processes.
The Qualitative Assessment Process
For LISCC and large and complex firms, the qualitative assessment of the annual CCAR exercise is the culmination of three supervisory activities that evaluate whether firms have sound practices and analyses for determining their capital needs on a forward-looking basis:
- assessment of the underlying analyses and support for firms' annual capital plan submissions,
- monitoring of firms' remediation of outstanding supervisory findings related to capital planning, and
- execution of targeted horizontal exams pertaining to capital planning throughout the year.21
As explained in more detail below, these three evaluations are conducted at different times throughout a given year and together allow the Federal Reserve to gain a comprehensive view into six areas critical to sound capital planning: (1) governance, (2) risk management, (3) internal controls, (4) capital policies, (5) scenario design, and (6) projection methodologies.22 See box 3 for explanations of these areas and examples of past deficiencies.
Box 3. The Importance of Capital Planning and Examples of Historical Deficiencies
Capital is central to a firm's ability to absorb unexpected losses and continue to lend to creditworthy businesses and consumers in times of stress. Firms must have in place sound capital planning practices that allow them to reliably determine their expected capital needs under stress on a forward-looking basis. This allows firms' boards of directors to make informed decisions about capital actions. The practices that are important for sound capital planning are also foundational to a firm's broader risk identification, measurement, and management frameworks.
The emphasis on strong capital planning practices is a direct response to many of the critical shortcomings that were exposed by the financial crisis and hindered firms' ability to effectively manage risk in the face of financial stress. For example, during and immediately following the crisis, a number of firms had significant problems identifying and measuring their risks, which undermined their ability to determine their capital needs. Some of the firms were unable to aggregate their total exposure to their major counterparties and lacked ready access to basic information about the location and value of the collateral they held.
As noted earlier, the Federal Reserve focuses on six key areas for capital planning when assessing a firm's capital planning processes: governance, risk management, internal controls, capital policies, scenario design, and projection methodologies. This box discusses why each area is essential to capital planning and gives examples of historical deficiencies at firms. The deficiencies described in these examples, standing alone, did not result in a qualitative objection. Firms receiving qualitative objections in past CCAR cycles generally had multiple deficiencies in one or more areas of capital planning.
1. Governance
Strong governance in capital planning requires a firm's senior management to design and oversee its capital planning process and its board of directors to periodically review and approve that process. In doing so, senior management must make informed recommendations to the board of directors regarding a firm's capital planning and capital adequacy. The recommendations should have sound analytical support and take into account the expectations of key stakeholders, including shareholders, rating agencies, counterparties, depositors, creditors, and supervisors. In order to make these recommendations, senior management must design and oversee the firm's capital planning process--including its use of models and other estimation approaches--as well as an independent review framework that identifies weaknesses within the capital planning process.
It is the responsibility of the board of directors to ensure that a firm's capital plan is consistent with the firm's strategic direction and its risk appetite. A common element of deficient capital plans has often included failure on the part of management to ensure that the analyses underlying the firm's capital plan was reliable or to accurately communicate the firm's full capital planning practices--including weaknesses therein--to the firm's board of directors.
Example: A firm was found to have deficient governance over capital planning because its senior management failed to inform its board of directors of potential uncertainties and gaps in significant areas of the firm's capital planning practices. In addition, the firm provided information to its board of directors and the Federal Reserve that incorrectly noted the remediation of a previously identified supervisory issue related to capital planning that, in fact, had not been remediated. This action raised significant concerns about management's oversight of the firm's capital planning process and, in turn, the reliability of the grounds upon which the firm made capital decisions.
2. Risk Management
A firm's risk management infrastructure should identify, measure, and assess its material risks, including specifically how they may evolve under stress, and provide a strong foundation for capital planning. A firm's risk identification process should include a comprehensive assessment of risks stemming from its unique business activities and associated exposures. The risk identification process should be dynamic and comprehensive, and drive the firm's capital adequacy analysis. Sound risk measurement processes inform a firm's senior management and board of directors about the size and risk characteristics of exposures faced by the firm under both normal and stressful operating conditions, thereby allowing the firm's leadership to make well-supported decisions about capital needs under stress.
Example: A firm's risk identification process was found to be inadequate for capital planning purposes because it was not integrated with the process used to develop the firm's capital plan. While the firm had a process to identify its material risks, these risks were not included consistently in the firm's stress scenarios or represented in its revenue and loss estimation approaches. As a result, material risks identified by the firm were not factored into the determination of its capital needs under stress.
3. Internal Controls
A firm's internal control framework supports its entire capital planning process. A sound internal control framework should have (a) policies and procedures that support consistent and repeatable processes, (b) validation of estimation methods for suitability, (c) reliable data and information systems, and (d) an internal audit function that oversees the entire capital planning process. This internal audit function should also ensure that appropriate independent review is occurring at all key levels within the capital planning process. A sound internal control framework helps ensure that all aspects of the capital planning process are functioning as designed and result in sound assessments of the firm's capital needs.
Example: A firm's internal controls were found to be inadequate because the process for estimating total losses was highly manual, without appropriate controls. This made it difficult to compile and verify final results, and led to fundamental errors in the firm's capital plan. This weak control environment rendered the firm's capital plan unreliable and led to its board of directors making capital distribution decisions based on incorrect information.
4. Capital Policy
A capital policy is a firm's written description of the principles and guidelines used for capital planning, issuance, usage, and distributions. The capital policy should reflect a number of factors, including the firm's business strategy, risk appetite, organizational structure, governance structure, post-stress capital goals, and real-time targeted capital levels. It should also establish the actions the firm will take in the event of breaching a post-stress capital goal, real-time targeted capital level, or early warning metric. A sound capital policy underpins the creation of post-stress capital goals that are aligned with a firm's risk appetite and risk profile. It is also critical to a firm's ability to appropriately manage its capital adequacy under normal circumstances and continue to be able to lend during times of stress. Prior to the crisis, most firms did not have forward-looking capital policies to guide their response to deteriorating financial conditions.
Example:A firm was found to have a deficient capital policy because the policy lacked detail in critical areas. The policy did not establish capital limits that were supported by forward-looking analysis of the firm's risks or considered the capital the firm needed to maintain the confidence of its counterparties. The capital policy also did not set forth the actions the firm could take to improve its capital position. These weaknesses inhibited the firm's senior management and board of directors from proactively addressing capital shortfalls.
5. Scenario Design
Scenario design entails creating a hypothetical economic environment over a specific period of time, including both a narrative of the situation and paths of economic variables that relate to the scenario. Well-designed scenarios should incorporate appropriately stressful conditions and events that could adversely affect a firm's capital adequacy. Firm-specific scenarios should reflect the specific vulnerabilities of the firm and directly link to the firm's risk-identification process and associated risk assessment. Scenario design is essential to testing the range of potential outcomes a firm could face in stress and contributes to informed capital planning processes.
Example: A firm's risk identification process was found to be inadequate because of its failure to capture unique risks arising from a portfolio material to its business. The firm was overly reliant upon events from the financial crisis in designing its stress scenarios, despite material changes in its risk profile and business mix since that time. As a result, this process resulted in a stress scenario that was not particularly stressful or applicable to the firm in its current state and, therefore, did not provide a useful means of determining capital adequacy.
6. Projection Methodologies
Forward-looking capital planning requires a firm to make projections of its future capital needs. In doing so, a firm should estimate losses, revenues, expenses, and capital using a sound method that relates macroeconomic and other risk factors to its projections. The firm should be able to identify the manner in which key variables, factors, and events in a scenario affect losses, revenue, expenses, and capital over the planning horizon. Sound projection methodologies allow a firm's senior management and board of directors to make appropriate, informed decisions regarding the firm's capitalization. Deficient projection methodologies may also be evidence of weak internal controls, such as model risk management.
Example: A firm's capital plan was found to be deficient because the models used to estimate losses for one of the firm's most material portfolios did not sufficiently capture relevant risk drivers, were based on unsupported assumptions, and used very limited data. The resulting models were not sensitive to the firm's risk characteristics and scenario conditions. These weaknesses raised significant concerns about the conceptual soundness of these methodologies and the loss estimates resulting from them. As a result, management of the firm was unable to provide reliable loss projections on a major portfolio to its board of directors, and the board of directors was unable to make informed decisions about capital adequacy at the firm.
Assessment of Capital Plan Submissions
In April of each year as a part of the CCAR exercise, firms submit to the Federal Reserve capital plans that include detailed descriptions of the firms' capital planning practices and underlying analyses, including descriptions of their internal processes for assessing capital adequacy and their policies governing capital actions. Those plans are then assessed by subject matter experts from across the Federal Reserve System over a three-month period. The assessment is also informed by related supervisory work conducted throughout the year.
Two groups of supervisors--dedicated supervisory teams (DSTs) and horizontal evaluation teams (HETs)--conduct an initial assessment of each firm's capital plan submission. DSTs, which are composed of Federal Reserve staff that focus on a single firm, assess the adequacy of firms' capital planning practices related to governance, risk management, internal controls, and scenario design. HETs are composed of Federal Reserve staff that are not assigned to a specific financial institution for purposes of the CCAR annual exercise but instead focus on the examination of practices across multiple firms. Some HETs assess the reasonableness of firms' stressed loss, revenue, and expense estimation approaches and the governance and controls around those approaches. Others, such as the capital planning review team, work closely with DSTs to provide a horizontal assessment across the DSTs' areas of focus.
The DST and HET assessments consider whether a firm's capital planning practices allow it to reliably estimate its capital needs on a forward-looking basis, given dynamic changes that can occur to a firm's risk profile. These assessments are based on previously articulated supervisory guidance and expectations. The horizontal element of the exercise assists the Federal Reserve in consistently applying its supervisory expectations to its assessment of each firm's capital planning practices.
After this initial assessment, the DSTs and HETs rate each firm's practices in each of the six areas noted above. These ratings, which indicate the extent to which a firm's capital planning practices meet previously communicated supervisory expectations, are used to determine the nature and severity of supervisory feedback. The initial supervisory assessments are subject to review by a committee comprising senior staff from across the Federal Reserve System that seek to confirm that
- evaluations are aligned with the supervisory expectations communicated to the industry;
- evaluations are well supported and are consistently applied across firms accounting for their size and complexity; and
- assessments, as reflected in the ratings, are appropriately calibrated to the materiality of the supervisory concern.
This committee also ranks firms based on the ratings for each assessment area, with consideration of the firms' individual risk profiles. The rankings assist the Federal Reserve in distinguishing the relative strength of each firm's capital planning practices and facilitating the consistent application of supervisory guidance across firms. However, the qualitative assessment of a firm's capital plan is based on an absolute assessment of an individual firm's capital planning practices relative to the Federal Reserve's expectations as set forth in SR letter 15-18 and not on comparative rankings. As such, a low ranking is not, in and of itself, a reason for an objection to a capital plan.
The DSTs formulate a recommendation to object or not object to a firm's capital plan based on the combined assessment, after extensive review by the national committee. The LISCC's Operating Committee, which comprises senior staff from across the Federal Reserve System, then reviews and presents its own recommendation for each LISCC firm to the director of the Board's Division of Supervision and Regulation.23 Reserve Banks responsible for the supervision of large and complex firms that are not LISCC firms make recommendations with regard to those firms, after review by a separate committee of senior staff. The director makes the final recommendations, with supervisory findings, to the Board of Governors, who makes the final decision whether to object to a firm's capital plan.
Objections on qualitative grounds can arise for reasons including, but not limited to
- unresolved material supervisory issues;
- inappropriate assumptions and analyses underlying a firm's capital plan; or
- inadequate governance and internal controls, risk management and risk identification in support of a firm's capital planning practices.24
Communication of Feedback
Soon after the completion of the CCAR exercise, whether a firm's capital plan is objected to or not, the Federal Reserve sends a letter to each firm, detailing weaknesses in the firm's capital planning analyses and processes and any actions these firms must take to remediate those weaknesses. Each firm is required to submit a plan detailing how it will address the identified weaknesses, and supervisors then assess whether those plans are likely to address those weaknesses in a reasonable period of time. The Federal Reserve then communicates its evaluation of the action plans to the firm. In this way, the feedback letters serve as a guide for firms and supervisors to develop a common understanding of how supervisory concerns will be remediated.
Monitoring Outstanding Findings
DSTs and HETs monitor each firm's progress in remediating outstanding supervisory findings consistent with the firm's remediation plan. Any resulting concerns are communicated to firms on an ongoing basis so that changes, if needed, can be made by the firm before the next CCAR exercise. The annual process is meant to give firms regular feedback so they know the issues they face--before, during, and after the CCAR qualitative assessment--and can make improvements throughout the year.
Horizontal Examinations
Horizontal examinations are assessments of a common area or practice (such as internal audit) across multiple firms by a coordinated team of examiners. Throughout the year, the Federal Reserve conducts horizontal examinations aimed at assessing whether firms have sound capital planning practices in place to enable them to reliably determine their capital needs under expected and stressful conditions. The focus of a given year's capital planning horizontal examinations are determined in the fall of each year, and findings from the exams serve as key inputs for the annual CCAR qualitative assessment.
Horizontal examinations conducted since CCAR 2016 have included firms' internal audit coverage of capital planning processes, the models and other approaches used to determine counterparty credit exposure under stress, pricing models used to estimate stressed trading losses, and approaches used to rate the credit risk of obligors and facilities for wholesale loans.
Qualitative Assessment Results
The Federal Reserve did not object to any firm's capital plan on qualitative grounds.
The Federal Reserve issued a conditional non-objection to Capital One's capital plan. The firm must submit a capital plan addressing weaknesses identified in its capital planning practices by December 28, 2017.
Qualitative Assessment Results
The qualitative assessment conducted as part of CCAR 2017 found that many firms continued to improve their capital planning practices, both in terms of the estimation methods used to conduct their stress tests and the risk measurement and management, internal controls, and governance supporting the firms' capital planning practices. Strong practices in these areas are critical to ensuring the integrity of the inputs into assessing capital adequacy and making decisions about capital distributions.
Most of the largest firms have made progress since CCAR 2016, though some firms continue to fall short of meeting supervisory expectations. This group of firms has made progress addressing many of their past capital planning weaknesses, including those relating to the way they identify and assess capital needs for their unique risks. These firms are making decisions based on more reliable post-stress capital assessments, which should strengthen their overall safety and soundness.
However, some of these firms continue to fall short of supervisory expectations in the following areas:
- identification of risks associated with firms' products and services that may materialize under stressed conditions, particularly risks stemming from the introduction of new products or changes in underwriting standards;
- use of loss estimation approaches that may be appropriate to use under expected conditions but are not suitable for stress testing, as they may materially underestimate losses in stressful conditions when the relationships between risk drivers and losses diverge from the relationships under expected conditions; and
- controls around data accuracy, model risk management, and internal audit, which are foundational areas of capital planning.
Reasons for Conditional Non-objection
The Board of Governors did not object to Capital One's capital plan. However, Capital One exhibited material weaknesses in its capital planning practices, which warrant further near-term attention. Notable weaknesses were identified in the oversight and execution of the firm's capital planning practices, which undermined the reliability of the firm's forward-looking assessment of its capital adequacy under stress. Specifically, the firm's capital plan did not appropriately take into account the potential impact of the risks in one of its most material businesses. Further, the firm's internal controls functions, including independent risk management, did not identify these material weaknesses in the firm's capital planning practices. Therefore, senior management was not in a position to provide the firm's board of directors with a reliable assessment upon which to determine the reasonableness of the capital plan.
As a condition of not objecting to Capital One's capital plan, the Board of Governors is requiring the firm to address the weaknesses noted above and resubmit its capital plan by December 28, 2017. If Capital One does not satisfactorily address the identified weaknesses in its capital planning practices by that time, the Board of Governors would expect to object to the resubmitted capital plan and may restrict the firm's capital distributions.
References
19. 12 CFR 225.8(f)(1). Return to text
20. See SR letter 15-18, "Federal Reserve Assessment of Capital Planning and Positions for LISCC Firms and Large and Complex Firms," December 18, 2015, www.federalreserve.gov/supervisionreg/srletters/sr1518.htm. Return to text
21. Horizontal examinations are assessments of a common area or practice (such as internal audit) across multiple firms by a coordinated team of examiners. Return to text
22. Ibid. Return to text
23. See SR letter 15-7. "Governance Structure of the Large Institution Supervision Coordinating Committee (LISCC) Supervisory Program," April 17, 2015, www.federalreserve.gov/supervisionreg/srletters/sr1507.htm. Return to text
24. For further information on the qualitative grounds upon which capital plans may be objected, see "Box 2. Considerations for Capital Plan Qualitative Assessments" of Board of Governors of the Federal Reserve System, Comprehensive Capital Analysis and Review 2016: Assessment Framework and Results (Washington: Board of Governors, June 2016), 9, www.federalreserve.gov/newsevents/pressreleases/files/bcreg20160629a1.pdf. Return to text