Inside the world of ESG ratings: Academic paper examines several process and outcome issues

“Recently, ESG rating providers have come under intense scrutiny over concerns about the reliability of their assessments,” write Stanford University scholars David Larcker and Brian Tayan in an article co-authored with Edward Watts from the Yale School of Management and Lukasz Pomorski from AQR Capital Management.

In their publication – a “closer review” of many studies on ESG ratings – they examine these concerns, while naming the document ESG ratings: A compass without direction provides a fairly clear view of their findings.

“We find that while ESG rating providers can convey important information about the non-financial impact of companies, significant gaps exist in their objectives, methodologies, and incentives, which undermine the informativeness of their ratings,” the authors write. .

“Unfortunately, it is rare for ratings providers to offer concrete, systematic evidence to back up claims about their ratings.”

The researchers say they were looking for answers to the following questions:

  • Why do ESG ratings often fail to achieve their stated goals? Is it due to the methodological choices of these companies or to the simple challenge of measuring a concept as broad and encompassing as ESG?
  • Are fund managers sufficiently motivated to ensure that ESG ratings are reliable in predicting risk and performance? What steps do they take to validate the ratings before using them?
  • Despite their weaknesses, do ESG ratings have a role to play for reporting and compliance purposes?
  • Would broader company disclosure improve the quality of ESG ratings? Is it even possible for companies to disclose the vast number of stakeholder-related metrics that feed into ESG ratings?
  • Should the SEC establish policies, procedures and safeguards to reduce conflicts of interest and improve market confidence in the quality of ESG ratings?

The 16-page document is a comprehensive overview of past research highlighting many gaps in the ESG rating process, from investors’ understanding of what ratings actually mean to conflicts of interest that see “sister companies” diverging. a top rated provider. Here, IR magazine presents a selection of results.

The large cap effect

The authors of the article write that “systemic patterns are observed in ESG ratings” and one of these patterns is related to company size: large companies receive higher average scores than small companies.

“This may be due to the greater resources larger companies are able to invest in ESG initiatives, or it may be due to larger companies having greater disclosure of ESG data,” the authors write. “A second trend is industry-related: while some ESG ratings are industry-adjusted, those that are not may have higher average scores for certain industries (such as banking and wireless communications ) than for others (such as tobacco and gambling).

‘It’s not clear [whether] these patterns are due to fundamental differences in ESG quality between sectors, or result from methodological choices and input variables that underlie ESG rating models.

Geography also has an impact. The researchers find that European companies have higher average ESG scores than American companies, which they believe could be due to policy and regulatory differences between countries. “Companies in emerging markets also have lower ratings than companies in more developed economies,” they add.

An upward drift

Research shows ‘upward drift’ in ESG ratings over time. A study cited by the researchers, which analyzes the aggregate ESG scores of all Russell 1000 companies as calculated by MSCI between January 2015 and December 2021, finds an overall improvement of 18% over the measurement period.

A number of factors – such as changes in the composition of the index as higher rated companies (researchers use Microsoft as an example) rise to represent a higher percentage of the total index, changes in weightings MSCI or increased company disclosure that a company’s underlying performance has in fact changed – could all account for some of this upward drift.

However, after adjusting for these structural changes, MSCI ratings were still subject to an overall adjusted improvement of 12% that could not be explained.

A separate review of MSCI ratings by Bloomberg finds that most upgrades happen for what Bloomberg calls “rudimentary business practices” rather than substantial improvements, the paper’s authors note. In justifying 155 upgrades, MSCI cites governance improvements almost half (42%) of the time – far more than social (32%) or environmental (26%) improvements.

“Upgrades were often driven by tick-box practices, such as conducting a survey of employees likely to reduce turnover, and rarely for substantive practices, such as an actual reduction in carbon emissions “, note the researchers. “Half of the companies were outclassed for doing nothing”, simply as a “result of methodological changes”.

ESG rating providers disagree

“Studies find low correlations between ESG rating providers,” the paper’s authors write, adding that “it is perhaps surprising that ESG ratings are meant to measure the same construct.”

Reviewing a number of research papers that have examined the issue of weak correlations, the paper’s authors note:

  • Differences in measurement (56%) and scope (38%) explain most of the discrepancy, with weighting differences account for only 6% of the variance. “This illustrates how fundamental methodological differences are between companies,” the authors write.
  • Company disclosure does not reduce ESG rating divergence but instead increases it – which is attributed to the subjective nature of ESG information and the fact that disclosure widens the possibilities for different interpretations of the information. This suggests that stricter environmental and social data disclosure requirements by companies might not lead to more consistent ESG ratings, according to the authors of the paper. In fact, ESG ratings can be similar to stock analyst ratings, where the rating ultimately depends on the interpretation of information rather than its availability.

Ratings don’t predict returns – or even ESG performance

What about the purpose of ESG ratings? Whether designed to show investors the impact the world will have on business performance, the impact of the business on the world or to be used as an indicator of the future financial Performance For those who believe that better ESG practices equal better returns, the authors of the article uncover a number of studies pointing to flaws in the rating process:

The authors of “A Directionless Compass” also consider the possibility that, although the ratings published by a single rating provider are not predictive of performance, the ratings of the various providers can be informative when considered as a whole. They look at a study that actually finds a a stronger relationship between ESG and performance by combining ratings from multiple vendors to reduce the “noise” of conflicting ratings.

Comments are closed.