Researcher Puts 1.1 Million Wall Street Analyst Notes Through An AI – Finds Some Mind-Boggling Results

In the fast-paced world of Wall Street, where numbers speak volumes and precision matters, 1.1 million analyst notes might seem like a goldmine or a pile of noise, depending on whom you ask.

Shikun Ye, a researcher at the Yale School of Management, conducted a compelling analysis using a large language model (LLM) to interpret a vast collection of analyst notes spanning from 1998 to 2023. His findings, amplified by Panmure Liberum Capital strategist Joachim Klement in a blog post, challenge traditional assumptions about the usefulness of analyst recommendations.

Surprisingly, investors who simply followed analyst price targets by going long on stocks with the highest targets and short on those with the lowest would have underperformed the broader market. This lends some credence to the cynic’s view that traditional analyst output may not be especially profitable.

But the story doesn’t end there. The LLM dug deeper and found that the written content of analyst notes, the narratives, observations, and qualitative assessments held predictive value for both future earnings and stock returns. This content-based strategy, using just the written analysis and ignoring price targets or formal recommendations, consistently beat the one relying on price targets alone.

A fascinating pattern emerged from the language used in these reports. Analysts’ focus shifts based on the economic climate: in boom periods, they emphasize profitability; during downturns, macroeconomic factors and financial health take center stage. Their attention also varies depending on the type of company—value stocks see more emphasis on profitability, while highly leveraged firms prompt concern over financial conditions.

When it came to earnings forecasts, analysts had an edge over the LLM in the short term, typically within a one-year window. However, over longer horizons, the LLM’s ability to extract meaning from the analysts’ words began to outperform the analysts’ estimates. This suggests that, even unconsciously, analysts may embed useful signals in their commentary that extend beyond their explicit predictions.

Ye attributed this to the “story-statistics gap”, a psychological phenomenon where individuals tend to remember and be influenced more by narratives than raw numbers. “A story more persuades people than statistics,” he noted, highlighting a key challenge analysts face: they’re better at processing and relaying numbers than synthesizing qualitative information into actionable insights.