A Monte Carlo measure to improve fairness in equity analyst evaluation

The Wall Street Journal’s “Best on the Street,” Star Mine and many other systems measure analyst stock-rating performance using variations on a method we term the “portfolio method,” whereby a synthetic portfolio is formed to track the analyst’s ratings. At the end of the evaluation period, analysts are compared by their respective portfolio returns. Of the pitfalls to this method, one most troubling is that the analysts are generally covering different sets of stocks over different time periods. Thus, each analyst has access to different opportunities and just comparing portfolio values is unfair. In response, we present a Monte Carlo (MC) method where, for each analyst, we generate numerous “pseudo-analysts” with the same coverage over the same time periods as the real analyst. Using this method, we are better able to compare analysts, adjusted for their individual opportunities. We draw comparisons between our results and the results from existing systems, showing that those systems are less precise in reflecting analyst performance.