Thanks to Felix Salmon, who sent me this essay by D-squared Digest. I understood . . . much of it. An excerpt:

*A somewhat overcomplicated estimator for talent*. So basically, a really excellent fund manager would have the characteristic that changes in his trading strategy anticipated changes in V. Investment in this view is a problem of forecasting structural breaks or regime changes. So if I had unlimited resources of data and mathematical ability, I would take data on securities prices, and on the trading positions of the fund under analysis. I would use a STOPBREAK (stochastic permanent breaks) model to identify regime changes in both, giving me a vector of dated regime-changes in the equity returns series, and a vector of dated regime-changes in the trading strategy series. I'd then use the normal toolkit to estimate the lead-lag relationship between the two series. Obviously a lead of the trading strategy series over the returns breaks series would be ideal, but a short lag would also be worth knowing about - while the man who can tell you when a big change is coming is a gem of great price to be treasured, the man who can spot when something's going wrong and stop losing, is also a highly useful lad to have around, and perhaps a bit more realistic to hope for. So that's what I'd do if I were in the manager research game.

*A less complicated and perhaps more sensible estimator*: This is a bit of a sledgehammer to crack a nut approach though. I would guess (and could probably prove if I had three weeks spare time and masochism to devote to relearning a load of stochastic processes again) that a reasonably good robust non-parametric estimator for the lead-lag based talent measure suggested above, would be the number of >10% drawdowns, which is a statistic that lots of funds will put in their risk disclosures anyway. I'd specifically divide the average return per year, by the average number of drawdowns per year, and give myself a measure of how much return these guys made per "mistake". I'd then take a look at >20% drawdowns and see whether they were prone to making big mistakes.