Skip to main content

Originally published March 3, 2012 at 8:03 PM | Page modified March 3, 2012 at 11:37 PM

  • Share:
  • Comments (1)
  • Print

Chuck Jaffe: No way to test new predictive ratings

While the Hulbert Financial Digest has long tracked investment newsletters, there is no track record for the recently employed forward-looking mutual-fund ratings. Investors should remain wary.

Syndicated columnist

Most Popular Comments
Hide / Show comments
There is a way to test the accuracy of one of the forecasting approaches that Chuck... MORE


The problem with building a better mousetrap is that you have to convince people that they need it.

Once you convince them, however, you'll have to stave off the competition, because the minute they know that the public is shopping for something better, they'll bring their own versions to market.

That's exactly what is happening to Morningstar, which last year introduced its new "analyst ratings," marking a significant change in how its analysis system worked because it purports to tell investors which funds the firm believes will be best in the future.

Morningstar's long-established star-rating system looks at risk-adjusted past performance, and was designed to be descriptive of a fund and its results in the past, even though investors used it as if it were the Good Housekeeping Seal of Approval.

For years, more than 90 percent of all money flowing into funds has gone into issues carrying Morningstar's top two star ratings; while stars are still awarded on the same criteria, they are now supplemented by the forward-looking, predictive gold, silver, bronze (or neutral/negative) of the analyst ratings.

But if investors are going to supplement their criteria with something that looks beyond past performance measures — whether those come from Morningstar, Lipper or simple raw performance — they should also take a peek at the competition, a pack that is heating up in pursuit of the industry leaders.

While investors have long been able to choose from newsletters like No-Load FundX or No-Load Mutual Fund Selections & Timing, they also have new services like and analysis from firms like New Constructs providing alternative analysis.

As a general rule, predicting fund performance is based on drilling down into the portfolio to see the kind of issues it holds, or it looks at the performance pattern of the portfolio.

New Constructs' ratings, for example, are based on the firm's stock analysis; a fund that holds many of the securities that the Nashville firm considers "most dangerous" is going to be a dangerous fund.

By comparison, FundReveal — a firm founded by former Fidelity executives — uses risk and return measures to size up "persistence"; the fund's measure on that scale indicates the decision-making ability of the fund manager.

The newsletters typically don't just rate funds, but also come up with portfolios for investors to follow. Still, it's a similar idea — finding the best funds for tomorrow — with a bit more guidance on when the status of those funds changes.

No-Load Mutual Fund Selections & Timing uses "comets" instead of stars, tilting toward the hot sectors; similarly, No-Load FundX — which has been published since the mid-1970s — ranks funds based on recent measures of performance, buys the hot ones and holds them for as long as they are outperforming the competition.

Whether it is the ratings firms, websites or newsletters, David Snowball of noted that very few "share anything about their methodology beyond the utterly vacuous. Some of the phrases come close to 'and then the magic happens.' "

That's why investors have to be wary.

Because each analytical idea has its own nuances — and time frames — each can claim to work its own way, because they are all calculating performance on their own.

While the Hulbert Financial Digest has long tracked investment newsletters, there is no track record for the new information and how well it works (or doesn't). There's no standardization.

"My problem with all of these prognostications is that you have no real idea how good their system is or how reliable they are, certainly not now while they are new," said industry consultant Geoff Bobroff of East Greenwich, R.I.

"They're dealing with stale information on what is in the fund's portfolio, and they are dealing with the influx of derivatives and strategies for using those derivatives that I'm not sure any outsider has a hope of analyzing accurately, at least not without information on strategy that the funds don't have to give, even to the ratings firms."

What's more, just as there are thousands of patents filed on mousetraps (really), there's no guarantee that any of the newcomers will succeed. If they do, they might go the way of, a portfolio-assessment service that was building a following when it basically went away, part of MSCI's purchase of RiskMetrics that put the data off-limits to Joe Average.

"Forward-looking exercises are about as valuable as forecasting in general ... not very," said Michael Falk of Focus Consulting Group in Chicago.

Time will tell whether that's true; in the meantime, fund investors should look at these exercises as interesting and intriguing, but they will want to learn a lot about how a system works before trusting any firm's ability to predict which funds will do the best in the future.

Said Bobroff: "The marketplace wants this, investors have demanded it, but that doesn't mean it works. It's just too early to tell, though it's tempting to look ... and if you are going to look, then you should evaluate everyone who is doing it to see whose system you are most comfortable with. But until there's enough history to test a system's veracity and validity, you should follow these recommendations very cautiously."

Chuck Jaffe is senior columnist for MarketWatch.

He can be reached at or at P.O. Box 70, Cohasset, MA 02025-0070.

News where, when and how you want it

Email Icon

Relive the magic

Relive the magic

Shop for unique souvenirs highlighting great sports moments in Seattle history.



NDN Video