Defensive stats make headway in baseball
An enduring image for some Mariners fans is that of former Seattle slugger Raul Ibanez rounding the bases after a home run. But for a growing...
Seattle Times staff reporter
Defensive stats systemsRevised zone rating (RZR)
Available free on The Hardball Times Web site, RZR was created by John Dewan as an upgrade to his original Zone Rating system. It measures the proportion of balls hit into a fielder's zone that he converts into outs. Unlike Zone Rating, it doesn't credit for double plays and separates a fielder's plays out of his zone into a separate, accompanying stat called OOZ.
Published in Dewan's annual The Fielding Bible book, this stat calculates the probability of fielding a ball based on where it went (distance/direction) and whether it was hit hard, medium or soft. Weighted scores are added or deducted from total, based on the degree of difficulty for each play made or not made.
Ultimate zone rating (UZR)
Available free on FanGraphs Web site and similar to Plus/Minus. UZR was created by Mitchel Lichtman and measures a fielder's value in terms of runs saved or cost a team, rather than plays made or not made. It uses a formula to convert all types of plays into specific run values.
Defensive runs saved (DRS)
The newest Dewan metric in The Fielding Bible Volume II is an all-encompassing system that converts Plus/Minus scores from plays made or not made into runs saved above or below average. Plus/Minus is the main component, but smaller ones add points for outfielder arms, double-play abilities, bunt defense, stolen-base defense and a catcher's ability to handle pitchers. Critics say not enough is known about the pitcher-catcher dynamic to have included much of the extra data.
Probabilistic model of range (PMR)
By David Pinto of Baseball Musings Web site, this metric annually measures a fielder's range by looking at each fieldable ball put in play. One of six criteria determines how difficult it was to field the ball, and the probability of turning it into an out is calculated into a defensive efficiency ratio (DER). That is divided into what the fielder's actual DER was to create a rating.
An enduring image for some Mariners fans is that of former Seattle slugger Raul Ibanez rounding the bases after a home run.
But for a growing number of discerning eyes, the prevailing sight was Ibanez plodding helplessly in left field as another blooped ball dropped in front of him. If anything irritated Ibanez nearly as much as losing during his final season with the Mariners, it was the buzz that he was no longer capable of playing the field.
And what really irked him was the vast array of newfangled defensive statistics trotted out by those criticizing his play.
"I have to say a number of things," he told The Times, shortly after leaving the Mariners this offseason to join the Philadelphia Phillies. "Number one, with sabermetrics in general, it's a statistical probability thing. And the way they come up with the defensive measurements, or ratings, is flawed. It's as flawed as the Gold Gloves. One of the reasons is, they don't consider things like ballpark factors, defensive positioning or alignment for certain hitters."
Two things stand out about those comments. The first was that a player even knew enough about sabermetric defensive stats to comment on them intelligently. And the second? Well, that Ibanez was partly right.
It's true that the most advanced defensive stats don't scientifically discern how hard a ball was hit, how much time a fielder had to get to it, nor gauge its trajectory with real accuracy. They either ignore positioning entirely, figuring it's up to players to get it right, or have imprecise ways of judging it. And while some top systems do account for quirky ballpark configurations, the practice involves plenty of guesswork.
But the reality is, for all the knocks against fledgling defensive stats being less reliable than their offensive counterparts, they do go well beyond traditional measurements like errors or fielding percentage. The gap between defensive and offensive stats is closing quickly and could be drastically reduced this season as new technology is unveiled.
Defensive systems on rise
"It's just remarkable how there's a lack of knowledge about this kind of stuff," said John Dewan, author of The Fielding Bible reference guide and founder of Baseball Info Solutions, a company that sells stats research to teams, agents, computer games and developers of defensive systems. "Why don't more teams try this? Baseball tradition has a lot to do with it. Teams are afraid to try something new. The minute they do and it goes wrong, they'll change it right away."
Not in Seattle, where the Mariners appear to be using defensive metrics more than ever. Seattle made a December trade that added center fielder Franklin Gutierrez and left fielder Endy Chavez, despite their underwhelming offensive stats.
It was on defense where they excelled.
Dewan's recently released second volume of The Fielding Bible contains a new metric called Defensive Runs Saved (DRS) and it claims Gutierrez was the top right fielder in baseball last season. As for Chavez, his totals in a backup role in left field were also above average.
The Mariners don't rely exclusively on any system, having created their own stats analysis department to come up with proprietary information. But the team will borrow information off many of the systems out there — some of which offer daily-updated numbers free of charge.
Describing the exact numeric formulas that go into each system can be a tedious exercise. Even discussing minute differences between systems could take entire newspaper articles.
Dewan introduced a well-known Plus/Minus system a few years back, then used it as the main component of his new DRS metric. But there are other top systems, like Mitchel Lichtman's Ultimate Zone Rating (UZR) or David Pinto's Probabilistic Model of Range (PMR), that intelligent baseball minds swear are just as good or superior.
"I like to look at all of them," said Tony Blengino, a special assistant to Mariners general manager Jack Zduriencik, who oversees the team's new stats branch. "It's kind of an emerging frontier of statistical analysis, and I think they all bring something to the table. More often than not they point in similar directions. Every now and then, they don't. But I think there's progress being made in the fielding analysis. I think they all have something to say."
The core of many systems is nearly identical. In fact, the best ones get their basic data, containing details on each batted ball, either from Dewan's Baseball Info Solutions, or a company he used to own called STATS, Inc.
Each system involves some level of dividing the field into specific "zones" of coverage, figuring out where a ball was hit and which fielder was responsible for getting to it. Fielders who make more plays inside — and outside — their zones than average players at their positions are judged to be above-average, while those who don't are below.
If only it was that simple.
The problem starts with the intangibles that occur on every play that impact fielding results. Things like runners on base, or a 38-foot-high wall in left field at Fenway Park, or another fielder who takes the ball in someone else's "zone" of coverage.
Those who study and create defensive stats systems refer to these intangibles as "noise" and have devised different ways of dealing with it. Some will adjust their numbers through complex formulas to account for intangibles, while others simply admit defeat on a certain issue and don't bother trying.
For instance, if New York Yankees shortstop Derek Jeter makes a successful play on a ball 40 percent of the time in a certain fielding zone, compared to a 50-percent rate by a typical shortstop, he won't automatically be scored the same by all systems.
"Some systems take into account that other people fielded that ball," UZR inventor Lichtman said. "If the second baseman fields it, some systems give no demerits [to Jeter]. Others give some demerits."
Hence a bunch of different scores that can vary wildly. Especially in the rather short time frame of a season.
It's why Dewan's DRS system states that the Mariners were the 13th best defensive team of 30 major-league squads last season, while Lichtman's UZR had them rated 20th. That's enough difference for even casual baseball fans to wonder whether they'd be better off throwing darts at a board for more accuracy.
But the good news is ...
But for all of their shortcomings, the new stats offer things traditional ones can't.
Errors and fielding percentage tell how a fielder did when he actually touched a ball. The new stats look at how a fielder did when the ball got nowhere near his glove.
There's a huge difference, the first case being visible to the naked eye while the other is much tougher to decipher. And it's the unveiling of this once-hidden information that, when interpreted and used correctly, could provide a major advantage to teams that know what they're doing with the numbers.
After all, the mantra of those using such stats is that "a run saved is as good as a run scored." And while teams know what to look for on the offensive side with on-base percentage, slugging, runs scored and the like, many are in the dark as to which numbers add up to the best defense.
Ibanez wasn't far off when he suggested the new defensive stats don't account for different ballpark designs. Many systems don't, and while some do, like Lichtman's UZR, he admits "park adjustments are not an exact science by any means."
But Lichtman still adjusts his raw numbers for, say, games in Boston where the "Green Monster" wall in left makes fly balls an adventure. While he admits the small tweaking done in favor of left fielders who play in Boston amounts to some guesswork, he figures it's the lesser of two evils and better than not altering them at all.
Such thinking has led to controversy for Dewan's new DRS system, which throws a hodgepodge of defensive equations into one all-inclusive formula. Some of the newer measurements Dewan introduces, like grades for catcher defense, have been criticized by those who say not enough is known about such intangibles to measure them accurately.
And that's the challenge faced by those on the cutting edge of these stats: trying to account for as many possibilities while not throwing overall numbers off to the point where they're useless.
How accurate are systems?
It's not just defensive stats that have glitches. As easy as hitting appears to be to measure, top analysts suggest offensive stats are only about 85 percent accurate in gauging a hitter's true talent.
"No system is perfect," Dewan said. "Whether we're talking about offense or defense."
A consensus is that defensive stats themselves are about three-quarters as accurate as their offensive counterparts.
Dewan's staff assigns a "soft" "medium" or "hard" grade to every fly ball they see on videotape. But that also raises another good point by Ibanez, that often the stats are compiled by non-major-leaguers trying to judge how tough a play was from the comfort of a video room. It's not just the location of batted balls that has to be judged, Ibanez said, but also the speed and angle at which they are hit.
"Trying to judge accuracy on a camera view is not the same," he said.
But the good news is that some major developments this year might allow for a Neil Armstrong-like leap forward when it comes to compiling such data with the accuracy Ibanez is looking for.
First off, Dewan and Lichtman will start recording the "hang time" of fly balls.
Dewan's crew will do so with computer software and time all fly balls hit during the 2009 season — working off video footage — from bat to glove. Lichtman will have 30 people armed with handheld stopwatches doing the same thing with fly-ball footage from the 2006 to 2008 seasons.
That's seen as great news by Dave Cameron, a co-founder of the U.S.S. Mariner Web site, which provides stats-based analysis for Mariners fans.
"I think the key is, what we really want to know is how far a fielder can go in a certain amount of time and we really don't have that now," Cameron said. "If Ichiro runs 180 feet to catch a ball, we're going to know how much time he had to get to it and whether he should have been there. Right now, that's just some dude's opinion in a box."
The second major development is the expected implementation later this year of a technology called Hit F/X.
Right now, MLB Advanced Media uses three high-speed cameras in every ballpark to record the speed and "break" of every pitch in a system known as Pitch F/X. The idea for Hit F/X is that the same cameras could record the speed and trajectory of every ball coming off a bat.
"Then we'll get more specific on the ability to make each play," said Cameron, also a freelance contributor to The Wall Street Journal. "Rather than going off zones and saying, 'He should have caught this and he should have caught that.' "
Still a work in progress
Lichtman said the two new developments could help create the "perfect defensive metric" and bridge the gap with offensive stats.
But others aren't so sure. After all, there will still be arguments about which systems are tackling the right intangibles the proper way.
Lichtman suggests a longer-term approach to viewing all systems.
"It's all going to even out in the long run," he said. "Once we have one, two or three years of data, we're pretty confident that we have an assessment of the value of that player."
On some levels, the systems agree. Lichtman's UZR and Dewan's DRS give out run value scores to players that differ in how they are calculated and cannot be directly compared. But the scoring gaps that occur between players in each system show the differences in specific conclusions reached.
For instance, both Lichtman's and Dewan's systems agree Ibanez was below average. They agree less on how poor Ibanez was in relation to other subpar left fielders.
Lichtman's numbers say Ibanez was terrible last year, his minus-11.9 score well below even Manny Ramirez (-2.3), and Pat Burrell (-4.5), trailing Jack Cust (-10.3) and barely better than Adam Dunn (-14.9). But Dewan's suggest Ibanez wasn't that bad, his minus-6 being close to Ramirez (-3) and Burrell (-5), and slightly better than Cust (-8) and Dunn (-10).
The difference is more dramatic over the last three years combined, with Lichtman's system scoring Ibanez at minus-38.7, about on-par with Ramirez (-41.2), better than Dunn (-48.7) and worse than Burrell (-32.9). But Dewan's shows Ibanez being demonstrably better at minus-8, than all of Ramirez (-28), Burrell (-30) or Dunn (-39).
The Phillies, if they looked at defensive stats, may have agreed with the latter when they let Burrell go and signed Ibanez for three years and $31.5 million.
"I don't think it's ever going to be to the point where there's the unanimity or the clarity that there is with offensive stats," Mariners assistant Blengino said. "Is it possible? I guess, theoretically, it's possible. But one's more of a science and one's more of an art."
And the art of measuring defense might never have to reach that scientific level to gain widespread acceptance. For folks like Blengino, willing to comparison shop and dig a little deeper, there's value to be found in the artists' sketchbooks as they attempt to create the next Mona Lisa.
Geoff Baker: 206-464-8286 or firstname.lastname@example.org
|Breaking down last season's defense|
|Charting the 2008 defensive statistics of former Mariners outfielder Raul Ibanez and current Mariners Franklin Gutierrez, Adrian Beltre and Ichiro. (Rankings are for players among the top 35 at their position for innings played.)|
|Raul Ibanez||31st||29th||17th, 8th||32nd||31st|
|Franklin Gutierrez||1st||1st||1st, 7th||3rd||1st|
|Ichiro (RF)||17th||3rd||15th, 9th||15th||5th|
|Mariners team||13th||20th||23rd, 11th||13th||16th|
Copyright © 2009 The Seattle Times Company
UPDATE - 8:27 PM
Catcher Gregg Zaun retires after 16 seasons