April 19th, 2007

photography

Listen to me when I say "Ignore me."

They weren't kidding yesterday … it really IS Christmas!

Every so often, I review equipment on this blog, but I've noticed that I tend to review items that are new to me, things that I can feel out and write down my impressions of, whether it's a new camera or lenses that I've rented. Meanwhile most of the equipment that I use every day and am intimately familiar with remain unwritten. Part of that is that I think the pictures are enough, part of that is that the simple fact of my ownership is a seal of approval, but most of it is just that I like novelty. So I'm going to begin a series of reviews of the stuff I use every day soon, which will mostly be a series of lens reviews. But first, a public service message: Don't pay too much attention to lens reviews.

I have a few major problems with lens reviews -- the first is that it's very hard to do them right, requiring an incredibly meticulous personality. But more importantly, even testers who do everything perfectly fall prey to the simplest problem in statistics: sample size. Almost invariably, they're reviewing one lens sample. How's that for margin of error? This creates a sort of weird version of the Heisenberg Uncertainty principle that the more finely you measure a certain lens trait, the less you know about the lens you'll actually get. You can learn a lot from general comments on build quality, how well a lens does its job, speed-of-focus, etc. -- broadly measured traits that won't vary much from sample to sample. But then we turn to the worst offender -- sharpness. Reviewers look at their single sample in the most minute detail and compare them to their other single samples in minute detail and give us comparisons that tell us essentially nothing.

For instance, the reviewer at 16-9.net has shown time and time again that his Canon 24mm f/1.4L is an amazing performer -- something which drives other 24mm f/1.4 owners nuts, since they consider theirs to be relatively mediocre. On the other hand, I've seen people look down on the Nikkor 85mm f/1.4 and 17-55 f/2.8 because one reviewer (whose methodologies are less-than-perfect) liked his samples less than other similar lenses -- certainly a surprise to me, since I use both of them every day and am constantly stunned by their output.

So, even if I were rigorous -- and I'm not -- these reviews are going to be in broad brush strokes, from the perspective of in-the-field use. In some cases this is more forgiving than laboratory testing, in other cases more rigorous -- for example, I consider the 70-200 f/2.8 to be one of my softer lenses in practice, because Vibration Reduction can never give the consistency of simple lens speed.

Why do reviews, then? First, people have asked for them. Secondly, you can learn about a lens if you take reviews with a grain of salt and, more importantly, read a large number of them. I've only tried one sample of the Tamron 28-75 f/2.8, but I have heard so many people sing its praises so universally, that I don't hesitate to recommend it even though it's a third-party lens (and its diigital successor, the 17-50, seems to follow the same pattern.) So, as always, feel free to ignore me.