I have read some posts like the ones below about testing a lens to determine how well it performs optically. What I didn't see was anything on determining whether the lens performs as it should and is not like the one described in an often-seen review like this:

Since 2007 I've owned five copies of the lens. It's a heavy and amazing lens if you can handle the size and be lucky enough to get a good copy. There is huge same variation and you need to make sure your copy is decent. Nikon can charge you a lot of money for fixing it. Among five copies I had, only two didn't need going back to Nikon for servicing.

I'm not one of those people who is looking for better than spec, but I do want to be able make an objective determination whether the lens I bought is up to par and I didn't get an outlier (a "bad copy"). I'm also not likely to hold more than one copy at a time to make a side-by-side comparison. This is something I want to be able to do within the warranty period, if there is one.

How would I make this objective determination? How can I determine that the $1600 lens I buy is not an outlier that is working like a very good $500 lens from the same company?

Related questions:
How can I test a new lens to make sure it is operating correctly?: Gives some good recommendations for making qualitative and quantitative tests, but doesn't cover how to compare to lens specifications.

What should one look for when buying a used lens?: Mostly covers cosmetic and mechanical issues that when discovered are obviously not what should be present with a new lens.

  • I'm going to take a shot in the dark that the quote you provided is describing a 24-70mm f/2.8 L lens that is notorious for having copy to copy variations, especially in the used market. Most modern lenses from Nikon and Canon have far less issues like that. Complex designs, zooms, and plastic parts all play into how much variation exists. If you are buying a simple 50mm prime I would expect much less variation for example. – dpollitt Feb 28 '15 at 17:46
  • 1
    Note that I believe the title should be updated if you want to compare a lens to the spec sheet. And if that is the case this is a duplicate of: What are the standard tests available to quantify lens parameters?. If your question is how to test if a lens is a dud, this is a duplicate of How can I test a new lens to make sure it is operating correctly? – dpollitt Feb 28 '15 at 17:52
  • @Your referenced duplicate doesn't explain anything about comparing with what can be expected (lens specs). The summary in Itai's answer begins "Then comes the subjective part. You need to judge if this is the quality you expect from such a lens." I'm looking for something objective that I can compare the lens in my hand to the objective expected performance. The referenced question doesn't go there. If nobody really has an objective way of doing this, and return lenses just because they don't seem right (subjectively), then maybe that's the answer. – Jim Feb 28 '15 at 18:21

I think that you are underestimating how bad a bad lens truly is.

It is going to be very obvious if you pickup a real dud and it is acting like a $500 vs $1,600 lens. You can and should run though the related question and its recommendations for testing any new lens: How can I test a new lens to make sure it is operating correctly?.

You state that you want to compare a lens to its lens specifications; why and for what purpose? Are you trying to prove that a manufacturer is truthful in their advertising? It sounds more like you just want to know that what you purchased is not outside of the standard deviation of acceptable resolution/sharpness. Comparing to the advertised MTF charts is not the same as that.

If you want to know that you didn't buy a dud(very obvious):

If you want to prove that a lenses specs match the real world or compare copy to copy(MTF, etc):

See also:

Overall, you will find that truly subpar or bad copies of lenses are outliers that can easily be identified with simple tools. If you really do want to compare to the lenses spec sheet, you that will require professional lab tools.

  • 2
    I chose that review arbitrarily, but I think you're right. (It was the first lens at the top of the B&H list that had the kind of wording I have seen so many times with lenses.) I'm the kind of guy who was always fine when my color TV (when not everyone had color TVs) was not color adjusted perfectly, and couldn't understand the fuss when someone felt it needed to be adjusted. But I'm serious about technical quality with photography. I wouldn't necessarily know yet if my $1600 lens is a dud if it acts like a $500 lens. I'm trying to figure out how these reviewers know. – Jim Feb 28 '15 at 18:16
  • 1
    If by "the reviewers" you mean reviews at DPReview and photozone.de for example - The answer is that they use lab tests such as Imatest and Dxo Analyzer. If your idea of "reviewers" is every guy with an opinion in a forum, then the answer is that most of them have no idea what they are talking about :) But most of the well trusted reviewers will use a combination of the two; subjective assessments and Imatest or similar. I'm not sure that Bryan Carnathan at The-Digital-Picture even uses lab tests per say, he seems to simply use similar methods as our Q&A on: "How can I test a new lens..." – dpollitt Feb 28 '15 at 21:21

Short of using professional lab tools you can't really determine if a single lens is performing up to spec or not. If you have real stinker you can usually tell that, but without a calibrated optical bench it is nigh impossible to tell if a lens is slightly better or slightly worse than the manufacturers stated performance.

In the real world what usually happens is that you start out using your theoretical $500 lens. At some point you discover how much that expensive (in your newbie mind) $500 lens lacks in certain areas compared to results other photographers are obtaining with higher end gear. You then go out and buy a mid-grade $1600 lens and when you begin using it you are disappointed with the marginal improvement you see between the $500 and $1600 lens. Sure, your results are a little better with the $1600 lens. But they are not the three times better you were expecting when you spent three times as much on the new lens!

Hopefully at some point you begin to realize that great photos are about a lot more than having the best gear. You can't capture a well lit photo in crappy light. It doesn't matter how sharp a lens is if you can't learn to focus it on the point you wish. That f/2.8 zoom lens doesn't help you find the best aperture and hyper-focal distance needed to maximize resolution and Depth of Field of a landscape vista. Just as your experience allows you to begin to understand what you need to do to capture the shot you envision, your experience should allow you to see if your lens is performing in the ballpark that you expect based on the reviews you read and samples you viewed before buying it. If you are not to the point where you trust your own experience to evaluate the performance of a lens, seek out a more experienced shooter in your area that uses the same system and see if they would give you an opinion on the lens. Photography clubs and groups are more common and popular than ever thanks to the ease of communication made possible by the social media revolution.

I'm a Canon guy and have been for years (decades, really). When I first discovered Bryan Carnathan's lens reviews at The-Digital-Picture, I looked at his reviews of all of the lenses I already owned. As I read the reviews I repeatedly found myself nodding in agreement. The strengths and weaknesses he identified in certain lenses matched my own experience with those same lenses. There was a single lens where my experience and his diverged quite markedly. I did a little investigation, borrowed a copy of the same lens from a friend, and compared it to my own copy which I hadn't used for several years. I discovered that my copy was moderately under performing. I also discovered that my own technique had improved during the interim and I could get better results with that same lens than I had gotten when I first used it! Prior to that experience I had routinely dismissed that lens as being a bad design. I had recommended against buying that lens to other Canon shooters. What I learned is that the next time I have a less than desired experience with a lens, I should probably send it to Canon before the lens is out of warranty instead of putting it back in the box it came in and never using it!

  • 1
    I agree with your analysis. Especially because the asker mentioned in a comment "But I'm serious about technical quality with photography. I wouldn't necessarily know yet if my $1600 lens is a dud if it acts like a $500 lens.". If one really cannot tell the difference between a $1,600 lens and a $500; they probably were not ready for the expensive version and should probably worry about other things. People throw money at photography all the time as if money is what makes art. – dpollitt Mar 1 '15 at 17:45

Your Answer

