Lens quality is generally a very subjective term. An architecture photographer, for example, will care a lot more about distortion than rest of us. The same goes for flare or vignetting which can become important factors for some photographers. The best way to check out these parameters is to simply to take a series of identical shots using different lenses. Unfortunately, this is almost always impossible. We can (and should) also look at professional reviews. This is a great source of information but unfortunately many reviewers will draw different conclusions from identical equipment. Try going to luminous-landscape or photo.net and then to Ken Rockwell's site to see what I mean. Another problem is that reviewers check out a single (very often carefully selected by the manufacturer) lens sample and we know that lens sample variation can be an important factor as well. Because of all this it's usually a great idea to also examine full size images taken by other photographers. This is a valuable source of information and can give us a great deal of knowledge about lens quality (including sharpness, chromatic aberrations, vignetting and flare) as well as sample variation.
When examining sample images, it's important to know what to look for as well as to understand how the different parameters (e.g. aperture, distance from the image center, ISO etc.) affect quality. One cannot for example compare a center crop of an image shot at f/8 with an edge of another image at f/2.8. Even the best lens may then seem to be worse than a kit zoom. Once we have this understanding we can check images and draw valuable conclusions.
The single most important factor we are usually concerned about is image sharpness. We often make a mistake of thinking that such or such camera will produce very sharp images. This is not true -- when we look at the market of digital SLRs we can see that all currently available models (both entry-level and professional) offer incredibly good quality, at least up to ISO 400-800 (naturally there are differences that justify price differences but they are not related to image quality). Only above these sensitivities we can tell that cameras can largely influence image quality. Lens sharpness (resolution) can be theoretically very well determined by looking at MTF charts available on most producers' websites. The problem is that these are dry numbers that are difficult to translate to actual image quality. Often they also represent a theoretical "perfect" sample. This is why we usually want to examine full size images ourselves. And how do we do this and what do we look for?
The most important thing we can ask for, besides a full size image, are the parameters used when taking a photo. We must know the aperture and focal length but it's also good to know the ISO (high ISO can reduce sharpness) and exposure time (long exposures can also cause a blurred image). Without these parameters we don't know how to interpret the image. Another problem is to know where the lens was focused at the time the image was taken. If nothing is in focus, we shouldn't draw any conclusions from the image (often this can be wrongly interpreted as a soft lens).
When we find the focal point we should look at the scene three-dimentionally and see what else should be in focus and what not (this is the depth of field or DOF). We need to remember that large apertures (meaning a low F-number such as f/1.4 or f/2.8) will make everything even slightly behind or in front of the focal point unsharp. For instance, a portrait shot of a person standing in front of a brick wall shot at a large aperture will result in the brick wall being unsharp, so we cannot say anything about how sharp the lens is in the corners (unless there's a second person at the edge standing at the same distance).
Now, getting back to the lens itself, we know that any single lens has will yield a different sharpness based on:
Distance from the center of the image
The center will always be the sharpest area and the edges will be more or less blurry depending on the lens and aperture:
Here are 100% crops from the center area taken with a high quality prime and an average quality zoom. As you can see there's not a big difference between the two crops!
Here are 100% crops from the top-right edge. While Nikkor 18-200 shows a large drop in sharpness, the excellent 85mm prime looks almost as good as in the center.
Best sharpness is usually obtained at f/8-f/11. Larger apertures (e.g. f/1.8) will not be as sharp (even at the focal point). The biggest difference between a large and optimal aperture (e.g f/1.8 vs f/8) will be visible in the corners of the image. The center will be only slightly worse.
Using smaller apertures (e.g. f/22) will also cause loss of sharpness because of the effect of diffraction.
This applies to zoom lenses only. Most zoom lenses perform best in the middle range and worse at the extremities (e.g. an 18-135mm zoom will perform better at 50mm than 18 or 135mm)
To illustrate this, here are 100% crops of the same subject (shot at a different distance to preserve scale:
A good lens will keep good (consistent) sharpness across all values of the above parameters (distance from the center of image, aperture, focal length). Here we have to think how much additional weight, size and money we want to sacrifice to get desired quality. This is exactly why looking at sample images is very helpful. Once you know what quality will be sufficient you can check whether less expensive lenses will do or not. Spending more money on a lens or carrying lots of weight along just because a review says that one lens is better than another (or because the MTF curve looks more exciting) is not very smart.
Unfortunately many lenses have a problem of inconsistent quality amongst different samples of the same lens. This problem seems to be especially big with wide-angle zoom lenses and tends to be worse for the less expensive lenses (except for prime lenses that are normally of great quality). You can tell if a lens is a bad sample by comparing different edges of an image. Every lens has the right to be better in the center but it should be just as sharp on the left side as it is on the right. If you see that many sample images have this problem (it's usually better visible at large apertures), make sure you can return the lens you purchase if you notice the problem. If returning the lens is not possible, you're better off sticking to lenses that generally don't have this problem (like primes or expensive zooms). Sample variation is also one reason why it's also good to look at sample images taken with different lenses instead of only sticking to MTF charts or reviews.
Chromatic aberration (CA)
This can be seen as fringes of color, especially around the dark areas of an image. CA can be a big and noticeable problem, especially in wide-angle lenses. When looking at an image at a 100% zoom, you should try to find dark shapes surrounded by bright backgrounds (such as a tree and a cloudy sky behind it) if you want to check for this. You should also note that CA gets worse near the edges of the image itself. Center of the image is not affected as much.
Here are two images we have already seen. The left one shows CA as a purple edge around the dark areas. The additional image shows another example of CA.
Lens flare is unwanted light (one that doesn't form the image) that gets to the camera sensor lowering image contrast and spoiling image with polygonal shapes or light streaks.
The amount of flare can vary considerably across different lenses. Generally lenses with fewer optical elements will show less flare. Unfortunately it's difficult to tell if a given sample image displays unusual flaring or not. Here also the best thing to do is look at many different images taking in bright sun light, or even better, take identical shots with different lenses (review sites are usually very helpful here).
Vignetting is the darkening of image near its edges (also called light fall-off). This problem can appear more often with wide-angle lenses, although generally these days it's no longer an important issue (partly because of lens design and partly because you can easily correct this in post-processing).
Distortion happens when straight lines in reality are no longer straight in the image. There are two general types of distortion: barrel and pincushion distortion. Most cases of lens distortion can be easily corrected in post-processing and this is not an important issue for most photographers. Usually this will only be easily seen in buildings or other cubic shapes with straight lines.