In this chapter, we describe the basic concepts and algorithms of the structural approach to image quality assessment. Unlike conventional bottom-up image quality assessment approaches that attempt to simulate the functions of the relevant components in the early visual system, the structural approaches follow a top-down philosophy that is based on hypothesized functionality of the overall visual system. We demonstrate that image distortions along different directions in the image space have different perceptual meanings. The structural approach attempts to separate the directions associated with non-structural distortions (those that do not change the structures of the objects in the visual scene) from the remaining structural distortions. This separation must be adapted to the local image content. As a specific example, we discuss in detail the structural similarity (SSIM) index and demonstrate its effectiveness using an image synthesis method that maximally differentiates two competing image quality metrics.