The SSIM
Index for Image Quality Assessment Zhou Wang, Alan C. Bovik,
Hamid R. Sheikh and Eero
P. Simoncelli
|
|
The Structural SIMilarity (SSIM)
index is a method for measuring the similarity between two images. The SSIM index
can be viewed as a quality measure of one of the images being compared, provided the other image is regarded as of perfect
quality. It is an improved version of the universal
image quality index proposed before. Detailed description is given
in the following paper:
1.
Z.
Wang, A.
C. Bovik, H.
R. Sheikh and E. P. Simoncelli,
"Image quality
assessment: From error visibility to structural similarity," IEEE Transactions
on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
A Matlab
implementation of the SSIM index (ssim_index.m) is
available here. You can download it for free, change it as you like and use it
anywhere, but please refer to its original source (cite the above paper and
this web page).
Before using the code, please go through the suggested
usage and demo tests below to get an
idea on how to use it and how it works.
Suggested Usage
The above (ssim_index.m) is a
single scale version of the SSIM indexing measure, which is most effective if
used at the appropriate scale. The precisely “right” scale depends on both the
image resolution and the viewing distance and is usually difficult to be
obtained. In practice, we suggest to use
the following empirical formula to determine the scale for images viewed from a
typical distance (say 3~5 times of the image height or width): 1) Let F = max(1, round(N/256)), where N is the number of pixels in
image height (or width); 2) Average local F by F pixels and then downsample the image by a factor of F; and 3) apply the ssim_index.m program. For example, for an 512 by 512 image,
F = max(1, round(512/256)) = 2, so the image should be
averaged within a 2 by 2 window and downsampled by a
factor of 2 before applying ssim_index.m.
The Matlab code (ssim.m) that includes the suggested downsampling
process described above is given here:
Other Useful Downloads
The SSIM values are
computed (using ssim.m) for 6 publicly available
subject-rated image databases, including LIVE database, Cornell A57 database,
IVC database, Toyama database, TID2008 database, and CSIQ database. When the images are in
RGB color, an Matlab
function “rgb2gray” is used to convert the images to gray scale. The results
(in Matlab .mat format) are provided here, together
with performance evaluations based on Spearman’s rank correlation coefficient
(SRCC) and Kendall’s rank correlation coefficient (KRCC), for future comparisons:
Database |
Toyama |
SRCC |
KRCC |
LIVE |
0.9479 |
0.7963 |
|
Cornell-A57 |
0.8066 |
0.6058 |
|
IVC |
0.9018 |
0.7223 |
|
Toyama-MICT |
0.8794 |
0.6939 |
|
TID2008 |
0.7749 |
0.5768 |
|
CSIQ |
0.8756 |
0.6907 |
A tutorial paper is
published at
2.
Z. Wang and A. C. Bovik, “Mean squared
error: love it or leave it? - A new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol.
26, no. 1, pp. 98-117, Jan. 2009.
Powerpoint and PDF figures of
the above paper are available here for references:
MATLAB, C++, Java and LabView implementations of the SSIM index have also been
made available online by other authors:
·
C++ implementation by unknown
author(s) at http://perso.wanadoo.fr/reservoir/
·
C++ implementation by Mehdi Rabah at http://mehdi.rabah.free.fr/SSIM/
·
C++ implementation by Naru Kinjo at http://denshika.cc/open/20100630/,
http://denshika.cc/open/20100630/batch.php
·
C++ OpenCV
implementation (wrapper of Mehdi Rabah
and Naru Kinjo’s work) by Norishige
Fukushima at http://nma.web.nitech.ac.jp/fukushima/opencv/ssim/ssim-e.html
·
Java implementation by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/ssim-index.html
and http://www.ucm.es/info/fismed/SSIM_family/SSIM_archivos/frame.htm
·
Java implementation of Multi-scale
SSIM by Gabriel Prieto at http://rsb.info.nih.gov/ij/plugins/mssim-index.html
and http://www.ucm.es/info/fismed/MSSIM/MSSIM_archivos/frame.htm
·
LabView implementation at LIVE: http://live.ece.utexas.edu/research/quality/SSIM.llb
·
More efficient MATLAB implementation of
SSIM by Adam Turcotte at http://ssim.rivetsforbreakfast.com/
Extensions and Related Papers
The following papers discuss the extensions and improvements of the
SSIM index approach:
3.
Z. Wang and Q. Li, "Information
content weighting for perceptual image quality assessment," IEEE Transactions
on Image Processing, accepted, to appear 2011. [Web and code]
4.
Z. Wang and Q. Li, "Video
quality assessment using a statistical model of human visual speed perception,"
Journal
of the Optical Society of America A, vol. 24, no. 12, pp. B61-B69, Dec. 2007.
5.
Z. Wang and X. Shang, “Spatial
pooling strategies for perceptual image quality assessment,” IEEE International Conference on Image Processing,
6.
Z. Wang and E. P. Simoncelli, “Translation
insensitive image similarity in complex wavelet domain,” IEEE International Conference on Acoustics, Speech and
Signal Processing, vol. II, pp. 573-576, Philadelphia, PA, Mar. 2005.
7.
Z. Wang, L. Lu, and A. C. Bovik, “Video
quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, special
issue on “Objective video quality metrics”, vol. 19, no. 2, pp. 121-132, Feb.
2004.
8.
Z. Wang, E. P. Simoncelli and A. C. Bovik, “Multi-scale
structural similarity for image quality assessment,” Invited Paper, IEEE Asilomar Conference on
Signals, Systems and Computers, Nov. 2003. [Matlab Code] [Java Code 1, Java Code 2]
Other
related papers:
9.
Z. Wang and E. P. Simoncelli, “Maximum
differentiation (MAD) competition: A methodology for comparing computational
models of perceptual quantities,” Journal
of Vision, vol. 8, no. 12, Sept. 2008.
10. Z. Wang and E. P. Simoncelli, “An
adaptive linear system framework for image distortion analysis,” IEEE International Conference on Image Processing,
11. Z. Wang, A. C. Bovik and E. P. Simoncelli, “Structural
Approaches to image quality assessment,” in Handbook
of Image and Video Processing (Al Bovik, ed.),
2nd edition, Academic Press, June 2005.
12. Z. Wang and E. P. Simoncelli, “Stimulus
synthesis for efficient evaluation and refinement of perceptual image quality
metrics,” Human Vision and Electronic
Imaging IX, Proc. SPIE, vol. 5292, Jan. 2004.
13. Z. Wang, H.
R. Sheikh and A.
C. Bovik, “Objective
video quality assessment,” in The Handbook of Video
Databases: Design and Applications (B. Furht and
O. Marqure, eds.), CRC Press, pp. 1041-1078, Sept.
2003.
14. Z. Wang, A. C. Bovik and L. Lu, “Why is image quality assessment so difficult?” IEEE International Conference on Acoustics, Speech,
& Signal Processing, May 2002.
15. Z. Wang, and A. C. Bovik, “A
universal image quality index,” IEEE Signal Processing
Letters, vol. 9, no. 3, pp. 81-84, March 2002.
Demonstration
In
the following, all distorted images have roughly the same mean squared error
(MSE) values with respect to the original image, but very different quality.
SSIM gives a much better indication of image quality.
Original, MSE = 0; SSIM = 1 |
MSE = 144, SSIM = 0.988 |
MSE = 144, SSIM = 0.913 |
MSE = 144, SSIM = 0.840 |
MSE = 144, SSIM = 0.694 |
MSE = 142, SSIM = 0.662 |
The following example illustrates the maximal/minimal SSIM
images synthesized along the equal-MSE hypersphere in
the space of all images. All images along the hypersphere
have the same MSE values with respect to the reference image, but have
drastically different perceptual quality.
The following example
demonstrates the SSIM index map, which provides a measurement of local image
quality over space. Note that severe JPEG compression produces annoying
pseudo-contouring effects (in the sky region) and blocking artifacts (along the
boundaries of the building), which are successfully captured by the SSIM index
map, yet poorly predicted by the absolute error map (in both distortion/quality maps,
brighter indicates better quality).
Original image |
JPEG compressed
image |
Absolute error
map |
SSIM index map |
Test on LIVE Image Database
The
SSIM indexing algorithm has been tested on the LIVE database created
at the Lab for Image and Video
Engineering (LIVE) at the University
of Texas at Austin. The database is
subject-rated and available for free download.
First Stage LIVE database
test
In
the first stage, the database contained 460 images, where 116 were original
images and the rest 344 are JPEG and JPEG2000 compressed. Two sample images
(cropped from 768X512 to 256X192 for visibility) are shown below. Note that quantization
in JPEG and JPEG2000 algorithms often results in smooth representations of fine
detail regions (e.g., the tiles in the upper image and the trees in the lower
image). Compared with other types of regions, these regions may not be worse in
terms of pointwise difference measures (as shown in
the absolute error map). However, since the structural information of the image
details are nearly completely lost, they exhibit poorer visual quality. Close
piece-by-piece comparison of the SSIM index and the absolute error maps, we
observe that the SSIM index is more consistent with perceived quality
measurement. Note: in both distortion/quality maps, brighter means better
quality.
Original Images |
JPEG/JPEG2000 Compressed Images |
Absolute Error
Map |
SSIM Index Map |
|
|
|
|
|
|
|
|
The scatter plots of the subjective
measurement (mean opinion score, MOS) versus the objective predictions (PSNR
and MSSIM) are shown below, where each point represent one test image. Clearly,
MSSIM is much better in predicting the perceived image quality.
|
|
PSNR vs. MOS |
MSSIM vs. MOS |
Full LIVE database test
The
LIVE database was later developed to contain more diverse distortion types,
including JPEG2000 compression, JPEG compression, Gaussian noise contamination,
Gaussian blur, and JPEG2000 compressed images undergoing fast fading channel
distortions. The full database contains 982 images, with 203 original and 779
distorted images. The SSIM index was computed with the full database and the
SSIM values for all images are provided in the following file (in Matlab .mat format) for future comparisons:
SSIM
results for LIVE database
Created Feb. 2003
Last updated Feb. 11, 2011