The Mean Square Error is a single number which embodies the total amount of error between two images. As Anna said on the newsgroup, one would have three graphs for each of the RGB's.
You would also combine the RGB values to form a luminosity L, as in the assignment:
L = (27R + 53G + 11B)/100
and perform the MSE calculation on the L value.
Transparency is not considered at all.
Let's try an example. Given a 2x3 image X:
r10 g20 b30 r30 g20 b10
r20 g30 b30 r10 g20 b30
r30 g20 b10 r10 g20 b30
and (modified) Y:
r20 g30 b30 r10 g20 b30
r10 g20 b30 r30 g20 b10
r30 g20 b10 r10 g20 b30
the difference Y-X is the pixel-by-pixel difference and is:
r10 g10 b00 r-20 g00 b20
r-10 g-10 b00 r20 g00 b-20
r00 g00 b00 r00 g00 b00
and the luminosity difference using the calculation in the assignment, L is:
2.7+5.3+0.0 -5.4+0.0+2.2 8.0 -3.2
-2.7-5.3+0.0 5.4+0.0-2.2 = -8.0 3.2
0.0 0.0 0.0 0.0
so
Mxy_red = sum((Yr-Xr)^2)/N = (2*100 + 2*400)/6 = 1000/6 = 166.66
Mxy_green = sum((Yg-Xg)^2)/N = 2*100/6 = 200/6 = 33.33
Mxy_blue = sum((Yb-Xb)^2)/N = 2*400/6 = 800/6 = 133.33
Mxy_lumin = sum((Yl-Xl)^2)/N = (2*64+2*10.24)/6 = 148.48/6 = 24.74
The SNR is then
Rxy_red = sum((Xr)^2)/(N*Mxy_red)
= (3*100+400+2*900)/1000 = 2500/1000 = 2.5
Rxy_green = sum((Xg)^2)/(N*Mxy_green)
= (5*400+900)/200 = 2900/200 = 14.5
Rxy_blue = sum((Xb)^2)/(N*Mxy_blue)
= (2*100+4*900)/800 = 3800/800 = 4.75
Rxy_lumin = sum((Xl)^2)/(N*Mxy_lumin)
= (3*16.6^2 + 2*19.8^2 + 24.6^2)/148.48 = 2215.9/148.48 = 14.92
and
Sxy_red = 10*log10(Rxy_red) = 10*log(Rxy_red)/log(10) = 3.9 dB
Sxy_green = 11.6 dB
Sxy_blue = 6.8 dB
Sxy_lumin = 11.7 dB
So what do these numbers mean?
Looking at the maths, the more difference there is between images X and Y, the more Mxy will increase. Because Mxy is on the denominator of Rxy, Rxy increases with increasing similarity between X and Y, and Sxy is a logarithmic version of that measurement.
Statistically, the SNR is a measure of the variance of the signal against the variance of the noise in the signal. Therefore, the larger the SNR, the better the signal.
Notice that Rxy is dependant not only on the difference Yi-Xi, but also on Xi. This means that the signal to noise ratio Rxy (and Sxy) are image-dependent. This makes it not so useful as a measure between different images.
To account for this, another common measure is the PSNR - the Peak Signal to Noise Ratio. It is defined as
Pxy = 10 * log10(255^2/MSE)
= 20 * log(255/sqrt(MSE))/log(10)
In other words, we are now comparing the error difference against a 'maximum-noise' image or statistically, the error variance against the maximum possible image variance. For the above example, the PSNR becomes:
Pxy_red = 20*log10(255/sqrt(Mxy_red)) = 25.9 dB
Pxy_green = 20*log10(255/sqrt(Mxy_green)) = 32.9 dB
Pxy_blue = 20*log10(255/sqrt(Mxy_blue)) = 26.9 dB
Pxy_lumin = 20*log10(255/sqrt(Mxy_lumin)) = 34.2 dB
If X were an original image and Y was the same image saved and reloaded as a JPEG file, then Sxy would typically be around 40dB. If you used a very low 'quality factor' when saving X as a JPEG format file (say, around 50%), then you might expect Sxy to be around 20dB or less. Compare that to the values above.
The actual value of the PSNR is not necessarily very meaningful. You can see in the MSE equation that it can't tell the difference between many small errors that add up to a certain amount and a few large errors that might add up to the same amount. Thus a perfext image copy with a piece missing might show a lower or higher PSNR than a an image copy blurred all over. Nevertheless, this is the most common method of comparing image damage. In the assignment, while you may choose to display the SNR or PSNR, the latter is preferred.