Performance Evaluation of Filtered Back Projection Reconstruction
and Interative Reconstruction method for PET Images
Cliff X. Wang1,2, Wesley E.Snyder1,2 Griff Bilbro2, Pete Santago1 Dept. of Radiology, Bowman Grey School of Medicine, Winston-Salem, NC 27157 USA1 Dept. of Electrical Engineering, North Carolina State University, NC 27695 USA2
Abstract--The Filtered-backprojection(FBP) algorithm and statistical model based iterative algorithms such as the maximum likelihood(ML) reconstruction or the maximum-a-posteriori (MAP) reconstruction are the two major classes of tomographic reconstruction methods. The FBP method is widely used in clinical setting while iterative methods has attracted research interests in the past decade. In this paper we studied the performance of the FBP, the ML and the MAP methods using simulated projection data. The experiment showed that the MAP algorithm generated superior image quality in terms of the bias, the variance, and the average MSE measures.
Image reconstruction, Performance evaluation, PET imaging, Bias and Variance, Mean square error
1. INTRODUCTION
Tomographic image reconstruction has been an active research field in recent years. The filtered-backprojection method(FBP) and statistical model based iterative algorithms are the two major classes of reconstruction methods. The FBP method is widely used in clinical settings for its speed and easy implementation. Iterative algorithms take account the statistical nature of the acquired projection data and incorporate the physical model into reconstruction. Typical iterative methods include the maximum likelihood and the maximum a-posteriori algorithm. Since these two classes of algorithms are quite different in their approaches to tomographic image reconstruction, it is useful to study and compare the performance of these methods.
There have been several previous studies on image quality and noise analysis of the FBP algorithm[1],[2] and the ML-EM algorithm[3],[4], but no comparison is made. In one recent study[5], the receiver operator characteristic (ROC) method was used and the results revealed that the iterative ML-EM algorithm didn't show obvious diagnostic advantage over the FBP method for cardiac nuclear medicine images. Wilson et al[6] used the covariance matrix to study the image noise magnitude and texture. Their findings were similar to that of reference [5]. The MAP reconstruction method, which incorporates a-priori image model to regularize the ML reconstruction, may provide superior image quality for certain type of images. A careful study needs to be carried out to compare the performance of the FBP, the ML, and the MAP reconstruction methods. In this study, we generated a sequence of 50 projection data using a PET simulation software. Image reconstruction were carried out on these data to produce a sequence of images. Using the variance, the bias and the average MSE criteria, the reconstruction performance of the FBP, the ML, and the MAP methods was evaluated.
2. METHOD
2.1 Sinogram Simulation
The PET scanner is used to detect the positron emission from the radionuclides. The annilation event of the positron which produces two photons in opposite direction is detected near simultaneously by a detector pair. Over a period of time, the coincidental photon detection on every possible detector pair forms the projected data(sinogram). For the Siemens 951 scanner, there are 192 pairs of detectors per angular position and measurements are made on 256 angular positions. Thus the sinogram is of size 256 rows by 192 columns.
The simulator models the Siemens 951 PET system to generate the projected data (sinograms). A square pattern source image was used to produce a sequence of 50 projected data (sinograms). Since the positron emission follows the Poisson distribution, the simulated sinograms differ from each other statistically. However, they came from the same source thus represented the same source image. These sinograms were then reconstructed using different algorithms.
2.2 Reconstruction
The image reconstruction was carried out using the FBP, the ML, and the MAP algorithms. For the FBP algorithm, the filter cutoff frequency is an important parameter to select. For this experiment, a Hann filter was chosen and the cutoff frequency ranging from 0.1 pixel/cycle to 1 pixel/cycle was examined. For the ML algorithm, the reconstructed image was saved for every 10 iterations up to 100 iterations. The MAP reconstruction was performed on the same set of sinograms. The initial estimate image for the MAP method was a ML reconstructed image after 30 iterations.
2.3 The Variance and Bias Measure
The variance and the bias measures are used to evaluate image quality. These two measures are calculated from a set of images generated from the same source and passed through the same reconstruction process. The bias measure is defined as the mean square error between the average image and the truth image. It indicates how much difference is generated between the group average image and the truth image. The variance is used to measure the difference among a set of images. It is a measure of how consistent these images are.
For a sequence of images, the bias measure can be defined as follows,
where Xi(T) is the value of the ith pixel in the truth image. 
is the ith pixel value of the average image, defined as
where Xi(j) is the ith pixel value of the jth image in the sequence. M is the total number of images in the sequence.
The variance of a sequence of images is defined as:
where M is the total number of images and N is the total number of pixels in the whole image or in the region of interests (ROI).
2.4 Mean Square Error (MSE) measurement
The mean square error measures the difference between a reconstructed image and the truth image. It is defined as follows,
where MSE(j) is the mean square error between the jth image and the truth image. For each image in the sequence, its mean square error can be calculated. Average mean square error is calculated from the sequence of images.
3. RESULTS
In this study, a PET simulator was used to produce the sinograms. For the square pattern image shown in Figure 7a, fifty sinograms were generated by simulating the positron emission in a Siemens 952 PET scanner. From these sinograms, images were reconstructed using the FBP, the ML, and the MAP methods with different parameters. The bias, the variance, and the MSE measures were calculated from these images. The results are tabulated in Table 1 for the FBP method, in Table 2 for the ML method, and in Table 3 for the MAP method. Figure 1 through 6 plot these results.
Table 1: Quality measures for the FBP reconstruction (per pixel)
------------------------------------------------------
Bias Variance Average MSE
MSE Std
------------------------------------------------------
No filter 5337.216 4.419 5340.897 32.729
Hann, cutoff 0.1 782.52 2.271 784.232 5.79
Hann, cutoff 0.2 305.949 11.935 316.456 3.836
Hann, cutoff 0.3 197.14 34.061 228.38 4.34
Hann, cutoff 0.4 159.856 70.954 226.064 5.517
Hann, cutoff 0.5 145.24 124.569 260.54 7.08
Hann, cutoff 0.6 139.518 189.465 316.449 8.794
Hann, cutoff 0.6 137.65 261.936 383.61 10.52
Hann, cutoff 0.8 137.486 330.523 451.81 12.138
Hann, cutoff 0.9 138.04 394.805 514.81 13.59
Hann, cutoff 1.0 138.853 451.042 570.200 14.866
------------------------------------------------------
Table 2: Quality measures for the ML reconstruction (per pixel)
------------------------------------------------------
Bias Variance Average MSE
MSE Std
------------------------------------------------------
ML 10 iterations 271.04 10.23 302.29 4.39
ML 20 iterations 140.22 28.59 190.43 4.49
ML 30 iterations 109.74 51.88 187.41 5.12
ML 40 iterations 97.92 78.71 205.80 5.99
ML 50 iterations 92.94 106.36 231.94 7.02
ML 60 iterations 89.22 136.04 261.87 8.14
ML 70 iterations 87.55 166.79 293.94 9.32
ML 80 iterations 86.66 198.34 327.36 10.54
ML 90 iterations 86.27 230.51 361.70 11.79
ML 100 iterations 86.22 263.17 396.69 13.06
------------------------------------------------------
Table 3: Quality measures for the MAP reconstruction (per pixel)
----------------------------------------
Bias Variance Average MSE
MSE Std
----------------------------------------
MAP 102.27 5.45 136.67 5.109
----------------------------------------
Table 4: The best measures of the bias, the variance, and the average MSE
------------------------------------------------------------------
Minimum Bias Minimum Variance Minimum MSE
------------------------------------------------------------------
FBP 137.486 2.271 226.064
(cutoff at 0.8) (cutoff at 0.1) (cutoff at 0.4)
ML 86.22 10.23 187.41
(100 iterations) (10 iterations) (30 iterations)
MAP 102.27 5.45 136.67
------------------------------------------------------------------
Table 4 lists the best measures of the bias, the variance, or the average MSE that each reconstruction method can produce. The FBP method generates the minimum bias at the filter cutoff frequency of 0.8 pixel/cycle, the minimum variance at 0.1 pixel/cycle, and the minimum average MSE at 0.4 pixel/cycle. The minimum bias for the ML algorithm is at 100 iterations. The minimum variance and the minimum MSE is at 10 iterations and 30 iterations respectively.
Figure 1. The bias of the 50 images reconstructed by the FBP algorithm. The X axis is the cutoff frequency of the FBP method and Y is the bias.
Figure 2. The variance of the 50 images reconstructed by the FBP algorithm. The X axis is the cutoff frequency of the FBP method and Y is the variance.
Figure 3. The average mean square error of the 50 images reconstructed by the FBP algorithm. The X axis is the cutoff frequency of the FBP method and Y is the average MSE.
Figure 4. The bias of 50 images reconstructed by the ML or the MAP algorithms. The X axis is the number of iterations for the ML method and Y is the bias.
Figure 5. The variance of 50 images reconstructed by the ML or the MAP algorithms. The X axis is the number of iterations for the ML method and Y is the variance.
Figure 6. The average mean square error of 50 images reconstructed by the ML or the MAP algorithms. The X axis is the number of iterations for the ML method and Y is the MSE.
Figure 7. Some of the images. Top row from left to right: a) The truth image; b) FBP reconstructed image with no filter; c) FBP reconstructed image with the Hann filter at cutoff frequency of 0.4 pixel/cycle; d) FBP reconstructed image with the Hann filter at cutoff frequency of 1 pixel/cycle. Bottom row from left to right: e) MAP reconstructed image; f) ML reconstructed image after10 iterations; g) ML reconstructed image after 30 iterations; h) ML reconstructed image after 100 Iterations.
4. DISCUSSION
4.1 Discussion on the FBP Result
The filtered back projection reconstruction is based on the Fourier transform theory and a linear signal model is assumed. A typical back projected image is blurred by a point spread function(PSF) of the form 1/(x2+y2)1/2[7]. To restore the blurred image, the high frequency band of the reconstructed image needs to be enhanced by using a Hann or some other type of filters. The selection of the filter cutoff frequency is of crucial importance to the final reconstructed image quality. A low cutoff frequency can not fully restore the blurring and the reconstructed image looks over smooth. A high cutoff frequency, on the other hand, introduces extra noise. An appropriate cutoff frequency selection would make the image sharp enough but keep the noise level low. In this project, the cutoff frequency from 0.1 pixel/cycle to 1 pixel/cycle was examined. The bias, the variance, and MSE measures are listed in Table 1. Corresponding plots are in Figure 1 through Figure 3.
The bias measure indicates how accurate the reconstructed image is and the variance measures the noisiness of the reconstructed images. From Table 1 and Figure 1 and 2, it can be seen that with the increase of the filter cutoff frequency, the bias falls but the variance increases. When no filter is applied, the reconstructed images are the least noisy. The over smoothed image (Figure 7b) differs significantly from the truth image and many details are missing. As the filter frequency increases, the image improves in sharpness but suffers from increased noise. A typical noisy reconstructed image is shown in Figure 7d. A practical selection of the filter cutoff frequency is to compromise the image sharpness with the suppression of noise. Sometimes the cutoff frequency may be selected according to clinical needs. For example, a high cutoff frequency may be used in order to observe a fine structure in the reconstructed image.
In addition to the bias and the variance measures, the mean square error(MSE) is another commonly used criteria to evaluate image quality. In this experiment, the average MSE is calculated and plotted in Figure 3 for the FBP method. It can be seen that a cutoff frequency in the range of 0.2 pixel/cycle to 0.4 pixel/cycle produces the minimum average MSE. This cutoff frequency range gives the best compromise between the bias and variance criteria, thus producing images with the minimum MSE. The image shown in Figure 7c has a cutoff frequency at 0.4 pixel/cycle, and appears to have a better quality than Image 7b and Image 7d.
4.2 Discussion on the ML Result
Since the introduction of the maximum likelihood (ML) reconstruction method for emission tomography by Shepp and Vardi[8], there has been a great interest on iterative reconstruction methods. However, since the ML algorithm is ill-conditioned and highly non-linear, it is very difficult to analyze and evaluate its performance. In this project, the performance of the ML algorithm is studied by applying the bias, the variance, and the MSE criteria over 50 reconstructed images. The results are listed in Table 2 and plotted out in Figure 4 to 6.
The iteration number of the ML algorithm behaves very similarly to the filter cutoff frequency of the FBP algorithm. At low iteration number the bias is high, but the variance is low. As more iterations are performed, the bias measure improves, but the reconstructed image becomes noisy and the variance measure starts deteriorating. Therefore the number of iteration for the ML method has to be selected carefully. The average MSE indicates that around 30 iterations provides the best image quality. Figure 7f through 7h displays the ML reconstructed images with different number of iterations.
4.3 Discussion of the MAP method in comparison with the FBP and the ML method
Table 3 lists the best measures of the bias, the variance, and the MSE from the three reconstruction methods under study. For both the FBP and ML algorithm, the image smoothness contradicts the image sharpness. At low filter cutoff frequency for the FBP method or at low iteration number of the ML method, the image smoothness (indicated by the variance measure) is achieved at the expense of image sharpness (fidelity). The over-smoothed image doesn't reflect the truth image's full features. On the other end, when the cutoff frequency is set high for the FBP method or after many iterations of the ML method, the reconstructed image does reveal the fine details of the truth image, but at the cost of excessive noise. A moderate filter cutoff frequency or a moderate number of iterations gives the best compromise. This medium range selection of the filter cutoff frequency or the iteration number generates the minimum MSE.
One major reason that the FBP method suffers from excessive noise is that it assumes a linear signal model to restore the blurred images. A space invariant blurring point spread function (PSF) is used to correct the image blurring. The Poisson noise which degrades the sinogram is not taken into consideration in the reconstruction process. In reality, the nuclear medicine images generated from radioactive isotopes are highly nonlinear and can often be modeled as a random field. This major drawback prompted the exploration of using statistical model based reconstruction in the past two decades. However, the FBP method is still the major reconstruction method in clinical use, largely due to its speed, the simplicity to implement, and its relative good image quality if the filter is used appropriately.
The ML method incorporates the statistical model of emission tomography into the reconstruction process. Given the projected data, the ML reconstruction attempts to estimate a final image which maximizes the probability of this reconstructed image. In practice, the ML algorithm has two major drawbacks. First, the ML algorithm is ill-conditioned and unstable. As a consequence, a good solution is not guaranteed. As iterations go on, the image estimate may deteriorate and become excessive noisy. Second, the ML algorithm converges slowly[9]. Two major approaches to improve the ML algorithm have been proposed. One is using some stopping criteria to halt the iteration before deterioration and the other is to apply regularization[9],[10].
In our approach to improve the ML algorithm, a uniform piecewise a-priori model is used to
regularize the ML reconstruction into a maximum a-posteriori (MAP) reconstruction[11]. The MAP reconstruction enforces the a-priori model to the ML algorithm. By taking the natural log of the a-posteriori probability and changing the sign, the MAP reconstruction is posed as a function minimization process [11]. The final estimate of the reconstructed image maximizes the a-posteriori probability, which contains the likelihood term and the a-priori term. As a result, the final reconstructed image is the optimum solution to the global energy function minimization and it is a compromise of the ML reconstruction and the a-priori image model.
The performance of the MAP algorithm is tabulated in Table 2 and plotted in Figure 4 through 6. From the results, it can be seen that the MAP algorithm generates very favorable quality measures. For the FBP and ML method, there is a compromise between the bias and the variance criteria, as discussed in the preceding sections. As an improvement to the ML algorithm, the MAP method can reconstruct images with low bias and variance measures at the same time. From Table 3, it can be seen that the MAP algorithm produces the smallest variance and average MSE measures among all three methods. The bias measure is a little bigger than that of the ML algorithm, however, it is almost in the same magnitude. In addition, if studying carefully on the bias and MSE results, we can suggest that the low bias measure for the ML algorithm is largely credited to the averaging of the 50 images instead of the reconstruction algorithm. The bias measure is the MSE between the average image and the truth image. The average MSE is the mean of the MSEs from every image in the sequence. If every image in the sequence is same to each other, then the bias measure equals the average MSE. For the MAP reconstruction, the average MSE is very close to the bias, indicating that the reconstructed images are very similar to each other and the random noise level is low. For the ML algorithm, the average MSE is almost twice to the bias measurement at 30 iterations and almost 4 times bigger at 100 iterations, which suggests that each image has considerable noise content. Averaging over the images greatly reduces the noise and results in low bias measure, especially at high number of iterations.
A typical MAP reconstructed image is shown in Figure 7e. It can be seen that the MAP image is much less noisy and maintains sharp edges at the same time. The analysis on the bias, the variance, and the average MSE measures indicates that the MAP algorithm can produce images with superior quality in comparison with the ML or the FBP methods.
5. CONCLUSION
In this study, we used the bias, the variance, and the average MSE measures on a sequence of simulated images to evaluate reconstruction algorithms. The result indicates that the best quality image that the FBP or the ML algorithm can generate is a compromise of image smoothness and image sharpness. The filter cutoff frequency for the FBP algorithm and the iteration number of the ML algorithm have to be selected carefully. By enforcing the a-priori model into the ML algorithm, the MAP reconstruction can generate superior quality image in terms of the bias, the variance, and the average MSE measures.
REFERENCES
1. S.J. Riederer, N.J. Pelc, D.A. Chesler, "The noise power spectrum in computed tomography", Phys.Med.Biol., 23, 446-454 (1978)
2. M.F. Kijewski, P.F.Judy, "The noise spectrum of CT images", Phys.Med.Biol., 32, 565-575 (1987)
3. D.L.Snyder, M.I. Miller, L.J.Thomas, D.G.Politte, "Noise and edge artifacts in maximum- likelihood reconstruction for emission tomography", IEEE Trans. Medical Imaging, 6,
313-319 (1987)
4. J.S. Liow, S.C. Strother, "Practical trade-off between noise, quantitation, and number of iterations for maximum likelihood-based reconstruction" IEEE Trans. Medical Imaging, 10, 563-572 (1991)
5. D.R. Gilland, B.M.W. Tsui, C.E. Metz, R.J. Jaszczak, J.R. Perry, "An evaluation of maximum likelihood-EM reconstruction for SPECT by ROC analysis", J. Nuclear Med., 33,
451-457 (1992)
6. D.W. Wilson, B.M.W. Tsui, "Noise properties of filtered backprojection and ML-EM reconstructed emission tomographic images", IEEE Trans. Medical Imaging, 40,
1198-1203(1993)
7. A.K. Jain, Fundamentals of Digital Image Processing, Prentice Hall (1989)
8. L.A. Shepp, Y. Vardi, "Maximum likelihood reconstruction for emission tomography", IEEE Trans. Medical Imaging, 1, 113-121 (1982)
9. T. Hebert, R.Leahy, "A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors", IEEE Trans. on Medical Imaging, 8, 194-202 (1989)
10. J.Llacer, E.Veklerov "Feasible images and practical stopping rules for iterative algorithms in emission tomography", IEEE Trans. on Medical Imaging, 8, 186-193 (1989)
11. Y.S. Han, "Annealing methods in iterative image processing with medical application", Ph.D thesis, North Carolina State University (1993)