Super-resolution microscopy based on interpolation and wide spectrum de-noising

In the conventional single-molecule localizations and super-resolution microscopy, the pixel size of a raw image is approximately equal to the standard deviation of the point spread function. Such a raw image is referred to herein as a conventional raw image, based on which better single molecule localization effect and efficiency can be achieved. It is found that both interpolation and de-noising can effectively improve the Signal to Noise Ratio of the conventional raw image. The conventional raw image, the de-noised, the interpolated and the de-noised interpolated are compared and analyzed and compressed sensing is used for super-resolution reconstruction. The simulation results show that both the highest Signal to Noise Ratio and the best super-resolution reconstruction can be obtained by de-noising the interpolated conventional raw image. This method also renders the best super-resolution reconstruction and minimum gradient in the real experiment. De-noising the interpolated conventional raw image is an effective method to improve the super-resolution microscopy.


Introduction
Optical microscopy imaging has several advantages such as magnification, real-time visualization and noninvasive observation, making optical microscope a basic tool for cell research.Owing to the diffraction phenomenon, the point light source forms a diffuse spot (i.e., an Airy disc) on the focal plane, which is the point spread function (PSF) of the microscope.The diffraction limit restricts an optical microscope to resolve the subcellular organization and cell organelles.However, super-resolution microscopy (SRM) imaging based on fluorescent probes can observe the inside of cells and subcellular life activities.It breaks the diffraction limit and increases the imaging resolution [1 -6].
Super-resolution microscopy (SRM) techniques include single-molecule localization microscopy (SMLM), stimulated emission depletion microscopy (STED), structured illumination microscopy (SIM) and superresolution optical fluctuation imaging (SOFI), etc., among which SMLM achieves the highest resolution.SMLM methods include (fluorescence) photoactivated localization microscopy ((F)PALM), stochastic optical reconstruction microscopy (STORM), and DNA-based point accumulation for imaging in nanoscale topography (DNA-PAINT), etc [1 -5, 7 -9].Since thousands frames of raw images in the same field of view are acquired for single molecule localization in SMLM, the temporal resolution of SMLM is very poor.Fluorescent molecules (fluorophores or emitters) must be sparsely activated in SMLM.If PSFs of some fluorescent molecules overlap, these fluorescent molecules will not be efficiently localized [3, 4, 10 -14].
Compressed sensing (CS) can reconstruct the raw image with high density of fluorescent molecules.PSFs of fluorescent molecules can overlap in CS, greatly reducing the number of frames of the raw image required to reconstruct the final super-resolution image.Thus the temporal resolution is much better than SMLM [13 -16].The CS mathematical model between a raw image and a super-resolution image is shown in Eq. (1).The vectors, y and x consist of columnwise concatenations of the raw image and the superresolution image (i.e., pixelated original image) respectively.The measurement matrix A is subject to the PSF of the microscope.The acquired raw image corresponds to the i th column of A if only one molecule emits fluoroscopic photons at the position index i of x [13 -16].
where xR N , yR M , AR M×N , M < N, M and N are natural numbers.x is a vector containing N elements.y is a vector containing M elements.A is a matrix of size M × N. min||x|| 0 is a objective function.y = Ax is a constraint function.If the objective function is min||x|| 1 rather than min||x|| 0 , Eq. ( 1) is transformed to a convex optimization problem.
In conventional SMLMs, if the pixel size of the raw image is approximately equal to the standard deviation (s.d.) of the PSF, good and fast localization can be achieved.The existing research results show that interpolation can also improve the CS reconstruction [13,14,17].Interpolation based on the bicubic method can make the pixel size of the raw image become half of the original's [14].
Noise in the raw image is inevitable.Especially in low light environment, noise can almost drown the signal.
The noises of a raw image mainly include a shot noise following a Poisson distribution, a readout noise following a Gaussian distribution and a background.The reconstruction of CS is also restricted by noise [4, 12, 17 -19].Large noise can even lead to reconstruction failure.Wide spectrum denoising (WSD) is a SRM denoising algorithm for various random noises, which helps improve the reconstruction of CS [18,20].Hereinafter, the raw image whose pixel size is approximately equal to the s.d is referred to as a conventional raw image (CR).CI is the interpolated CR.CW and CIW respectively represent the de-noised CR and CI with WSD.
The denoising effect and CS reconstruction of CIW are studied in this paper, and they also compared with those of CR, CW and CI.It is found that both interpolation and de-noising can effectively improve the Signal to Noise Ratio (SNR) of the CR and the highest SNR can be obtained by de-noising the CI.The simulation and real experimental results show that the best super-resolution reconstruction can be obtained by de-noising the CI.Thus de-noising the CI is an effective method to improve the super-resolution microscopy.

Wide spectrum de-noising and quantitative evaluation
WSD can effectively remove random noise such as Poisson and Gaussian noise from very low density to ultra-high density fluorescent molecular distribution scenarios.If the measurement matrix A is operated by orthogonalization and normalization, the measurement matrix A O can be obtained.Row orthogonal normalization is an operation in the matrix theory.It makes the rows of the matrix completely orthogonal.Moreover, it makes the 2norm of each row of the matrix equal to 1. Through A O and A, the operator matrix T which is equivalent to the row orthogonal normalization operation can be obtained, where T = A O A T (AA T ) -1 .Therefore, y = Ax can be equivalently converted to Ty = TAx.If a singular value decomposition is applied to T, T = USV T .S is a diagonal sparse matrix that is composed of singular values of T. V T is a transpose matrix of V. Therefore, Ty = TAx can be equivalently converted to SV T y = SV T Ax.Let z = SV T y. z's elements greater than cri are set to cri.The new z is represented by z WSD .The threshold value cri is the largest absolute values of elements of z between the indices M×0.9 and M×0.95.The de-noised raw image is y WSD , y WSD = T -1 Uy WSD [18,20].
SNR can indicate the overall quality of the de-noised raw image and the reconstructed super-resolution image [14].Structural similarity index measure (SSIM) can measure the similarity between two images.The SSIM values range between 0 and 1.If the two images are identical, the value of SSIM is equal to 1 [10,21].
where x is a true signal, xR N ; |||| 2 is the norm of a vector; and xR denotes the reconstructed signal corresponding x.
where  x and  x represent the mean and the variance of image x, respectively. xy represents the covariance of image x and y. c 1 = (k 1 L) 2 and c 2 = (k 2 L) 2 .L is the dynamic range of pixel values.k 1 = 0.01, k 2 = 0.03.

Interpolation and de-noising of simulated conventional raw images
In real experiments, an inverted optical microscope with a 100× oil immersion objective lens (N.A. 1.40) was used.Tubulin of HeLa cells was stained with Alexa-647.The fluorescence wavelength was 670 nm.The fluorescent lights were acquired using an EMCCD.Twenty CR frames were collected.The PSF was a Gaussian function.
The full width at half maximum (FWHM) of the PSF was 239.286 nm.The same parameters were used in simulation experiments.
To be distinguished from the pixels of the CR, the pixels of the super-resolution image were referred to as grids.The super-resolution image's grid was 1/8 of the pixel size of the CR.If the size of the CR was 7 × 7 pixels, it was located in the middle of the 64 × 64 grid superresolution image.The pixel size of the CI was 1/2 of the pixel size of the CR.
To evaluate the de-noising performance of WSD for different molecular densities and pixel sizes for CR and CI, simulated CRs with known real molecular positions were generated.The simulation randomly placed K molecules in a 64 × 64 grid region.K represented the sparsity in CS.K ranged from 1 to 16, with corresponding molecular densities from1.869 to 30 m 2 .The grid size was 11.43 nm.The effective pixel size of the CR, 45.714 nm, matched the pixel size of the microscope setup and was roughly equal to half of the s.d. of the PSF of the setup.The size of the CR was 7 × 7 pixels in the middle of the 64 × 64 grid super-resolution image.The simulation was for a photon number of 3,000 per molecule and a uniform background of 16 photons per pixel.Poisson noise and Gaussian noise (Gaussian noise variances of 0.01 and 0.0025) were added to each CR frame.The size of the CI was 13×13 pixels.
Fig. 1 is mean SNRs of 500 frames of CR and CI before and after de-noising.For each K, the simulation was repeated 500 times as shown in Fig. 1.CI is the CI.CW and CIW respectively represent the CW and CI with WSD.
The de-noising performance analysis curves are shown in Fig. 1a and b.Both interpolation and de-noising can effectively improve the SNR of the CR.The highest SNR can be obtained by de-noising the CI.In Fig. 1a for the high noise environment (Gaussian noise with a variance of 0.01), the SNR of the CI is lower than that of the CW.In Fig. 1b for the low noise environment (Gaussian noise with a variance of 0.0025), the SNR of the CI is almost the same as that of the CW.

Simulation data analysis
To evaluate the CS reconstruction for CIW, 20 frames of simulated super-resolution image with known true molecular positions and their corresponding CR frames were generated.The simulation randomly placed molecules in a grid region.The molecular density was 6.304 m -2 .
Fig. 4a is a CR frame without noise.Fig. 4b is a CI frame without noise.Fig. 4c is the true superresolution image.5a and c.SSIMs of Fig. 5b and d increased by 0.14 and 0.075 than Fig. 5a and c.
The lower row of Fig. 5 is the final super-resolution image reconstructed by CS.SSIM and SNR of Fig. 5h reached the maximum, respectively 0.671 and -2.021 dB.The cell microtubule structure is clearer and thinner.SSIM and SNR increased by 0.097 and 0.072 dB respectively compared with Fig. 5f.The CS reconstruction of Fig. 5e and g failed.No valid microtubule structures can be seen.Although some scattered structures can be seen in Fig. 5g, the result is too poor to be useful.The microtubule structure of the rectangle area in Fig. 5h is cleaner than that in Fig. 5f.All three microtubules are clearly identifiable in Fig. 5h.However, only two microtubules can be seen clearly in Fig. 5f.Therefore, the CS reconstruction of Fig. 5h is the best.No valid microtubule structures can be seen.The area between microtubules in the rectangular area in Fig. 6h has almost no discrete points, which is cleaner than that in Fig. 6f and g.Therefore, the CS reconstruction of Fig. 6h is the best.

Real experimental data analysis
Fig. 7 shows the real experimental CR and CI before and after de-noising and the corresponding CS reconstruction results.Fig. 7a -d of the upper row is CR and CI frames before and after de-noising.Since the real raw image cannot be known, the sum of the absolute values of the horizontal and vertical gradients of the raw image is used as an indicator of measuring the de-noising effect.The gradients of CR, CW, CI, and CIW were 7.458×10 5 , 7.146×10 5 , 4.25×10 5 and 3.974×10 5 respectively.Through comparison of the images, it is obvious that the raw image after denoising is smoother than that before denoising, indicating prominent denoising effects.
Comparing the CS reconstruction results of CR, CW, CI and CIW in the lower row, we can see noticeably improved spatial resolutions of CI and CIW and clearer, thinner cell microtubule structure.The super-resolution reconstruction of Fig. 7f is the worst, and Fig. 7g is the second-worst.The peaks in Fig. 7e and j correspond to the photoncount profiles at the red line in Fig. 7h and i.The highest peak value is 4300.The peak in Fig. 7j is thinner and less than Fig. 7e.Therefore, the super-resolution reconstruction of Fig. 7i is the best.

Conclusion
In the conventional single-molecule localizations and super-resolution microscopy, the pixel size of a raw image is approximately equal to the standard deviation of the point spread function.Both interpolation and denoising can effectively improve the SNR of the CR.It is found that the highest SNR and minimum gradient can be obtained by de-noising the CI.In the high noise environment, the SNR of the CI is lower than that of the CW and the CI cannot achieve effective super-resolution reconstruction, while in the low noise environment, the SNR of the CI is almost the same as that of the CW.The simulation and real experimental results show that the best super-resolution reconstruction can be obtained by de-noising the CI.In conclusion, de-noising the CI is an effective method to improve the super-resolution microscopy.

Fig. 1 . 2 .
Fig. 1.Comparison of mean SNRs based on 500 simulated CRs before and after interpolation and de-noising.The simulation is for a photon number of 3,000 per molecule and a background of 64 photons per pixel.Poisson noise and Gaussian noise are added to each frame of the CR.The y axis is labeled as SNR (dB).The x axis is labeled as both molecular density and signal sparsity.a) Gaussian noise variance of 0.01, b) Gaussian noise variance of 0.0025

3 . 5 . 6 .
Fig. 5a is one CR frame after adding background, Poisson noise and Gaussian noise with variance of 0.01 to Fig. 4a.Fig. 5b is the CW frame corresponding to Fig. 5a.Fig. 5c and d respectively are the CI and the CIW frames.SNRs of Fig. 5b and d increased by 4.338 dB and 2.176 dB than Fig.5a and c.SSIMs of Fig.5b and dincreased by 0.14 and 0.075 than Fig.5a and c.The lower row of Fig.5is the final super-resolution image reconstructed by CS.SSIM and SNR of Fig.5hreached the maximum, respectively 0.671 and -2.021 dB.The cell microtubule structure is clearer and thinner.SSIM and SNR increased by 0.097 and 0.072 dB respectively compared with Fig.5f.The CS reconstruction of Fig.5eand g failed.No valid microtubule structures can be seen.Although some scattered structures can be seen in Fig.5g, the result is too poor to be useful.The microtubule structure of the rectangle area in Fig.5his cleaner than that Fig. 6a is one CR frame after adding background, Poisson noise and Gaussian noise with variance of 0.0025 to Fig. 4a.SNRs of Figs.6b and d respectively increased by 3.599 dB and 2.104 dB than Figs.6a and c.SSIMs of Figs.6b and d respectively increased by 0.097 and 0.06 than Figs.6a and c.The second row of Fig. 6 is the final super-resolution image reconstructed by CS.SSIM and SNR of Fig. 6(h) reached the maximum, respectively 0796 and -1.411 dB.The cell microtubule structure is clearer and thinner.SSIM and SNR increased by 0.159 and 0.349 dB respectively compared with Fig. 6f.SSIM and SNR are better than Fig. 6g, too.The CS reconstruction of Fig. 6e failed.

7 .
Comparison of the experimental (interpolated) CR and the corresponding final super-resolution images before and after denoising.(a)-(d) of the upper row is one frame of the experimental CR and CI before and after de-noising.The number indicate gradient of the images.(f)-(i) of the lower row is corresponding final super-resolution images before and after denoising, respectively.(e) and (i) is plots of photon-count profiles obtained by measurements made along the red lines in (h)-(i).The maximum wave peak is 4300.Scale bars: 1 μm.a) CR 7.458×10 5 , b) CW 7.146×10 5 , c) CI 4.25×10 5 , d) CIW 3.974×10 5 , e) CI-profile, f) CR, g) CW, h) CI, i) CIW, j) CIW-profile