Super-resolution microscopy based on wide spectrum denoising and compressed sensing

WSD can effectively remove random noise of a raw image from very low density to ultra-high density fluorescent molecular distribution scenarios. The size of the raw image that WSD can denoise is subject to the used measurement matrix. A large raw image must be divided into blocks so that WSD denoises each block separately. Based on traditional single-molecule localization and super-resolution reconstruction scenarios, wide spectrum denoising (WSD) for blocks of different sizes was studied. The denoising ability is related to block sizes. The general trend is when the block gets larger, the denoising effect gets worse. When the block size is equal to 10, the denoising effect is the best. Using compressed sensing, only 20 raw images are needed for reconstruction. The temporal resolution is less than half a second. The spatial resolution is also greatly improved.


Introduction
Far-field optical microscopy imaging in the visible light band has the advantages of noncontact, being nondestructive, and allowing internal sample probing.Owing to the diffraction phenomenon, the point light source cannot form a point image on the focal plane after passing through the microscope imaging system.It forms a diffuse spot (i.e., an Airy disc).Because of the diffraction limit, the lateral resolution is only 200 nm [1 -4].
Far-field optical super-resolution microscopy imaging beyond the diffraction limit has always been a popular research topic [5 -7].Both Stochastic optical reconstruction microscopy (STORM) and (fluorescence) photoactivated localization microscopy ((F)PALM) are based on the imaging of sparsely distributed molecules and singlemolecule localizations (SMLs) [1,2,8,9].In these SMLbased microscopic techniques, only a portion of the sparsely distributed molecules is imaged and localized in each raw image.The final super-resolution image requires thousands of raw images.The lateral resolutions are approximately 20 nm [1,2,7,10,11].
The noise of a raw image acquired by an electronmultiplying charge-coupled device (EMCCD) mainly includes a shot noise following a Poisson distribution, a readout noise following a Gaussian distribution and a background [9, 12 -14].SML algorithms, such as the Gaussian fitting method, have good anti-noise ability [1, 2, 7 -9, 15].Therefore, the denoising preprocessing is not necessary for raw images.Various studies on fluorescent molecular localization in the field of super-resolution microscopy rarely report denoising of raw images before localization [1, 2, 5 -9, 15].
Compressed sensing (CS) is a super resolution microscopy reconstruction method which is different from SMLs.CS can increase the maximum density of fluores-cent molecules in a raw image, even if these fluorophores are not distributed so sparsely that their images overlap with one another [7,9,16,17].Because the readout noise of EMCCD cameras and its variance are small, raw image simulations sometimes consider only the background and the Poisson noise [9, 12 -14].
The requirements of super-resolution microscopy experiment for microscopes, cameras and fluorescent dyes are very high.If cheap microscopes, cameras and fluorescent dyes are used, the acquired raw images often contain a lot of noises.It is almost impossible to reconstruct any effective super-resolution cell microtubule structure [8,17,18].Wide spectrum denoising (WSD) is applicable to various random noises.It is an efficient denoising algorithm for raw images [17,18].
In the traditional SMLs and super-resolution reconstruction, if the size of the raw image pixel is approximately equal to the standard deviation (s.d.) of the point spread function (PSF), good localization and reconstruction effects can be achieved [6 -9, 16].Therefore, a large number of traditional experimental data accumulated in the past were based on the size of the raw image pixel about equal to the s.d. of the PSF [5,6,10,19].
Based on the traditional experimental data, we studied the denoising effect of WSD on low-density and highdensity raw images and its relationship with the block sizes.Simulation and experimental results show that WSD has good denoising ability to the traditional experimental data, and the corresponding super-resolution reconstruction is obviously better and faster.The temporal resolution of half a second was realized.

PSF-based measurement matrices corresponding to different sizes of raw images
The PSF is the light-intensity distribution function at work in the image plane when light from an infinitely small point object passes through an optical system, such as a microscope.The imaging process of the optical system is the convolution of the object function and PSF [9, 12,14,16,20].
Eq. ( 1) shows a mathematical model between a raw image and a super-resolution image after converting a 2D image to a 1D vector [16 -18].The vectors, y and x consist of row-wise or column-wise concatenations of the raw image and the super-resolution image (i.e., pixelated original image) respectively.
The measurement matrix A is determined by the PSF of the imaging system.The acquired raw image corresponds to the i th column of A if only one molecule emits fluoroscopic photons at the position index i of x [9,16].,  y Ax (1) where xR N , yR M , AR M×N , M < N, M and N are natural numbers from 1. x is a vector containing N elements.y is a vector containing M elements.A is a matrix of size M ×N.
To be distinguished from the pixels of the raw image, the pixels of the super-resolution image are referred to as grids.The super-resolution image's grid is 1/8 of the pixel size of the raw image.If the size of the raw image is 7×7 pixels, it is located in the middle of the 64 × 64 grid super-resolution image.At this point, the size of the corresponding measurement matrix A is 49 × 4096.The matrix A is labeled as A7.If the size of the raw image is 37 × 37 pixels, the size of the corresponding measurement matrix A (i.e., A 37 ) is 1369 × 92417.In this paper, 31 matrices (i.e., A 7 -A 37 ) were studied.

Wide spectrum denoising (WSD)
WSD can be effectively used to remove random noise such as Poisson and Gaussian noise from very low density to ultra-high density fluorescent molecular distribution scenarios [17,18].If the measurement matrix A is operated by orthogonalization and normalization, the measurement matrix A O can be obtained.Row orthogonal normalization is an operation in the matrix theory.It makes the rows of the matrix completely orthogonal.Moreover, it makes the 2-norm of each row of the matrix equal to 1. Through A O and A, the operator matrix T which is equivalent to the row orthogonal normalization operation can be obtained, where T = A O T (AA T ) -1 .Therefore, y = Ax can be equivalently converted to Ty = TAx.If a singular value decomposition is applied to T, T = USV T .S is a diagonal sparse matrix that is composed of singular values of T. V T is a transpose matrix of V. Therefore, Ty = TAx can be equivalently converted to SV T y = SV T Ax.
According to the geometrical and physical meanings of the square matrices S and V, V can change the direction of a vector, and S can change the magnitude (i.e., l 2norm) of a vector.S can be regarded as a highdimensional ellipsoid projection operator with the singular values of T as the axis of the high-dimensional ellip-soid.The long axis is much longer than the short axis in the high-dimensional ellipsoid [17].
Regardless of the type of noise, it is approximately orthogonal to the signal as long as it is random noise.The noise-free raw image was projected onto the short-axis region of the high-dimensional ellipsoid of S. Thereafter, the noise that was approximately orthogonal to this image was projected onto the long-axis region of the highdimensional ellipsoid of S, resulting in a sharp amplification of the noise.S is a sparse matrix whose elements are zero except for the diagonal elements.In the matrix theory, such a matrix is referred to as a high-dimensional ellipsoid with diagonal elements as its long-and short-axes [17].
Let z = SV T y.The threshold value cri is the largest absolute values of elements of z between the indices M×0.9 and M×0.95.M is the number of rows of the measurement matrix A and it also represents the number of pixels in the block.z is a vector containing M elements.M×0.9 and M×0.95are respectively the integer part of M × 0.9 and M × 0.95.The index is the serial number (or, sequence number) of elements of z from 1 to M. Through numerous experiments, we have found that the values of the elements in z without noise can rarely be greater than cri.Therefore, z's elements greater than cri are set to cri.The new z is represented by z WSD .In this way, the noise of the raw image can be effectively removed.The denoised raw image using WSD is y WSD , y WSD =T - 1 Uz WSD [17].
The current main criterion for judging the performance of the measurement matrix is the maximum absolute value of the coherence coefficients between the columns of the measurement matrix (i.e.| max |) [21].The coherence coefficient () is the cosine of the included angle between two column vectors (i.e., the inner products between any two distinct normalized columns).The | max | values of all 31 matrices are 0.999511.The | max | does not change with the change of the matrix size, if the grid size of super-resolution image and the pixel size of raw image remains unchanged.
The raw image of 37 × 37 pixels is small.The raw image of 7 × 7 pixels is smaller.If the raw image is large, the raw image must be divided into blocks according to block sizes (i.e., 7 -37).We denoised these small blocks using WSD respectively.Thereafter, the denoised blocks are merged block by block.If the size of the raw image is 14 ×14 pixel, and the number of rows of the measurement matrix A is 49, the raw image must be divided into 4 blocks.Then WSD denoises each block separately.The size of each block is 7 × 7 pixel.

Simulation data and division of raw images
To evaluate the denoising performance of WSD for different block sizes of raw images, we generate simulated 500 super-resolution images with known true molecular positions and corresponding 500 raw images.The pixels of the super-resolution image are referred to as grids.Signal to Noise Ratio (SNR) can indicate the overall quality of the denoised raw images and the reconstructed super-resolution image [16].Structural similarity index measure (SSIM) is used for measuring the similarity between two images [22,23].The SSIM values range between 0 and 1.If the two images are identical, the value of SSIM is equal to 1. SSIM can be used to measure the quality of raw image denoising and super-resolution reconstruction.In this paper, SNR and SSIM are used to measure the denoising effect of raw images and the reconstruction effect of super-resolution images.
where x is a real signal, xR N ; |||| 2 is the norm of a vector; and xR denotes the other signal corresponding x.
(2 )(2 ) , , x y xy where  x and  x represent the mean and the variance of image x, respectively. xy represents the covariance of image x and y. c 1 = (k 1 L) 2 and c 2 = (k 2 L) 2 .L is the dynamic range of pixel values.k 1 = 0.01, k 2 = 0.03.

Denoising and reconstruction of simulated lowdensity raw images based on different block sizes
Fig. 1 is the mean value of SNRs of 500 simulated raw images before and after denoising.Each frame contains 34 fluorescence molecules.SNR are inversely proportional to block size after denoising.SNR decreases as the block size gets bigger.The SNR before denoising was 3.113 dB at block size 7. SNR is increased by more than 5 dB after denoising.The maximum value of SNRs is 9.252 dB, and the block size is equal to 10.
The upper row in Fig. 2 is a frame of raw images before and after denoising and the true noiseless raw image.After denoising, SSIM was increased from 0.062 to 0.096, an increase of about half of 0.062.SNR increased from 2.426 dB to 7.939 dB, increasing by 5.513 dB.Fig. 2b is smoother than Fig. 2a and the denoising effect is obvious.WSD is an efficient denoising algorithm.

The y axis is labeled in SNR (dB). The x axis is labeled in block size
The middle row is the super-resolution image reconstructed by CS based on the raw image before and after denoising and the true super-resolution image.After denoising, SSIM increased from 0.011 to 0.758, increasing by about 0.747.SNR increased from -14.891 dB to 0.241 dB, increasing by 15.132 dB.WSD has a significant enhancement effect on superresolution reconstruction.
The lower row is enlarged images from the center areas marked by yellow squares in the middle row.The reconstruction failed in Figs.2d and g.No valid microtubule structures were seen.Compared with Figs.2h and i, although the microtubule structure in Fig. 2h is thicker than that in Fig. 2i, it can well express the real microtubule structure.The cell microtubule structure is clear.

Denoising and reconstruction of simulated highdensity raw images based on different block sizes
Fig. 3 is the mean value of SNRs of 20 simulated raw images before and after denoising.Each frame contains 863 fluorescence molecules.SNR is inversely proportional to block size after denoising.SNR decreases as block size gets bigger.The SNR before denoising was 5.545 dB at block size 7. SNR is increased by about 4.5 dB after denoising.The maximum value of SNRs is 10.238 dB, and the block size is equal to 10, too.
Fig. 4 is a frame of raw images before and after denoising and the true noiseless raw image.After denoising, SSIM was increased from 0.259 to 0.416, increasing by about 0.157.The SNR increased from 5.332 dB to 9.981 dB, increasing by 4.649 dB.Fig. 4b is smoother than Fig. 4a and the denoising effect is obvious.WSD is an efficient denoising algorithm.Comparing the enlarged images from the top and bottom areas marked by yellow squares in the left column, the reconstruction failed in Figs.5a-c.No valid microtubule structures were seen.Although the microtubule structure in Figs.5e and f is thicker, it can well express the real microtubule structure.The cell microtubule structure is clear.

Real experimental data analysis
Fig. 6 shows an experimental raw image with a highdensity molecular distribution before and after denoising, and the corresponding super-resolution reconstruction results.The upper row in Fig. 6 is a frame of 20 raw images before and after denoising.Comparing the images, it is obvious that the denoised raw images are smoother than the raw images.The denoising effect is obvious.Comparing the reconstruction results of the raw images before and after denoising in Figs.6e-h, the spatial resolution is obviously improved and the cell microtubule structure is clearer and thinner.After denoising, CS can converge faster, and the number of iterations is reduced, increasing the computational speed.The CS calculation time of one frame image 21 min and 14 min before and after denoising respectively.The calculation time was reduced by 7 min (i.e., 1/3).
Real denoising experiments of WSD prove that a complex cell microtubule structure can be reconstructed only using 20 frame high-density raw images and CS.The raw images were recorded at a frame rate of 46.436 Hz.The frame rate of 46.436 Hz means that 20 raw images can be acquired per second.20 frame / 46.436 frame × 1s = 0.4307 s.The temporal resolution of the super-resolution image sequence reaches approximately 0.4307 s (20 frames).

Conclusion
In traditional SMLs and super-resolution reconstruction, if the raw pixel size is approximately equal to the PSF's s.d., good localization and reconstruction effects can be achieved.Based on similar scenarios, we studied the denoising ability of WSD for blocks of different sizes.The denoising ability of WSD is related to the block sizes.The larger the block is, the worse the denoising effect is.When the block size is equal to 10, the best denoising effect can be achieved.At this time, the number of rows of the measurement matrix is 100.10 × 10 = 100.The size of the block WSD can denoise is 10 × 10 pixel.SNR can be increased by 5 dB at low density and 4.5 dB at high density.The denoising effect is better at low density.Denoising has great influence on super-resolution reconstruction effect and reconstruction time.Better superresolution reconstruction and shorter reconstruction time can be achieved after denoising.Based on real experimental data and WSD, it can monitor the life process of living cells with a temporal resolution of half a second (0.4307 s at a frame rate of 46.436 Hz).The superresolution reconstruction time reduced by 1/3.The cell microtubule structure is clearer and thinner.The | max | does not change with the change of the matrix size, if the grid size of super-resolution image and the pixel size of raw image remains unchanged.
The effective grid size is 11.429 nm.The simulation ran-Computer Optics, 2023, Vol.47(3) DOI: 10.18287/2412-6179-CO-1172 domly places 34 molecules in a 1024 × 1024 grid region.1024 × 11.429 nm = 11.7 μm. 1 μm = 1000 nm.The area of the grid region is 11.7 μm × 11.7 μm.We generate simulated 20 super-resolution images with known true molecular positions and corresponding 20 raw images again.The simulation randomly places 863 molecules in a 1024 × 1024 grid (11.7 μm × 11.7 μm) region.The corresponding molecular densities are 0.248 and 6.304 m -2 respectively.The super-resolution image's grid is 1/8 of the pixel size of the raw image.The effective pixel size of the raw image is 91.429 nm.The size of the raw image is 128 × 128 pixels.The PSF is a Gaussian function.The s.d. of the PSF is equal to 1.11 times of the raw pixel size.The simulation is for a photon number of 3,000 per molecule and a uniform background of 64 photons per pixel.Poisson noise and Gaussian noise (Gaussian noise variances of 0.01) are added to each frame of the raw image.

Fig. 1 .
Fig. 1.Comparison of the mean SNRs based on 500 simulated raw images before and after denoising.The simulation is for a photon number of 3,000 per molecule and a background of 64 photons per pixel.Each frame contains 34 fluorescent molecules.Poisson noise and Gaussian noise (Gaussian noise variances of 0.01) are added to each frame of the raw image.The y axis is labeled in SNR (dB).The x axis is labeled in block size

Fig. 2 .Fig. 3 .
Fig. 2. Comparison of the raw image and the super-resolution image before and after denoising.The upper row is a frame of the 500 raw images before and after denoising, and the true raw image.The middle row is the corresponding super-resolution image before and after denoising based on the 500 raw images, and the true super-resolution image.The lower row is enlarged images from the center areas marked by yellow squares in the middle row.The yellow squares in the middle row are 3640 × 3640 nm.The number pairs above the images indicate SSIM and SNR (dB) of the images.Scale bars: 1 μm

Fig. 5
Fig.5is the super-resolution image reconstructed by CS based on the raw images before and after denoising.After denoising, SSIM increased from 0.302 to 0.735, increasing by about 0.433.SNR increased from -17.183 dB to -1.459 dB, increasing by 15.724 dB.WSD has a significant enhancement effect on super-resolution reconstruction.Comparing the enlarged images from the top and bottom areas marked by yellow squares in the left column, the reconstruction failed in Figs.5a-c.No valid microtubule structures were seen.Although the microtubule structure in Figs.5e and f is thicker, it can well express the real microtubule structure.The cell microtubule structure is clear.
Computer Optics, 2023, Vol.47(3) DOI: 10.18287/2412-6179-CO-1172 (0.259, 5.332 dB) (0.416, 9.981 dB) a) Before b) After c ) True Fig. 4. Comparison of a frame of the 20 raw images before and after denoising.(a) The raw images before denoising.(b) The raw images after denoising.(c) The true raw image.The number pairs above the images indicate SSIM and SNR (dB) of the images.Scale bars: 1 μm (0.302, -17.183 dB) a) Before b) Enlarged c) Enlarged (0.735, -1.459 dB) d) After e ) Enlarged f) Enlarged Fig. 5. Comparison of the super-resolution image before and after denoising.The upper row is the super-resolution image before denoising based on the 20 raw images.The lower row is the super-resolution image after denoising based on the 20 raw images.The yellow squares in the left column are 3640 × 3640 nm.The middle and right columns are enlarged images from the center areas marked by yellow squares in the left column.The number pairs above the images indicate SSIM and SNR (dB) of the images.Scale bars: 1 μm f) After g) Before h) After Fig. 6.Comparison of the experimental raw images and the super-resolution images before and after denoising.The upper row is 2 frames of raw images from 2 different raw image sets before and after denoising.The lower row is corresponding super-resolution images before and after denoising respectively.Scale bars: 1 μm