Light field microscopy

Light field microscopy (LFM) is a scanning-free 3-dimensional (3D) microscopic imaging method based on the theory of light field. This technique allows sub-second (~10 Hz) large volumetric imaging ([~0.1 to 1 mm]3) with ~1 μm spatial resolution in the condition of weak scattering and semi-transparence, which has never been achieved by other methods. Just as in traditional light field rendering, there are two steps for LFM imaging: light field capture and processing. In most setups, a microlens array is used to capture the light field. As for processing, it can be based on two kinds of representations of light propagation: the ray optics picture and the wave optics picture. The Stanford University Computer Graphics Laboratory published their first prototype LFM in 2006 and has been working on the cutting edge since then.

Light field generation

A light field is a collection of all the rays flowing through some free space, where each ray can be parameterized with four variables. In many cases, two 2D coordinates–denoted as ( s , t ) {\displaystyle (s,t)} & ( u , v ) {\displaystyle (u,v)} –on two parallel planes with which the rays intersect are applied for parameterization. Accordingly, the intensity of the 4D light field can be described as a scalar function: L f ( s , t , u , v ) {\textstyle L_{f}(s,t,u,v)} , where f {\displaystyle f} is the distance between two planes. LFM can be built upon the traditional setup of a wide-field fluorescence microscope and a standard CCD camera or sCMOS. A light field is generated by placing a microlens array at the intermediate image plane of the objective (or the rear focal plane of an optional relay lens) and is further captured by placing the camera sensor at the rear focal plane of the microlenses. As a result, the coordinates of the microlenses ( s , t ) {\displaystyle (s,t)} conjugate with those on the object plane (if additional relay lenses are added, then on the front focal plane of the objective) ( s ′ , t ′ ) {\displaystyle (s',t')} ; the coordinates of the pixels behind each microlens ( u , v ) {\displaystyle (u,v)} conjugate with those on the objective plane ( u ′ , v ′ ) {\displaystyle (u',v')} . For uniformity and convenience, we shall call the plane ( s ′ , t ′ ) {\displaystyle (s',t')} the original focus plane in this article. Correspondingly, f {\displaystyle f} is the focal length of the microlenses (i.e., the distance between microlens array plane and the sensor plane). In addition, the apertures and the focal-length of each lens and the dimensions of the sensor and microlens array should all be properly chosen to ensure that there is neither overlap nor empty areas between adjacent subimages behind the corresponding microlenses.

Realization from the ray optics picture This section mainly introduces the work of Levoy et al., 2006.

Perspective views from varied angles Owing to the conjugated relationships as mentioned above, any certain pixel ( u j , v j ) {\displaystyle (u_{j},v_{j})} behind a certain microlens ( s i , t i ) {\displaystyle (s_{i},t_{i})} corresponds to the ray passing through the point ( s i ′ , t i ′ ) {\displaystyle (s_{i}',t_{i}')} towards the direction ( u j ′ , v j ′ ) {\displaystyle (u_{j}',v_{j}')} . Therefore, by extracting the pixel ( u j , v j ) {\displaystyle (u_{j},v_{j})} from all subimages and stitching them together, a perspective view from the certain angle is obtained: L f ( : , : , u j , v j ) {\textstyle L_{f}(:,:,u_{j},v_{j})} . In this scenario, spatial resolution is determined by the number of microlenses; angular resolution is determined by the number of pixels behind each microlens.

Tomographic views based on synthetic refocusing

Step 1: Digital refocusing

Synthetic focusing uses the captured light field to compute the photograph focusing at any arbitrary section. By simply summing all the pixels in each subimage behind the microlens (equivalent to collecting all radiation coming from different angles that falls on the same position), the image is focused exactly on the plane that conjugates with the microlens array plane:

… excerpt ends here. Continue reading the full article.