IMAGE PROCESSING Construction of holographic images using cross sample modified laplacian method

AbstractThis
paper presents a

measurement, selection of

the the size,

shape-from-focus method,which improved regard to is with the

neighborhood surface through interpolation, surface

refinement

and

mathematical operator contrast used for

postprocessing. Three-dimensional

models human

of faces

living are

from-focus, reconstruction.

3-D

presented with such a high resolution hairs I. INTRODUCTION

that

single

are visible.

Key

words
THE acquisition of 3-D information out of 2-D images is one of the most

Autofocusing, depth estimation, depth-

from focus, facial measurement, focus analysis, focus focusing, measure, shape shape-

important issues in computer Techniques executing this task vision.

holography, estimation,

are often referred to as shape-from-x, where x is a visual cue such as shading, texture, focus, contour, and

is

taken

while

changing the focus setting in small

steps. One image of such a sequence,

stereo,

further on referred to as focus series, contains a focused region of the imaged

motion [1]. Blurring phenomena defocusing are among the due to

object as well as an unfocused background. determining By the

important cues for depth recovery.

Shape-from focus is a method where a sequence of images

focus setting, that locally optimizes

image

focus,

3-D about

the recorded object is stored by the

information

the imaged object is retrieved [2]. If the object is moving and cannot be expected to remain in a

superimposing

back-scattered light coherently reference Through amplitude this, and with a

wave. the the

certain

position

during the capturing time, an alternative method for 3-D is where the 3-D about

phase information of the scattered light field are stored as an interference

measurement holography, part of

pattern, for example in an analogue

information

photosensitive

material. This kind of acquisition of 3-D data has the

way to get access to the information reconstructing hologram. stored is the

tremendous advantage of storing the high-resolved in an

information extremely amount namely duration of in of

short time, the the Fig. 1. Optical reconstruction of the hologram and digitalization of a set of

recording laser pulse (35 ns for the setup described However, the here). only

2-D projections at different positions. axial

reference

beam,

which produces the so-called real image, a 3-D light field with one-to-one

This optically,

is

done if was an

correspondence

to

therecorded object. It has to be

thehologram stored analogue material. realized illuminating hologram complex with on

digitized into a set of 2-D projections, because of the 2-D nature recording devices.The result of digital

recording It is by the the

conjugate

of the holographic

reconstruction is a set of images focused unfocused similar to

reconstructions [3] [7].

containing and regions the gained subsequent recording. why the

This paper discusses improvements shape-from-focus methods, applicable of

focus

series through

This

is

to all kind of focus series. application, provides exemplary The that the material

same designed

methods for

shape-from-

focus in microscopy are applicable for

for the discussion, is the analogue

holographic

holographic recording human that of living For of

completely image, from

focused extracted

faces. kind

the

focus

series in combination with the 3-D serves

application, the full advantage extremely recording of the short time

information,

as precisely fitting texture. Such highly resolved textured 3D models of living human faces are of. great many interest kinds in of among

arises since motion artifacts caused by changes of the face due to mimic, or can be The

breathing, heartbeat avoided.

applications,

others in medicine

and forensicscience [8][14].

3-D computer model of the face. . SHAPE-FROMFOCUS The focus considered series may

consist of gray-scale Fig. 1(a) shows exemplarily one image from the focus series and (b) screenshot of the resulting, textured images, characterized their each by

intensity

distribution in the plane with . the The

coordinate distinction

between

focused unfocused parts with focus is a

and image realized so-called a

referred to as focus values. A

maximization of the focus values along the direction

criterion,

mathematical operator which

localizes the focused regions and produces information about

measures the local image contrast of

the 3-D geometry of the (see imaged object Fig. 2). This

each pixel in each image with the axial position with the aid of a local pixel

process is further on called extraction. addition to surface In the

neighborhood. These values are further on

topometrical the procedure

data, can

models. The pixelto-pixel correspondence between texture and geometry, which is inherent described to the

be used to generate an image with

unlimited depth of field by compounding the detected sharp patches from each image into one image described by . This so-called texture

process,

makes any further registration between the model and the texture expendable

image can be used for texturing the 3-D

[see Fig. 1(b)]. Pioneering microscopic work in

resulting computer

shape-

from-focus was done

by Nayar [2]. It is used for a wide

such

as

micro-

cogwheel[17]. The optimization of the focus depth-formprocedure

range of applications like measurement the of

firearm bullets and cartridge cases and for for purposes material analysis mineralogical research [16] or for micro structures machine tools

depends strongly on thenatural frequency content texture object recorded. objects intrinsic active with of of to the the be For low

inspection [15], for

structure in

contrast, illumination

can be used [18]. In

the case of human faces, the intrinsic contrast of the skin is sufficient for

used

in

many

different applications. The

performance of the different auto focus algorithms strongly properties depends on of the the

shape-from focus. OPTIMIZED FOCUS CRITERIA The quintessence of the depth-from-

object to be focused on, as well as those of the imaging of was

focus procedure is thedetection of

focused image parts, the same task that is performed by auto focus algorithms

systems.Alot research performed

for

finding

the

a whole image or a specified part. In

suitable for the

algorithm different [19]

shape-from-focus, a sharpness assigned value to is

applications

[26]. The core of an autofocusing algorithm mathematical operator, a focus is a

every

pixel. It is calculated using a local pixel neighborhood. To

find a suitable focus criterion holographic measurement, quantitative comparison extracted of the facial for facial a

criterion, working on the pixel

representation of the digital image and assigning a

quantitative value to

surface

was

Laplacian

(XSML).

performed with the help of a simulated focus series with

The latter is a newly developed modification Nayars of SML

different

amounts

of added noise. The evaluated criteria were among others

operator, presented for the first time in this paper. The

Tenengrad [31], [19], Squared Absolute Sum Gradient, Gradient, Modified

Laplacian in the SML takes only direct

neighbors in `the xand y-directions into account. Additionally, also the diagonal neighbours

Laplacian (SML) [2], Variance, and Cross Sum Modified

could

be

included, with of a to for A discussion criteria detailed on and the their would

weighted factor

compensate

their larger distance to the central point where is a square pixel of neighborhood the parameter

performance

go beyond the scope of the paper. An indepth study can be found in [7]. It that

can be adjusted to the size characteristic of texture in the

demonstrates

elements

the newly invented XSML criteria

recorded scene.

produces the lowest

deviation

between

ADAPTIVE SELECTION OF

the extracted and a reference surface

NEIGHBORHOOD SIZE The choice of the neighborhood size is a critical step in the procedure surface The of

for any amount of added XSML shows higher against the a noise. The

operator slightly

robustness noise than

extraction. lateral

SML-criterion

due to the increased number used of for points the

resolution of the extracted surface is limited by the size of the neighborhood, since structures

calculation.

smaller size

than cannot

this be

which leads to 3-D geometry information the object. On the other hand, small neighborhoods are very sensitive to noise. The lower the signal-to-noise ratio, the larger the about

resolved. Thus, the neighborhood size

should be chosen as small as possible.

Fig.

2.

For

each

neighborhood should be. Even in the

fixed lateral point (x,y), values the focus

absence of noise, the minimal neighborhood size is

F(x,y,z) are

maximized,

limited to a certain extent by the

size,one always has to between resolution robustness compromise lateral and against

feature size of the texture recorded of the object.

With a neighborhood smaller feature impossible than it any is to

noise. . In regions of high contrast, mustache proband, intrinsic like of a the the high-

distinguish between focused defocused regions. choosing neighborhood or image When a

quality surface can be extracted using a neighborhood size of 3

distinguished. regions intrinsic Fig. 3. Exemplary maps from a of

In lower

contrast,

however, the surface extracted with a height extracted

neighborhood size of 3 is affected Using by a

holographic data set with a neighborhood size of 3 (a) and 17 (b)

noise.

neighborhood size of 17 pixels hand on the allows

other

for a noise resistant pixelspixels with a lateral resolution so high that even single hairs can be surface extraction in regions of lower

intrinsic while the

contrast, lateral is To this an algorithm developed

confidence value for each found surface point and several

resolution reduced. overcome tradeoff, adaptive was

neighborhood sizes . The surface

extraction is done by comparing sharpness values calculated

choosing locally the smallest possible

with a neighborhood size d along the and

neighborhood size so that the surface is not corrupted by

direction Z finding the

noise. It is based on the definition of a

Fig. 4. Histograms of confidence values

maximum. The higher the maximum to in the

for a typical facial data set calculated with a neighborhood size of 3 (a), 5 (b), and 11 pixels (c). An increasing neighborhood leads to of size the a

comparison mean value, reliable generated

sharpness the is more the surface

point.is defined as follows:

appearance second

(1) with

peak

belonging to reliable surface points. (2)

The

normalization

neighborhood and extracted

size accept surface

leads to confidence values between zero and one, a higher value more indicates a

points only if they are reliable. If not, a larger neighborhood is considered. can with of be the the

reliable

surface point. The confidence values of one surface point

Reliability defined help

differ for different neighborhood sizes.

confidence values by selecting a threshold . The thresholds are determined by

The concept of the adaptive algorithm is to start with the smallest

regarding histogram of the

the

For

small size, reliable

neighborhood confidence very few

values created with different neighborhood sizes

surface points are present. histogram mainly a The shows normal of

for a typical data set of a recorded face.

distribution

confidence values of unreliable points

Fig. 4 shows such histograms for a

with

slight caused reliable Increasing of the

asymmetry by the

neighborhood size of 3 pixels (a), 5 pixels (b), and 11 pixels (c).

points. the size

neighborhood,

the

depending thresholds were

number of reliable points increases,so

determined for 25 different data sets manually.4

that a second peak appears center higher values. points excluded with shifting its to

confidence Unreliable can if be the

Fig. 5 shows the mean thresholds, as well as an

exponential fit. The resulting curve is

threshold is chosen such that the first peak is eliminated. In this manner, the neighborhood size

further on used for automated threshold selection distinction and the between

reliable unreliable points manually the

and surface without analyzing

the

surface

extracted with the maximal

neighborhood can be smoothed by

histograms.Limiting the maximal

weighted averaging. The confidence

neighborhood size to a certain value, there are still points where no reliable

values act thereby as weights. These

values are used to fill the gaps if

surface value could be found. In order to fill these gaps,

desired. The result can be seen in

Fig. 6. It shows a surface not

The thresholds were averaged values, over the 25

corrupted by noise with the maximal lateral 4An

error

bars represent the standard deviation.

achievable resolution.

automated selection is also possible, but is not necessary, as is explained in the following. . neighborhood Fig. 5. Threshold against size Fig. 6. Height map created with the presented algorithm corresponding to the height maps created with a fixed

with exponential fit.

neighborhood size shown in Fig. 3.

section is shown in (b). Another example can be seen in Fig. 7. It shows a section of

Fig. 7. Section of the texture extracted from a holographically recorded face digitized with a resolution of 40 _m (a). A screenshot of the 3-D computer model of the marked

the texture image extracted form a holographic recording, digitized with a resolution of 40 m. The adaptive approach presented in this paper permitted the creation of a closed

surface with a local resolution so high that single hairs are visible in the 3-D model. SURFACE INTERPOLATION The set of 2-D Fig. focus fitted 8. Exemplary with

profile

Gaussian

curve. The dashed line represents the

images, from which the information extracted, represents holographical the image surface is

threshold for points to be considered. tions. Thus, focus

information is also available discrete only at

positions

at discrete position.

and with it surface

information.

Hence,

proposed

an

the choice of the interslice limits the achievable resolution. distance maximal axial An

interpolation method based on a Gaussian distribution (3) By choosing the

maximal focus value and its two the

interpolation of the discrete focus values would continuous information consequently potentially and a higher provide

neighbors,

unknown parameters and can be

calculated just by solving a system of algebraic With equations. this,

resolution. Nayar [2]

continuous values are

produced not anymore

that

are

the height of the maximum above the average focus values and consider only

restricted to

thediscretesliceposi tions.An alternative approach is to fit the Gaussian model mentioned in (3) to a particular number of chosen points.

points in the upper half of that interval (dashedline in Fig. 8). Additionally, the

points have to lie in an interval of 20

For the selection of points used for the fit it was found best to first determine

tomograms and behind

before the to

maximal

value,

eliminate noisy data points coincidentally

lying

above but

the far the The

the Gaussian serves as coordinate.The interpolation method is based upon three focus values exactly met by the curve, while the fit method takes up to twenty points into account and finds thecurve by minimizing the new

threshold away

from

maximum.

fitting algorithm was implemented using a downhill method simplex in

multidimensions[27, p. 408] proposed by Nelder and Mead

[28]. Fig. 8 shows the with whole the profile fitted

overalldeviation. That makes the fit more accurate and

curve. The center of

more stable in the presence However, of noise. its

to be a useful tool for refinement. quantitative comparison interpolation methods above mentioned and an of the surface A

computational effort is much higher since an iterative

optimization step is involved.In holographic facial

additional one can be found [7].Additionally the robustness noise, the in to

measurement, where computational time is not as important as in real time the

increased against

applications,

Gaussian fit proved

fit

also

eliminates

considerably through the Gaussian fit, as can be seen in Fig. 9. The height profile gained with the fit is shifted for better comparability. SURFACE POSTPROCESSING The sections new invented previous presented methods for the

step like structures in the extracted

surface,which especially appear if using neighborhood large sizes.

An example can be seen in the height profile in Fig. 9. The steps in the height map are much larger than the interslice distance.These steps are lessened

improvement of the classical depth-

from-focus method.However, despite sophisticated surface extraction

details. that a

We

found

smoothing

method proposed by Tasdizen [29] based on anisotropic

methods,little noisy surface cannot fluctuation be avoided

diffusion of normals worked perfectly for the of postprocessing facial

entirely.Often denoising with filtering, reduces noise, but also blurs sharp features and is done

models.Anisotropic diffusion of normals is the natural of the

low-pass which

expansion

anisotropic diffusion method for 2-D

images introduced by Perona [30] and to Malik curved Fig. 10 shows a facial model the adaptive gained with

surfaces in the 3-D space. Smoothing in regions of high total is through

presented algorithm

curvature suppressed

and Gaussian fitting without postprocessing with smoothing with diffusion normals.5 any (a),

the introduction of a nonconstant diffusion coefficient depending on the

isotropic (b) and

anisotropic of While

total curvature.

small features are

almost obliterated in (b) they are nearly as pronounced in (c) as in the raw

they originate from classic focus, anisotropic smoothing does not behave all stable.With improvements in this shape-fromthe

data.The generation of surface continuous data

through the Gaussian fit or Gaussian is the the

presented

paper applied, the computation time on a standard PC for facial models as

interpolation obligatory appliance for of

anisotropic diffusion of normals. For

shown in this paper increases from 1 min for the standard

discrete surfaces, as

procedure approximately min, whereas

to 30 the

texture information from holographic

reconstructions, especially for the creation of

Gaussian requires the most of

fitting

the

highly resolved facial models.We evaluated several mathematical

additional time Conclusions This demonstrated paper that

operators contrast measurement

for

the classical shapefrom-focus method

and

found that the best criterion application newly for is this the

is perfectly suited for the extraction of 3-D geometry and

developed

XSML

criterion,

modification of SML operator by proposed [2]. we an for the The size small neighborhood is chosen as as

Nayar

Additionally, presented algorithm automated

possible

adaption

while preventing corruption through

of the neighborhood size to of the the

noise. With this, an optimal resolution achieved. lateral can It be was that facial

conditions data.

demonstrated in

measurement hairs visualized this can

single be

continuous

surface

not restricted to the digitization positions is created. This is compulsory for the appliance anisotropic smoothing methods, which were in also this of

through It

procedure.

was also shown that a Gaussian fit to the focus profile

improves the quality of the surface

tremendously. Artifacts are eliminated and the influence of noise is reduced, the same while time at a

presented paper. The method

holographic for facial is to

measurement superior

conventional measurement techniques since it captures 3-D

created for various applications.

information of the face with one single laser pulse of 35-ns duration.In extremely that short Fig. 11. Schematic top view, side view, and photograph of the holographic camera for portrait holography.

amount of time, no movement artifacts

can occur and highly resolved models can facial be

REFERENCES

1.IEEE journals and magazines&website 2.google engine search