INTERACTIVE NATURAL IMAGE SEGMENTATION

VIA SPLINE REGRESSION

Abstract
We propose an interactive algorithm for segmentation of natural images in this project.
The task is formulated as a problem of spline regression, in which the spline is derived in
Sobolev space and has a form of a combination of linear and Greens functions. Besides its
nonlinear representation capabilit, one advantage of this spline in usage is that, once it has been
constructed, no parameters need to be tuned to data. We define this spline on the user specified
foreground and background pi!els, and solve its parameters "the combination coefficients of
functions# from a group of linear e$uations. To speed up spline construction, %&means clustering
algorithm is emploed to cluster the user specified pi!els. B taking the cluster centers as
representatives, this spline can be easil constructed. The foreground object is finall cut out
from its background via spline interpolation. The computational comple!it of the proposed
algorithm is linear in the number of the pi!els to be segmented. The proposed algorithm is
implemented and tested using '(T)(B simulation
BLOCK DIAGRAM
EXISTING SYSTEM
*olor +mage Segmentation, (dvances (nd -rospects, +mage segmentation is ver
essential and critical to image processing and pattern recognition. This surve provides a
summar of color image segmentation techni$ues available now.
DISADVANTAGES,
. Basicall, color segmentation approaches are based on monochrome segmentation
approaches operating in different color spaces.
PROPOSED SYSTEM
We proposed a novel algorithm for natural image segmentation. We formulated the task
as a problem of spline regression. The spline is a combination of linear and Greens functions,
with adaptabilit to diverse natural images. We also anal/ed the connections of our spline
regression algorithm to other algorithms, including those developed in inductive learning setting,
transductive learning setting, regulari/ation on graph and general functional spaces. *omparative
e!periments illustrate the validit of our method.
ADVANTAGES
. (dvantage of this spline in usage is that, once it has been constructed, no parameters
need to be tuned to data.
. To speed up spline construction, %&means clustering algorithm is emploed to cluster the
user specified pi!els. B taking the cluster centers as representatives, this spline can be
easil constructed.
DOMAIN: DIGITAL IMAGE PROCESSING
0igital +mage processing is the use of computer algorithms to perform image processing
on 0igital +mages. (s a subfield of digital signal processing, digital image processing has man
advantages over analog image processing1 it allows a much wider range of algorithms to be
applied to the input data, and can avoid problems such as the build&up of noise and signal
distortion during processing.
SOFTWARE REQUIREMENT
'(T)(B 2.3 and above
'(T)(B is a high&performance language for technical computing. +t integrates
computation, visuali/ation, and programming in an eas&to&use environment where problems and
solutions are e!pressed in familiar mathematical notation. Tpical uses include,
'ath and computation
. (lgorithm development
. 'odeling, simulation, and prototping
. 0ata analsis, e!ploration, and visuali/ation
. Scientific and engineering graphics
. (pplication development, including Graphical 4ser +nterface building
'(T)(B is an interactive sstem whose basic data element is an arra that does not re$uire
dimensioning. This allows ou to solve man technical computing problems, especiall those
with matri! and vector formulations, in a fraction of the time it would take to write a program in
a scalar non&interactive language such as * or 567T7(8.
INTRODUCTION:
9!tracting the foreground objects in natural images is one of the most fundamental tasks
in image processing and understanding. Generall, this task can be formulated as a problem of
image segmentation. The efforts in segmentation have surged in recent decades, with the
development of numerous approaches and proposals for real world applications. +n spite of man
thoughtful attempts, it is still ver difficult to develop a general framework which can ield
satisfactor segmentations for diverse natural images. The difficulties lie in the comple!it of
perceiving and modeling the numerous visual patterns in natural images and the intrinsic
ambiguit of grouping them to be the needed objects.
To reduce the comple!it and intrinsic ambiguit, one method is to design interactive
frameworks, which can allow the user to specif the foreground and background according to
her:his own understanding about the image. +n such a work setting, the user can also act as a
judge to accept or refuse the current segmentation results, or add more strokes to obtain better
segmentation. +n view of image perception, the user specified strokes give us the visual hints to
model and group the visual patterns. With such supervised information, man e!isting algorithms
developed in machine learning can be emploed to formulate the task of image segmentation.
The goal of interactive image segmentation is to cut out a foreground object from its
background with modest user interaction . There are two main methods. 9dge based and region
based. 9dge&based methods need the user to label the points near the object boundar. 7ecentl,
researches mainl focus on region&based methods. +n this method, the interaction stle is largel
improved. The user can label the regions of foreground object and its background b simpl
dragging the mouse. The main advantage is that it does not re$uire the user to stare at and stroke
along the object boundar. With the help of statistical inference or machine learning algorithms,
the developed region&based interaction frameworks have achieved great success.
SCOPE OF TE PRO!ECT:
To develop a novel algorithm for natural image segmentation. The task is formulated as a
problem of spline regression. The line is a combination of linear and Greens functions, with an
adaptabilit to diverse natural images. (nd to anal/e the connections of our spline regression
algorithm to other algorithms, including those developed in inductive learning setting,
transductive learning setting, regulari/ation on graph and general functional spaces.
MODULE SEPERATION:
'604)9 ;, -roblem formulation and Spline 7egrssion
'604)9 <, 5eature =ectors of -i!els
'604)9 >, *lustering the 4ser Specified -i!els
'604)9 ?, Spatial 8eighbourhood (ssignment
'604)9 @, -ost processing steps
'604)9 ;, Spline 7egrssion A 5eature =ectors of -i!els
'604)9 <, *lustering the 4ser Specified -i!els A Spatial 8eighbourhood (ssignment
'604)9 >, 8atural +mage Segmentation
MODULE DESCRIPTION:
MODULE ":
+t is a task of data classification. 'an e!isting classification methods, such as )00 B %&
8 as classifier , semi&supervised classification , graph cut , random walks can be applied to this
task. Cere spline regression is followed. 5irst a spline function is reconstructed. This task can be
considered in a general regulari/ation framework containing data fitting and function
smoothness.
Cere, each pi!el is described as a @&0 feature vector, i.e ,Xi =[R G B X YD in which "7 G
B# is the normali/ed color for each pi!el and "!,# is the spatial coordinate normali/ed with
image width and height. The reason that we consider the spatial coordinates is that the
discrimination between pi!els with similar colors can be enhanced, especiall when the
foreground object and its background contain e!actl identical colors.
MODULE #:
Generall, there are thousands of pi!els which ma be scribbled b the user. (ccordingl,
linear e$uations are to be solved with a large number of unknown parameters. Since the
coefficient matri! is a dense matri!, the computation comple!it of solving the linear e$uations
have to be solved here. Cowever, in most cases, the foreground object and its background onl
consists of a few number of different colors. Thus the user specified foreground and background
pi!els can be clustered and emplo the cluster centers as their representatives.
+n our spline regression framework, once the spline is constructed, it can be used to map
the unlabeled pi!els one b one. Thus, the spatial relationship between pi!els on the image grid
will be simpl ignored. To utili/e the spatial structure of the image, we assign the regressed value
of each pi!el to its neighbors.
MODULE $:
+n this section, three post processing methods will be introduced for user to obtain better
segmentation. The are connectivit analsis, edge fairing, and segmenting with more strokes.
5inall the Segmented output is given for the input natural images.
ALGORITM USED:
Spline 7egression
FUTURE ENANCEMENT:
Some e!isting incorrect small blobs can be corrected or deleted via connectivit analsis
and area statistics.
ADVANTAGES:
. (dvantage of this spline in usage is that, once it has been constructed, no parameters
need to be tuned to data.
. +t has an adaptabilit to diverse natural images.
. 9ach part of this segmentation achieves the highest accurac.
APPLICATIONS:
. 7emote sensing
. 'edical image processing
. 'orphological image processing.
LITERATURE SURVEY:
E;D 6bject Based +mage 9diting
William (. Barrett barrettFcs.bu.edu
(lan S. *hene cheneaFcs.bu.edu
0epartment of *omputer Science, Brigham Goung 4niversit
This paper introduces 6bject Based +mage 9diting "6B+9# for real&time animation and
manipulation of static digital photographs. +ndividual image objects are selected, scaled,
stretched, bent, warped or even deleted "with automatic hole filling# H at the object, rather than
the pixel level & using simple gesture motions with a mouse. 6B+9 gives the user direct, local
control over object shape, si/e, and placement while dramaticall reducing the time re$uired to
perform image editing tasks. 6bject selection is performed b manuall collecting "subobject#
regions detected b a watershed algorithm. 6bjects are tessellated into a triangular mesh,
allowing shape modification to be performed in real time using 6penG)s te!ture mapping
hardware. Through the use of anchor points, the user is able to interactivel perform editing
operations on a whole object, or just part"s# of an object & including moving, scaling, rotating,
stretching, bending, and deleting. Indirect manipulation of object shape is also provided through
the use of sliders and Be/ier curves. Coles created b movement are filled in real&time based on
surrounding te!ture. When objects stretch or scale, we provide a method for preserving texture
granularity or scale. +t present a texture brush, which allows the user to IpaintJ te!ture into
different parts of an image, using e!isting image te!ture"s#. 6B+9 allows the user to perform
interactive, high& level editing of image objects in a few seconds to a few tens of seconds.
%#& I't(ract)*( I+a,( S(,+('tat)-' .s)', a' a/a0t)*(
GMMRF +-/(1
(. Blake, *. 7other, '. Brown, -. -ere/, and -. Torr
'icrosoft 7esearch *ambridge 4%,
2 KK Thomson (venue, *ambridge *B> 35B, 4%.
http,::www.research.microsoft.com:vision:cambridge
The problem of interactive foreground:background segmentation in still images is of
great practical importance in image editing. The state of the art in interactive segmentation is
probabl represented b the graph cut algorithm of Bokov and Koll "+**= <33;#. +ts
underling model uses both colour and contrast information, together with a strong prior for
region coherence. 9stimation is performed b solving a graph cut problem for which ver
efLcient algorithms have recentl been developed. Cowever the model depends on parameters
which must be set b hand and the aim of this work is for those constants to be learned from
image data. 5irst, a generative, probabilistic formulation of the model is set out in terms of a
Gaussian 'i!ture 'arkov 7andom 5ield. "G''75#. Secondl, a pseudolikelihood algorithm is
derived which jointl learns the colour mi!ture and coherence parameters for foreground and
background respectivel. 9rror rates for G''75 segmentation are calculated throughout using a
new image database, available on the web, with ground truth provided b a human segmenter.
The graph cut algorithm, using the learned parameters, generates good object&segmentations with
little interaction. Cowever, pseudolikelihood learning proves to be frail, which limits the
comple!it of usable models, and hence also the achievable error rate.
%2& I't(ract)*( Gra03 C.ts 4-r O0t)+a1 B-.'/ar5 6 R(,)-' S(,+('tat)-' -4
Ob7(cts )' N8D I+a,(s
Guri G. Bokov 'arie&-ierre Koll
Siemens *orporate 7esearch
uriFcsd.uwo.ca
+n this paper it describes a new techni$ue for general purpose interactive segmentation of
8&dimensional images. The user marks certain pi!els as IobjectJ or IbackgroundJ to provide
hard constraints for segmentation. (dditional soft constraints incorporate both boundar and
region information. Graph cuts are used to find the globall optimal segmentation of the 8&
dimensional image. The obtained solution gives the best balance of boundar and region
properties among all segmentations satisfing the constraints. The topolog of our segmentation
is unrestricted and both IobjectJ and IbackgroundJ segments ma consist of several isolated
parts. Some e!perimental results are presented in the conte!t of photo:video editing and medical
image segmentation. We also demonstrate an interesting Gestalt e!ample. ( fast implementation
of segmentation method is possible via a new ma!&flow algorithm.
REFERNCES:
E;D W. (. Barrett and (. S. *hene, I6bject&based image editing,J in roc! "IGGR#$, San
(ntonio, TM, <33<, pp. 222H2N?.
E<D (. Blake, *. 7other, '. Brown, -. -ere/, and -. Torr, I+nteractive image segmentation using
an adaptive gmmrf model,J in roc! %uropean &onf! &o'puter (ision, -rague, */ech 7epublic,
<33?, pp. ?<NH??;.
E>D 5. Bookstein, I-rincipal warps, Thin&plate splines and the decomposition of deformations,J
I%%% )rans! attern #nal! *ach! Intell!, vol. ;;, no. O, pp. @O2H@N@, Kun. ;PNP.
E?D G. Bokov and '. Koll, I+nteractive graph cuts for optimal boundar A region segmentation
of objects in n&d images,J in roc! Int! &onf! &o'puter (ision, =ancouver, B*, *anada, <33;,
pp. ;3@H;;<.
E@D C. *heng, M. Kiang, G. Sun, and K. Wang, I*olor image segmentation, (dvances and
prospects,J attern Recognit!, vol. >?, no. ;<, pp. <<@PH<<N;, <33;.
EOD G. G. *huang, B. *urless, and 0. S. abd 7ichard S/eliski, I( Baesian aproach to digital
matting,J in roc! Int! &onf! &o'puter (ision and attern Recognition, <33;, vol. <, pp. <O?H
<2;.
E2D K. 0uchon, , (. 0old and B. 9ckmann, 9ds., ISplines minimi/ing rotation& invariant semi&
norms in sobolev spaces,J in &onstructive )heory of +unctions of "everal (ariables. 8ew Gork,
Springer&=erlag, ;P22, pp. N@H;33.
END 7. 0uda, -. Cart, and 0. Stork, attern &lassification, <nd ed. 8ew Gork, Wile, <33;.
EPD '. Gleicher, I+mage snapping,J in roc! "IGGR#$, )os (ngeles, *(, ;PP@, pp. ;N>H;P3.
E;3D ). Grad, I7andom walks for image segmentation,J I%%% )rans! attern #nal! *ach!
Intell!, vol. <N, no. ;;, pp. ;2ONH;2N>, 8ov. <33O.
E;;D +. T. Kolliffe, rincipal &o'ponent #nalysis, <nd ed. 8ew Gork, Springer, <33<.