FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

FaceDetectionusingOpenCV
Howtocompileandrunthefacedetect.cisoneofthefrequentlyasked
questionintheOpenCVYahoo!Groups.I'lltrytoexplainittomybest.Feel
freetoedititifyouhavesomemoredetails..
HaarLikeFeatures:Whatisthat?
Arecognitionprocesscanbemuchmoreefficientifitisbasedonthedetectionof
featuresthatencodesomeinformationabouttheclasstobedetected.Thisisthecase
ofHaar-likefeaturesthatencodetheexistenceoforientedcontrastsbetweenregions
intheimage.Asetofthesefeaturescanbeusedtoencodethecontrastsexhibitedby
ahumanfaceandtheirspacialrelationships.Haar-likefeaturesaresocalledbecause
theyarecomputedsimilartothecoefficientsinHaarwavelettransforms.
TheobjectdetectorofOpenCVhasbeeninitiallyproposedbyPaulViolaand
improvedbyRainerLienhart.First,aclassifier(namelyacascadeofboosted
classifiersworkingwithhaar-likefeatures)istrainedwithafewhundredsofsample
viewsofaparticularobject(i.e.,afaceoracar),calledpositiveexamples,thatare
scaledtothesamesize(say,20x20),andnegativeexamples-arbitraryimagesofthe
samesize.
Afteraclassifieristrained,itcanbeappliedtoaregionofinterest(ofthesamesize
asusedduringthetraining)inaninputimage.Theclassifieroutputsa"1"ifthe
regionislikelytoshowtheobject(i.e.,face/car),and"0"otherwise.Tosearchfor
theobjectinthewholeimageonecanmovethesearchwindowacrosstheimageand
checkeverylocationusingtheclassifier.Theclassifierisdesignedsothatitcanbe
easily"resized"inordertobeabletofindtheobjectsofinterestatdifferentsizes,
whichismoreefficientthanresizingtheimageitself.So,tofindanobjectofan
unknownsizeintheimagethescanprocedureshouldbedoneseveraltimesat
differentscales.
Theword"cascade"intheclassifiernamemeansthattheresultantclassifierconsists
ofseveralsimplerclassifiers(stages)thatareappliedsubsequentlytoaregionof
interestuntilatsomestagethecandidateisrejectedorallthestagesarepassed.The
word"boosted"meansthattheclassifiersateverystageofthecascadearecomplex
themselvesandtheyarebuiltoutofbasicclassifiersusingoneoffourdifferent
boostingtechniques(weightedvoting).CurrentlyDiscreteAdaboost,RealAdaboost,
GentleAdaboostandLogitboostaresupported.Thebasicclassifiersaredecision-tree
classifierswithatleast2leaves.Haar-likefeaturesaretheinputtothebasic

1of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

classifers.Thefeatureusedinaparticularclassifierisspecifiedbyitsshape,position
withintheregionofinterestandthescale(thisscaleisnotthesameasthescaleused
atthedetectionstage,thoughthesetwoscalesaremultiplied).
Ok.Enoughofthetheorypart.Nowtothecodingpart:(Thiscodemighthavea
memoryleak.Scrolldowntothebottomofthepageforinformationonhowtofix
it.)
Hereismycommentedfacedetect.cfile:
//OpenCVSampleApplication:facedetect.c
//Includeheaderfiles
#include"cv.h"
#include"highgui.h"
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<assert.h>
#include<math.h>
#include<float.h>
#include<limits.h>
#include<time.h>
#include<ctype.h>
//Creatememoryforcalculations
staticCvMemStorage*storage=0;
//CreateanewHaarclassifier
staticCvHaarClassifierCascade*cascade=0;
//Functionprototypefordetectinganddrawinganobjectfrom
animage
voiddetect_and_draw(IplImage*image);
//Createastringthatcontainsthecascadename
constchar*cascade_name=
"haarcascade_frontalface_alt.xml";
/*"haarcascade_profileface.xml";*/
//Mainfunction,definestheentrypointfortheprogram.
intmain(intargc,char**argv)
{
//Structureforgettingvideofromcameraoravi
CvCapture*capture=0;
//Imagestocapturetheframefromvideoorcameraorfrom
file
IplImage*frame,*frame_copy=0;
//Usedforcalculations
intoptlen=strlen("--cascade=");

2of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

//Inputfilenameforaviorimagefile.
constchar*input_name;
//Checkforthecorrectusageofthecommandline
if(argc>1&&strncmp(argv[1],"--cascade=",optlen)==
0)
{
cascade_name=argv[1]+optlen;
input_name=argc>2?argv[2]:0;
}
else
{
fprintf(stderr,
"Usage:facedetect--cascade=\"<cascade_path>\"
[filename|camera_index]\n");
return-1;
/*input_name=argc>1?argv[1]:0;*/
}
//LoadtheHaarClassifierCascade
cascade=(CvHaarClassifierCascade*)cvLoad(cascade_name,0,
0,0);

//Checkwhetherthecascadehasloadedsuccessfully.Else
reportanderrorandquit
if(!cascade)
{
fprintf(stderr,"ERROR:Couldnotloadclassifier
cascade\n");
return-1;
}

//Allocatethememorystorage
storage=cvCreateMemStorage(0);

//Findwhethertodetecttheobjectfromfileorfrom
camera.
if(!input_name||(isdigit(input_name[0])&&input_name[1]
=='\0'))
capture=cvCaptureFromCAM(!input_name?0:
input_name[0]-'0');
else
capture=cvCaptureFromAVI(input_name);
//Createanewnamedwindowwithtitle:result
cvNamedWindow("result",1);
//Findifthecaptureisloadedsuccessfullyornot.
//Ifloadedsuccesfully,then:
if(capture)
{
//Capturefromthecamera.
for(;;)
{
//CapturetheframeandloaditinIplImage
if(!cvGrabFrame(capture))
break;

3of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

frame=cvRetrieveFrame(capture);
//Iftheframedoesnotexist,quittheloop
if(!frame)
break;

//Allocateframecopyasthesamesizeoftheframe
if(!frame_copy)
frame_copy=cvCreateImage(
cvSize(frame->width,frame->height),
IPL_DEPTH_8U,
frame->nChannels);
//Checktheoriginofimage.Iftopleft,copythe
imageframetoframe_copy.
if(frame->origin==IPL_ORIGIN_TL)
cvCopy(frame,frame_copy,0);
//Elseflipandcopytheimage
else
cvFlip(frame,frame_copy,0);

//Callthefunctiontodetectanddrawtheface
detect_and_draw(frame_copy);
//Waitforawhilebeforeproceedingtothenext
frame
if(cvWaitKey(10)>=0)
break;
}
//Releasetheimages,andcapturememory
cvReleaseImage(&frame_copy);
cvReleaseCapture(&capture);
}
//Ifthecaptureisnotloadedsuccesfully,then:
else
{
//Assumetheimagetobelena.jpg,ortheinput_name
specified
constchar*filename=input_name?input_name:
(char*)"lena.jpg";
//Loadtheimagefromthatfilename
IplImage*image=cvLoadImage(filename,1);
//IfImageisloadedsuccesfully,then:
if(image)
{
//Detectanddrawtheface
detect_and_draw(image);
//Waitforuserinput
cvWaitKey(0);
//Releasetheimagememory
cvReleaseImage(&image);
}

4of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

else
{
/*assumeitisatextfilecontainingthe
listoftheimagefilenamestobeprocessed-one
perline*/
FILE*f=fopen(filename,"rt");
if(f)
{
charbuf[1000+1];
//Getthelinefromthefile
while(fgets(buf,1000,f))
{
//Removethespacesifany,andcleanup
thename
intlen=(int)strlen(buf);
while(len>0&&isspace(buf[len-1]))
len--;
buf[len]='\0';
//Loadtheimagefromthefilenamepresent
inthebuffer
image=cvLoadImage(buf,1);
//Iftheimagewasloadedsuccesfully,then:
if(image)
{
//Detectanddrawthefacefromthe
image
detect_and_draw(image);

//Waitfortheuserinput,andrelease
thememory
cvWaitKey(0);
cvReleaseImage(&image);
}
}
//Closethefile
fclose(f);
}
}
}

//Destroythewindowpreviouslycreatedwithfilename:
"result"
cvDestroyWindow("result");
//return0toindicatesuccessfullexecutionoftheprogram
return0;
}
//Functiontodetectanddrawanyfacesthatispresentinan
image
voiddetect_and_draw(IplImage*img)
{
intscale=1;

5of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

//Createanewimagebasedontheinputimage
IplImage*temp=cvCreateImage(
cvSize(img->width/scale,img->height/scale),8,3);
//Createtwopointstorepresentthefacelocations
CvPointpt1,pt2;
inti;
//Clearthememorystoragewhichwasusedbefore
cvClearMemStorage(storage);
//Findwhetherthecascadeisloaded,tofindthefaces.If
yes,then:
if(cascade)
{
//Therecanbemorethanonefaceinanimage.So
createagrowablesequenceoffaces.
//Detecttheobjectsandstoretheminthesequence
CvSeq*faces=cvHaarDetectObjects(img,cascade,
storage,
1.1,2,
CV_HAAR_DO_CANNY_PRUNING,
cvSize(40,40));
//Loopthenumberoffacesfound.
for(i=0;i<(faces?faces->total:0);i++)
{
//Createanewrectanglefordrawingtheface
CvRect*r=(CvRect*)cvGetSeqElem(faces,i);
//Findthedimensionsoftheface,andscaleitif
necessary
pt1.x=r->x*scale;
pt2.x=(r->x+r->width)*scale;
pt1.y=r->y*scale;
pt2.y=(r->y+r->height)*scale;
//Drawtherectangleintheinputimage
cvRectangle(img,pt1,pt2,CV_RGB(255,0,0),3,8,0
);
}
}
//Showtheimageinthewindownamed"result"
cvShowImage("result",img);
//Releasethetempimagecreated.
cvReleaseImage(&temp);
}

Hopeyouunderstoodwhatthecodemeant.
Howtorunthiscode?Assaidinthecode,youhavetospecifytheexactlocationof
thehaarclassifierxmlfile.Itisnormallypresentinthe/OpenCV
/data/haarcascadeslocation.Youshouldverifythatthe
6of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

filehaarcascade_frontalface_alt.xmlispresentinsidethefolderornot.
SupposethatyouhaveinstalledOpenCVinitsdefaultlocation:C:/ProgramFiles.
Then,theexactlocationoftheclassifieris:"C:/ProgramFiles/OpenCV
/data/haarcascades/haarcascade_frontalface_alt.xml"Now,youcanrunthe
facedetectsamplewiththiscommandintheprompt:facedetect
--cascade="C:/ProgramFiles/OpenCV/data/haarcascades
/haarcascade_frontalface_alt.xml"
IfyouareusinganIDE,andwanttoplaywiththesourcecode,you'llneedtoaddthe
classifierlocationtotheadditionalcommandlinearguments.InVisualC++,youcan
gotoProject->Debugging->CommandArgumentsInAdditionalOptionstab,enter
this:--cascade="C:/ProgramFiles/OpenCV/data/haarcascades
/haarcascade_frontalface_alt.xml"Now,thesourcecodewillcompileandrun
fine.
Youcanoptionallyspecifytheavifileorimagefiletothecommandprompt,after
specifyingtheclassifier.Forexample,Ifyouhaveyourownimage:"face.jpg",then
youcanaddthattothecommandlinelikethis:
--cascade="C:/ProgramFiles/OpenCV/data/haarcascades
/haarcascade_frontalface_alt.xml"face.jpg

OnLinux
1Gotothedirectory:opencv-0.9.7/samples/c/
2Changethepermissionsofbuild_all.shfile:chmod777build_all.sh
3Runthebuild_all.shfile:./build_all.sh
4Runfacedetect:./facedetect--cascade="/home/bob/opencv-0.9.7/data/haarcascades
/haarcascade_frontalface_alt.xml"
(makesureyouspecifytheexactpathofanxmlfile!!!)
5Torunwithaparameter:./facedetect--cascade="/home/bob/opencv-0.9.7/data
/haarcascades/haarcascade_frontalface_alt.xml"baboon.jpg

Whatifyouwanttoaddthisasapartofyour
sourcecode?
Passyourimagetothedetect_and_drawfunction.Thatsit....Hereisasamplecode:

7of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

//SampleApplicationtodemonstratehowFacedetectioncanbe
doneasapartofyoursourcecode.
//Includeheaderfiles
#include"cv.h"
#include"highgui.h"
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<assert.h>
#include<math.h>
#include<float.h>
#include<limits.h>
#include<time.h>
#include<ctype.h>

//Createastringthatcontainstheexactcascadename
constchar*cascade_name=
"C:/ProgramFiles/OpenCV/data/haarcascades
/haarcascade_frontalface_alt.xml";
/*"haarcascade_profileface.xml";*/

//Functionprototypefordetectinganddrawinganobjectfrom
animage
voiddetect_and_draw(IplImage*image);
//Mainfunction,definestheentrypointfortheprogram.
intmain(intargc,char**argv)
{
//Createasampleimage
IplImage*img=cvLoadImage("face.jpg");
//Callthefunctiontodetectanddrawthefacepositions
detect_and_draw(img);
//Waitforuserinputbeforequittingtheprogram
cvWaitKey();
//Releasetheimage
cvReleaseImage(&img);

//Destroythewindowpreviouslycreatedwithfilename:
"result"
cvDestroyWindow("result");
//return0toindicatesuccessfullexecutionoftheprogram
return0;
}
//Functiontodetectanddrawanyfacesthatispresentinan
image
voiddetect_and_draw(IplImage*img)
{

8of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

//Creatememoryforcalculations
staticCvMemStorage*storage=0;
//CreateanewHaarclassifier
staticCvHaarClassifierCascade*cascade=0;
intscale=1;
//Createanewimagebasedontheinputimage
IplImage*temp=cvCreateImage(
cvSize(img->width/scale,img->height/scale),8,3);
//Createtwopointstorepresentthefacelocations
CvPointpt1,pt2;
inti;
//LoadtheHaarClassifierCascade
cascade=(CvHaarClassifierCascade*)cvLoad(cascade_name,0,
0,0);

//Checkwhetherthecascadehasloadedsuccessfully.Else
reportanderrorandquit
if(!cascade)
{
fprintf(stderr,"ERROR:Couldnotloadclassifier
cascade\n");
return;
}

//Allocatethememorystorage
storage=cvCreateMemStorage(0);
//Createanewnamedwindowwithtitle:result
cvNamedWindow("result",1);
//Clearthememorystoragewhichwasusedbefore
cvClearMemStorage(storage);
//Findwhetherthecascadeisloaded,tofindthefaces.If
yes,then:
if(cascade)
{
//Therecanbemorethanonefaceinanimage.So
createagrowablesequenceoffaces.
//Detecttheobjectsandstoretheminthesequence
CvSeq*faces=cvHaarDetectObjects(img,cascade,
storage,
1.1,2,
CV_HAAR_DO_CANNY_PRUNING,
cvSize(40,40));
//Loopthenumberoffacesfound.
for(i=0;i<(faces?faces->total:0);i++)
{
//Createanewrectanglefordrawingtheface
CvRect*r=(CvRect*)cvGetSeqElem(faces,i);

9of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

//Findthedimensionsoftheface,andscaleitif
necessary
pt1.x=r->x*scale;
pt2.x=(r->x+r->width)*scale;
pt1.y=r->y*scale;
pt2.y=(r->y+r->height)*scale;
//Drawtherectangleintheinputimage
cvRectangle(img,pt1,pt2,CV_RGB(255,0,0),3,8,0
);
}
}
//Showtheimageinthewindownamed"result"
cvShowImage("result",img);
//Releasethetempimagecreated.
cvReleaseImage(&temp);
}

HowtomaketheFaceDetectorwork
inMFCApplication:
ContributedbySalmanAslam
Ifyou'reworkingwithanMFCapplication,followthesestepstogetyour
HaarFacialDetectorworking:
Step1InstallIntelOpenCV.Iwillassumeit'sinstalledinthedefaultdirectory:
c:\programfiles\opencv
Step2CreateastandardMFCapplicationusingtheAppWizard.
Step3InProjectSettings(Alt+F7),gototheC/C++tab,thenselectCategory
Preprocessor,andforAdditionalIncludedirectories,addthefollowing:C:\Program
Files\opencv\cv\include,C:\ProgramFiles\opencv\otherlibs\highgui,C:\Program
Files\opencv\cxcore\include,C:\ProgramFiles\opencv\cvaux\include
Step4InProjectSettings(Alt+F7),gototheLinktab,thenselectCategory"Input",
andfor"Object/librarymodules:",addthefollowing:cxcored.libcvd.libhighguid.lib
(seethattherearenocommaslikeinStep3above)
Step5WhilestayingintheLinktab,andCategory"Input",forthe"Additional
Librarypath:",addthefollowing:C:\ProgramFiles\opencv\lib
Step6Inyourmainapplicationfile,egSimple.cpp,includethefollowingfiles:

10of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

#include"cv.h"#include"highgui.h"
Step7Inthesamefile,gototheInitInstance()function.Rightafteryouseethelines:
m_pMainWnd->ShowWindow(SW_SHOW);
m_pMainWnd->UpdateWindow();

andbeforetheline
returnTRUE;

addthefollowinglinesofcode
//declarations
CvHaarClassifierCascade*cascade;
CvMemStorage*storage;
IplImage*image;
CvSeq*faces;
constchar*file_name;
inti;
//initializations
storage=cvCreateMemStorage(0);
cvFirstType();
file_name="haarcascade_frontalface_alt.xml";
cascade=(CvHaarClassifierCascade*)cvLoad(file_name,NULL,
NULL,NULL);
image=cvLoadImage("lena.jpg",1);
faces=cvHaarDetectObjects(image,cascade,storage,1.2,2,
CV_HAAR_DO_CANNY_PRUNING,cvSize(0,0));
//drawrectangles
for(i=0;i<(faces?faces->total:0);i++)
{
CvRect*r=(CvRect*)cvGetSeqElem(faces,i);
CvPointpt1={r->x,r->y};
CvPointpt2={r->x+r->width,r->y+r->height};
cvRectangle(image,pt1,pt2,CV_RGB(255,0,0),3,8,0);
}
//createwindowandshowtheimagewithoutlinedfaces
cvNamedWindow("faces",1);
cvShowImage("faces",image);
cvSaveImage("Result.jpg",image);
cvWaitKey();
cvReleaseImage(&image);//afterakeypressed,
releasedata
cvReleaseHaarClassifierCascade(&cascade);
cvReleaseMemStorage(&storage);

Step8Nowgotoc:\programfiles\opencv\bin,andcopythefollowingfilestothe
folderwhereyourMFCfilesreside:

11of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

cv097d.dllcxcore097d.dllhighgui097d.dll
Step9Nowgotoc:\programfiles\opencv\samples\candcopylena.jpgtothefolder
whereyourMFCfilesreside.
Step10Gotoc:\programfiles\opencv\data\haarcascades,andcopythefollowingfile
tothefolderwhereyourMFCfilesreside:
haarcascade_frontalface_alt.xml
Step11Runyourapplication.
Alotofpeoplehavebeenreceivinganerrorwhileexecutingthefollowingline:
cascade=(CvHaarClassifierCascade*)cvLoad(file_name,NULL,NULL,
NULL);

Theerrorlookslikethis:Unspecifiederror(Thenodedoesnotrepresentauser
object(unknowntype?))infunctioncvRead,C:\ProgramFiles\OpenCV\cxcore
\src\cxpersistence.cpp(5040)
Thesolutionistousecxcored.lib,cvd.libandhighguid.libinsteadofcxcored_i7.lib,
cv.libandhighgui.lib.Youcanusehighgui.lib,butyougetanerroraftertheface
detectedimagehasbeendisplayedandyou'reunloadingit.
Anothersimpleworkaroundistocallanyfunctionfromcv.libbeforethecallto
cvLoad.Forexample:createadummyemptyimage,applycvErodetoitandrelease
theimage.
Theabovecode(functiondetectanddraw)hasaseriousmemoryleakwhen
runinaninfiniteforloopforrealtimefacedetection.Pleaseadd
"cvReleaseMemStorage(&storage);"afteryoureleasethetempimageinthe
detectanddrawfunction.ThecvClearMemStorageonlyclearsthepreviousset
ofvaluesbutdoesnotfreememory.Ididnotmodifytheinitialcodeasitis
simplynotmine.Ihavealsocleared"faces"usingcvClearSeqinmyprocessof
tryingtofindthememoryleakastheprogramwouldquicklycrashfor
640x480res30fpsvideo.Iamverythankfultotheoriginalprogrammerfor
thecode.Pleaseletmeknowifthishelps.--AbhinayE
References:
OpenCVDocumentation
OpenCVSamples

12of13

06/01/201106:45PM

FaceDetection-OpenCVWiki

http://opencv.willowgarage.com/wiki/FaceDetection

Testedon:
OpenCVBeta5
MicrosoftVisualC++
WindowsXP
OpenCVWiki:FaceDetection(lastedited2011-02-0322:00:22byGaryBradski)

13of13

06/01/201106:45PM