FacelandmarkdetectionusingCNN

Task

Theobjectiveofthistaskistopredictkeypointpositionsonfaceimages.

Usage

trackingfacesinimagesandvideo

analyzingfacialexpressions

detectingdysmorphicfacialsignsformedicaldiagnosis

biometrics/facerecognition

Method

DeepLearning

CNNregression

ComputerVision

FacelandmarkdetectionusingCNN

SomeDatasets

CUHK: MALF&MTLF

12995,20000images

5keypoints

Kaggle: FacialKeypointDetection

7049Images

96x96

15keypoints

ComputerVision

/notes/2014/12/17/using-

convolutional-neural-nets-to-detect-facial-keypoints-thutttop:r/ia/lm/#mthlaeb-.hk/projects/TCDCN.ht

FacelandmarkdetectionusingCNN

Method

CNN

Regression

Whatlossshouldweuse?

ComputerVision

.hk/projects/TCDCN.ht

ml

FacelandmarkdetectionusingCNN

Method

CNN

Regression

Whatlossshouldweuse?

Consider:

Whatlabelwehave?

Whattargetwewant?

Howtocomputeloss?

ComputerVision

.hk/projects/TCDCN.ht

ml

FacelandmarkdetectionusingCNN

Method

CNN

Regression

Howtocomputeloss?

Whatlossshouldweuse?

ComputerVision

Consider:

Whatlabelwehave?

Keypoints,

e.g.,{(x1,y1),(x2,y2)…

Whattargetwewant?

Locationsofkeypoints

.hk/projects/TCDC

N.html

FacelandmarkdetectionusingCNN

?

ComputerVision

10 E

/.hk/projects/TCDCN.html

10

GroundTruth

Loss

FacelandmarkdetectionusingCNN

?

ComputerVision

FacelandmarkdetectionusingCNN

?

ComputerVision

Schedule

FacelandmarkdetectionusingCNN

CNNregression

Faciallandmarkdetection

GenderClassificationonFaceimageusingCNN

TransferLearning

AdvancedTopics

RNN/LSTMincomputervision

Cuttingedgetechniquesindeeplearning

ClassSummary

ComputerVision

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Problem:

Estimategender(Male/Female)givenafaceimage

Data:

FaceImage

GenderInformation(2classproblem)

Database:

CUHK:MALF&MTLF

CUHK:CelebA

ComputerVision

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Method:

CNN

Whatisourtrainingdata?

Faceimage

Gender:{0,1}=>{Male,Female}

Howtoorganizethetraining?

Trainfromscratch

Usepre-trainedmodelandfine-tuning

Whatnetworkshouldweuse?

Whatlossshouldweuse?

Howtotestourmodel?

ComputerVision

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Whatisourtrainingdata?

Faceimage

Gender:{0,1}=>{Male,Female}

Howtoorganizethetraining?

Trainfromscratch

Usepre-trainedmodelandfine-tuning

Whatnetworkshouldweuse?

AlexNet,VGG,ResNet18,etc…

Whatlossshouldweuse?

Softmax

CrossEntropy

Howtotestourmodel?

ComputerVision

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Method:

Usepre-trainedmodelandfine-tuning

Idea:

FaceRecognitionmodel

+

FacewithGenderData

+

Training

=

GenderModel

ComputerVision

Fine-tuningtakesanalreadylearnedmodel,adaptsthearchitecture,andresumestrainingfromthealreadylearnedmodelweights.

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Method:

Usepre-trainedmodelandfine-tuning

Idea:

FaceRecognitionmodel

+

FacewithGenderData

+

Training

=

GenderModel

TransferLearning

ComputerVision

FaceGenderClassification

GenderClassificationonFaceimageusingCNN

Method:

Usepre-trainedmodelandfine-tuning

Pre-trainedmodel:

VGGFaceRecognitionmodel(forclassificatione.g.,1000identities)

Fine-tunethenet:

FaceGenderData

Similarnetworkarchitecture

Changelastlayer(s)tothegenderclassificationtask

Extension:

Task1

Task2

Task3

Multi-taskDCNN

ComputerVision

Schedule

FacelandmarkdetectionusingCNN

CNNregression

Faciallandmarkdetection

GenderClassificationonFaceimageusingCNN

TransferLearning

AdvancedTopics

RNN/LSTMincomputervision

Activationsfunctions

AdvancedLayers

AdvancedNetworkarchitectures

TrainingTricks

ClassSummary

ComputerVision

RNN/LSTMinComputerVision

RecurrentNeuralNetwork(RNN)

ComputerVision

Aloopallowsinformationtobepassedfromonestepofthenetworktothenext

http://colah.github.io/posts/2015-08-Understanding-

RNN/LSTMinComputerVision

RecurrentNeuralNetwork(RNN)

WhyweneedRNN?

ComputerVision

Handwave?Standup?

RNN/LSTMinComputerVision

RecurrentNeuralNetwork(RNN)

RNNProblem

LSTM

Long-ShortTermMemory

ComputerVision

StandardRNN

LSTM

RNN/LSTMinComputerVision

RecurrentNeuralNetwork(RNN)

ApplicationsinComputervision:

ObjectTracking

ActionRecognition

VideoCaptioning

Videoanalysis

Imagegeneralization

Applicationsinothermlarea:

Translation

NLP(e.g.,wordprediction)

Speechrecognition

ComputerVision

AdvancedTopics

ActivationFunction

Sigmoid,Tanh,ReLU

AdvancedActivationFunctions:

LeakyReLU

ParametricReLU

RandomizedRuLU

ELU

ComputerVision

AdvancedTopics

AdvancedLayers

DilatedConv

BNLayer

RecurrentLayer

RNN

LSTM

ComputerVision

AdvancedTopics

AdvancedLayers

DilatedConv

ComputerVision

Dilatedconvolutions“inflate”thekernelbyinsertingspacesbetweenthekernelelements.

largerreceptivefield,

efficientcomputationandlessermemoryconsumption

Poolingmakesreceptivefieldsmallerandsmaller

Up-samplingcannotrestorelostinformation

Dilatedconvhelpkeepthereceptivefiled

AdvancedTopics

BNLayer

IssuesWithTrainingDeepNeuralNetworks

InternalCovariateshift

VanishingGradient

AdvantagesofBN:

Reducesinternalcovariantshift.

t

Reducesthedependenceofgradientsonscaleoftheparametersortheirinitialvalues.

Regularizesthemodelandreducestheneedfordropout,localresponsenormalizationandotherregularizationtechniques.

Allowsuseofsaturatingnonlinearitiesandhigherlearningrates.

ComputerVision

AdvancedTopics

NetworkStructure

DenseNet

ResNext

SqueezeNet

TinyDarknet

ComputerVision

AdvancedTopics

NetworkStructure

DenseNet

ResNext

SqueezeNet

TinyDarknet

ComputerVision

AdvancedTopics

NetworkStructure

DenseNet

ResNext

SqueezeNet

TinyDarknet

ComputerVision

AdvancedTopics

NetworkStructure

DenseNet

ResNext

SqueezeNet

TinyDarknet

ComputerVision

AdvancedTopics

NetworkStructure

1x1conv

Combinemultiplechannels

Dimensionreduction

ComputerVision

1x1convWith32

56 filters 56

Eachfilter

56 hassize 56

64 1x1x64,and 32

performsa64dimdotproduct

TrainingTricks

GPU 分布式訓(xùn)練

Synchronous

PlacesanindividualmodelreplicaoneachGPU.SplitthebatchacrosstheGPUs.

UpdatesmodelparameterssynchronouslybywaitingforallGPUstofinishprocessingabatchofdata.

Asynchronous

ComputerVision

TrainingTricks

GPU 分布式訓(xùn)練

Synchronous

“l(fā)astexecutor”effect

ComputerVision

synchronoussystemshavetowaitontheslowestexecutorbeforecompletingeachiteration.

TrainingTricks

GPU 分布式訓(xùn)練

Asynchronous

Stalegradientproblem

ComputerVision

TrainingTricks

DataNormalization

InputData

Continuousdata:

Normalizeto[0,1]or[-1,1],ormean=0&std=1

DiscreteLabeldata: Onehotvector

E.g.,3classes[0,1,2] ?[[1,0,0],[0,1,0],[0,0,1]]

ComputerVision

Note:Normalizationmethodintrainingantestingmustbethesame!

TrainingTricks

WeightInitialization

Principle:

Nottoolarge,Nottoosmall

Xavier

Gaussian

biasusuallysettoconstant(e.g.,0)

Etc.

ComputerVision

TrainingTricks

EpochandIteration

EpochUsually>>1

#ofIterations=#ofEpoch*data_size/batch_size

Small#ofEpoch:

Underfitting

Large#ofEpoch:

Overfitting

Howtodecide?

Earlystopping

ComputerVision

TrainingTricks

LearningRate

Oneofthemostimportantparamintraining

Toosmall:

Slow,sometimesnotconverge

Toolarge:

Noconvergence

Usuallyrange:

0.1---1e-6

Howtodecide?

Visualizetraining

TrainingfromscratchandFinetuning

ComputerVision

TrainingTricks

Activationfunction

Hiddenlayers:

ReLUandLeakyRelu

LSTM:

SigmoidandTanh

ComputerVision

OutputLayers:

Classification:Softmax

Regression:Identity

no-opactivation,usefultoimplementlinearbottleneck,returnsf(x)=x

TrainingTricks

Lossfunction:

Yournet’spurposedeterminethelossfunctionyouuse.

Forexample,

inclassificationproblem:usemulticlasscrossentropyloss.

inregressionproblem:useEuclideanloss.

ComputerVision

TrainingTricks

Regularization:

Helppreventoverfitting

L1andL2regularization

UsuallyL2,decay1e-3to1e-6

Dropout

Usually0.3or0.5

EarlyStopping

ComputerVision

TrainingTricks

BatchSize:

Toosmall:

Slowtraining

DonotutilizeGPU

Toolarge:

Overfitting(ICLR2017paper)

Usually:

16,32,128…

ComputerVision

TrainingTricks

Solver/Optimizer:

SGD

Momentum

Adam/RMSProp

ComputerVision

Schedule

FacelandmarkdetectionusingCNN

CNNregression

Faciallandmarkdetection

GenderClassificationonFaceimageusingCNN

TransferLearning

AdvancedTopics

RNN/LSTMincomputervision

Activationsfunctions

AdvancedLayers

AdvancedNetworkarchitectures

TrainingTricks

ClassSummary

ComputerVision

ComputerVision

ClassSummary

Week 1

機器的力量：將數(shù)據(jù)轉(zhuǎn)化為知識

機器學(xué)習(xí)的整體概念

監(jiān)督學(xué)習(xí)，非監(jiān)督學(xué)習(xí)，增強學(xué)習(xí)

機器學(xué)習(xí)系統(tǒng)的Roadmap

Machinelearning經(jīng)典算法:機器學(xué)習(xí)≠深度學(xué)習(xí)

K-meansclustering

K-NN,SVM

Regression Task

Experience

LearnedProgram

Tas