Artificial Intelligence Our Ultimate Dream

Mohammad Abdelaziz Gowayyed May 2013

Agenda
Introduction Example: How to build a machine to play chess? Artificial Intelligence Subfields Feature Extraction for Classification Artificial Neural Networks Future of Artificial Intelligence

What is Artificial Intelligence?

The study of computer systems that attempt to model and apply the intelligence of the human mind

Artificial Intelligence Directions

Build Systems that:
act like humans act rationally think like humans think rationally

Act like human: The Turing Test

Turing Test:
If a man had a conversation with a machine and he couldnt tell, whether it was a machine or a human, this machine passes the Turing Test It means that the machines perfectly acts like a human Does it need to think like a human to act like a human?

Think like human: Cognitive Science

We must somehow determine first how human think, do we know? Introspection Psychological experiments Cognitive Science: construct precise and testable theories of the workings of human minds through computer models and psychological experiments.

Think rationally: Laws of Thought

Formalizing logic Main emphasis is on inference Example: Socrates is a man; all men are mortal; therefore Socrates is mortal. By 1965, programs existed that could, given enough time and memory, take a description of a problem in logical notation and find the solution to the problem, if one exists. Problems?

Act rationally: The rational agent

Rational behavior: doing the right thing The right thing: that which is expected to maximize goal achievement, given the available information Take decisions Reinforcement Learning: learn from environment feedback based on my choices

Is there a benchmark for human intelligence?

May be Chess?

How to program a machine to play Chess?

How to program a machine to play Tic Tac Toe?

Easy?

What is the difference between Chess and Tic Tac Toe?

The general idea is to build a tree representing different possible paths, each path represents sequence of choices from you and your opponent. The tree will help the computer choose That is called a Game Tree

Min-Max Algorithm
Two agents are playing Zero-Sum Game
Any loss to you is an equal win for your opponent

Assume that you are playing with a rational player; he will try to maximize his profit At each step, you want to maximize your profit and assume that the opponent will minimize it

Min-Max Algorithm
Is it feasible to create the whole tree? Is it finite? What about the depth? Will we be forced to choose random moves? How to evaluate each state?
Heuristic function (e.g. number of safe pieces)

Different algorithms for traversing the tree

Depth first Breadth First A*

Do we need to visit the whole tree?

Alpha-Beta pruning

Heuristic Function

Can we say that the computer is now intelligent because it can defeat human in chess?

There are still a lot of tasks that human does easily while it is very hard for machines Can you name a few?

Chinese Room

Chinese Room

Machine vs. Human

Machine
Easy Tasks
Multiply 4878 * 1254 Play chess

Human
Easy Tasks
Recognize faces Recognize voices Understand concepts

Hard Tasks
Understand concepts Recognize faces Recognize voices

Hard Tasks
Multiply 4878 * 1254 Play chess

Some Artificial Intelligence Subfields

Machine Learning
Let machines learn rules (e.g. to classify or to choose between alternatives Reinforcement Learning)

Computer Vision
Teach machines how to recognize visual world

Natural Language Processing

Teach machines how to recognize human languages

Speech Recognition
Teach machines how to recognize human voices

Some Computer Vision Tasks

Face Detection

Image Classification

Human Action Recognition

Feature Representation

Feature Representation

How is computer perception done?

Feature Representation

Computer Vision Features

Audio Features

NLP Features

Sensor Representation in Human Brain

Can we mimic what already happens in human brains?

The human brain consists of a network of neurons. Human brains learns to recognize images by the same method as it learns to recognize voices. Scientists still dont know how the brain works. Scientists only know some concepts that brain applies (e.g. sparse and general representations)

Human Nervous System

Around 85,000,000,000 neurons each one connected to around 1000 others

Number of neurons in elephant brains

11,000,000,000 neurons

Number of neurons in horses brains

1,200,000,000 neurons

Neuron

Neuron

Number of neurons in human brains

From 19,000,000,000 to 23,000,000,000 neurons

Artificial Neural Networks

We know that human brains consists of huge neural network, lets study simpler artificial neural networks. And some day, we may be able to create artificial neural network that works the same as the real neural network works. Some people thought that it will never work! They had their arguments. Thats why some researchers dont like neural networks.

Neural Networks

Neural Networks

Backpropagation

Backpropagation

How to build better Neural Networks?

Increase Layers?
How would we train them?

The network with more than 1 hidden layer is called a deep network Training a deep network is not easy using Backpropagation.

The birth of Deep Learning

Until 2006, Support Vector Machines always outperformed neural networks on benchmark classification tasks. At 2006, Hinton started the new revolution on machine learning: Deep Learning

The birth of Deep Learning

At 2006, Hinton proposed a greedy layer-wise algorithm for training a deep belief network. That was the first successful learning algorithm for a deep network. Achieved best accuracy on MNIST hand written digits dataset. Restricted Boltzmann Machines. The network works also as a generative model to generate data.

Deep Belief Network

Demo

Classical Approach
Train a classifier using labeled data; then test the classification accuracy on another testing data. Row data can be: images, videos, audio, natural language We start with extracting appropriate features. Hand-crafted feature extraction will differ according to the nature of the data.

Deep Learning Approach

Pre-train the network using non-labeled dataavailable. Fine-tune the weights using some labeled data. Also called: Unsupervised Feature Learning Deep Learning Models:
Deep Belief Networks Stacked Autoencoders Convolutional Neural Networks

Sparse Autoencoders

What each neuron learnt?

What each neuron learnt?

Motivating Results
Unsupervised feature extraction outperformed hand-crafted features in set of popular machine recognition tasks:
Human Activity Recognition Handwritten digits Recognition Audio classification ImageNet classification

Biological Mapping
Interesting paper at 2005: Invariant visual representation by single neurons in the human brain They showed that some neurons respond with high selectivity to, for instance, images, sound and name of the actress Halle Berry!

How Many Computers to Identify a Cat? 16,000

Andrew NG, with Google at 2012, trained a very large network: on16,000 machines. They trained the network on random unlabeled videos from Youtube, without any guidance. They found that one neuron of the network learnt to detect human faces and another one learnt to detect cat faces!

Some neurons learnt human face!

Other neurons learnt cat faces!

How Many Computers to Identify a Cat? 16,000

Future of Artificial Intelligence

With the increase of computational power, every day, scientists are able to build more complex neural networks Challenges
Network topologies Learning algorithms

Neuroscientists can lead an AI revolution if they came closer to how the brain works

Acknowledgment
Some slides are taken from Andrew Ng deep learning tutorial. Images used here are for educational purposes, any further use should return back to the original copyright holders.

Thanks You

Questions?