Lecture Slides for

INTRODUCTION TO

Machine Learning
Presented by : Tirth sheladiya
CHAPTER 1:

Introduction
Why “Learn” ?
■ Machine learning is programming computers to optimize
a performance criterion using example data or past
experience.
■ There is no need to “learn” to calculate payroll
■ Learning is used when:
◻ Human expertise does not exist (navigating on Mars),
◻ Humans are unable to explain their expertise (speech
recognition)
◻ Solution changes in time (routing on a computer network)
◻ Solution needs to be adapted to particular cases (user
biometrics)

3
What We Talk About When We
Talk About“Learning”
■ Learning general models from a data of particular
examples
■ Data is cheap and abundant (data warehouses, data
marts); knowledge is expensive and scarce.
■ Example in retail: Customer transactions to consumer
behavior:
People who bought “Da Vinci Code” also bought “The Five
People You Meet in Heaven” (www.amazon.com)
■ Build a model that is a good and useful approximation to
the data.

4
Data Mining
■ Retail: Market basket analysis, Customer relationship
management (CRM)
■ Finance: Credit scoring, fraud detection
■ Manufacturing: Optimization, troubleshooting
■ Medicine: Medical diagnosis
■ Telecommunications: Quality of service optimization
■ Bioinformatics: Motifs, alignment
■ Web mining: Search engines
■ ...

5
What is Machine Learning?
■ Optimize a performance criterion using example data or
past experience.
■ Role of Statistics: Inference from a sample
■ Role of Computer science: Efficient algorithms to
◻ Solve the optimization problem
◻ Representing and evaluating the model for inference

6
Applications
■ Association
■ Supervised Learning
◻ Classification
◻ Regression

■ Unsupervised Learning
■ Reinforcement Learning

7
Learning Associations
■ Basket analysis:
P (Y | X ) probability that somebody who buys X also
buys Y where X and Y are products/services.

Example: P ( chips | beer ) = 0.7

8
Classification
■ Example: Credit
scoring
■ Differentiating
between low-risk
and high-risk
customers from their
income and savings

Discriminant: IF income > θ1 AND savings > θ2

9
Classification: Applications
■ Aka Pattern recognition
■ Face recognition: Pose, lighting, occlusion (glasses,
beard), make-up, hair style
■ Character recognition: Different handwriting styles.
■ Speech recognition: Temporal dependency.
◻ Use of a dictionary or the syntax of the language.
◻ Sensor fusion: Combine multiple modalities; eg, visual (lip
image) and acoustic for speech
■ Medical diagnosis: From symptoms to illnesses
■ ...

10
Face Recognition
Training examples of a person

Test images

11
Regression

■ Example: Price of a used

12
Regression Applications
■ Navigating a car: Angle of the steering wheel (CMU
NavLab)
■ Kinematics of a robot arm
(x,y) α1= g1(x,y)
α2= g2(x,y)
α2

α1

13
Supervised Learning: Uses
■ Prediction of future cases: Use the rule to predict the
output for future inputs
■ Knowledge extraction: The rule is easy to understand
■ Compression: The rule is simpler than the data it
explains
■ Outlier detection: Exceptions that are not covered by the
rule, e.g., fraud

14
Unsupervised Learning
■ Learning “what normally happens”
■ No output
■ Clustering: Grouping similar instances
■ Example applications
◻ Customer segmentation in CRM
◻ Image compression: Color quantization
◻ Bioinformatics: Learning motifs

15
Reinforcement Learning
■ Learning a policy: A sequence of outputs
■ No supervised output but delayed reward
■ Credit assignment problem
■ Game playing
■ Robot in a maze
■ Multiple agents, partial observability, ...

16
Resources: Datasets
■ UCI Repository:
http://www.ics.uci.edu/~mlearn/MLRepository.html
■ UCI KDD Archive:
http://kdd.ics.uci.edu/summary.data.application.html
■ Statlib: http://lib.stat.cmu.edu/
■ Delve: http://www.cs.utoronto.ca/~delve/

17
Resources: Journals
■ Journal of Machine Learning Research www.jmlr.org
■ Machine Learning
■ Neural Computation
■ Neural Networks
■ IEEE Transactions on Neural Networks
■ IEEE Transactions on Pattern Analysis and Machine
Intelligence
■ Annals of Statistics
■ Journal of the American Statistical Association
■ ...
18
Resources: Conferences
■ International Conference on Machine Learning (ICML)
◻ ICML05: http://icml.ais.fraunhofer.de/
■ European Conference on Machine Learning (ECML)
◻ ECML05: http://ecmlpkdd05.liacc.up.pt/
■ Neural Information Processing Systems (NIPS)
◻ NIPS05: http://nips.cc/
■ Uncertainty in Artificial Intelligence (UAI)
◻ UAI05: http://www.cs.toronto.edu/uai2005/
■ Computational Learning Theory (COLT)
◻ COLT05: http://learningtheory.org/colt2005/
■ International Joint Conference on Artificial Intelligence (IJCAI)
◻ IJCAI05: http://ijcai05.csd.abdn.ac.uk/
■ International Conference on Neural Networks (Europe)
◻ ICANN05: http://www.ibspan.waw.pl/ICANN-2005/
■ ...

19