Scribd
Upload
3 views

01-intro

The document outlines the course ECE 6254 on Statistical Machine Learning, focusing on learning effective models from data for practical inference and signal processing problems. It covers various learning approaches, including supervised and unsupervised learning, and emphasizes the importance of probabilistic models. Prerequisites include knowledge of probability, linear algebra, and multivariable calculus, with no formal textbook required, and grading based on tests, homework, and projects.

Uploaded by

Mark Davenport
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

01-intro

The document outlines the course ECE 6254 on Statistical Machine Learning, focusing on learning effective models from data for practical inference and signal processing problems. It covers various learning approaches, including supervised and unsupervised learning, and emphasizes the importance of probabilistic models. Prerequisites include knowledge of probability, linear algebra, and multivariable calculus, with no formal textbook required, and grading based on tests, homework, and projects.

Uploaded by

Mark Davenport
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

ECE 6254

Statistical Machine Learning


Spring 2024

Mark A. Davenport
Electrical & Computer
Disclaimer!
None of what I just said was written by me…

Me at 11pm last night:

– Lesson learned: If you want comedy (and nonsense), use Bard

Caveats
– I disavow the statement “this course is not all about math and equations”
– Natural language processing and self-driving cars are not going to be a central focus
Statistical machine learning
• How can we
– learn effective models from data?
– apply these models to practical inference and signal processing problems?

• Example problems: classification, regression, prediction, data modeling,


clustering, and data exploration/visualization

• Our approach: statistical inference

• Main subject of this course


– how to reason about and work with probabilistic models to help us make inferences
from data
What is machine learning?
learn: gain or acquire knowledge of or skill in (something) by
study, experience, or being taught

How do we learn that this is a tree?


My daddy told me that a tree is
a perennial plant with an
elongated stem, or trunk,
supporting leaves or branches.

This has a trunk and branches.

Therefore it is a tree.
What is machine learning?
learn: gain or acquire knowledge of or skill in (something) by
study, experience, or being taught

How do we learn that this is a tree?

EXAMPLES!

A good definition of learning for this course:


“using a set of examples to infer
something about an underlying process”
Why learn from data?
Traditional signal processing is “top down”

Given a model for our data, derive the optimal algorithm

A learning approach is more “bottom up”

Given some examples, derive a good algorithm

Sometimes a good model is really hard to derive from first principles


Examples of learning
The Netflix prize (2007)

Predict how a user will rate a movie


10% improvement = $1 million prize

• Some pattern exists


– users do not assign ratings completely at random – if you like Godfather I, you’ll
probably like Godfather II

• It is hard to pin down the pattern mathematically

• We have lots and lots of data


– we know how a user has rated other movies, and we know how other users have
rated this (and other) movies
Examples of learning
• Recommendation systems
• Speech recognition
• Image classification
• Object detection
• Language modeling
• Spam filtering
• Machine translation
• Time series forecasting (traffic, weather, markets, etc.)
• Search
• Fraud detection
• Medical diagnosis
• …
Supervised learning
We are given input data

Each represents a measurement or observation of some natural or man-made


phenomenon
– may be called input, pattern, signal, feature vector, instance, or independent
variable
– the coordinates may be called features, attributes, predictors, or covariates

In the supervised case, we are also given output data


– may be called output, label, response, or dependent variable

The data are called the training data


Supervised learning
We can think of a pair as obeying a (possibly noisy) input-output
relationship

The goal of supervised learning is usually to generalize the input-output


relationship so that we can predict the output associated with a previously
unseen input

The primary supervised learning


problems are
– classification:

– regression:
Unsupervised learning
The inputs are not accompanied by labels

The goal of unsupervised learning is typically not related to future observations

Instead we want to understand that structure in the data sample itself, or to infer
some characteristic of the underlying probability distribution

Examples of unsupervised learning problems include


– clustering
– density estimation
– dimensionality reduction/feature selection
– visualization
– generative modeling
Other variants of learning
• semi-supervised learning
• self-supervised learning
• active learning
• online learning
• reinforcement learning
• anomaly detection
• transfer learning
• multi-task learning
• …

In general, most learning problems can be thought of as variants of traditional


signal processing problems, but where we have no idea (a priori) how to model
our signals
Prerequisites
• Probability
– random variables, expectation, joint distributions, independence, conditional
distributions, Bayes rule, multivariate normal distribution, …

• Linear algebra
– norms, inner products, orthogonality, linear independence, eigenvalues/vectors,
eigenvalue decompositions, …

• Multivariable calculus
– partial derivatives, gradients, the chain rule, …

• Python or similar programming experience (C or MATLAB)


Text
There is no formally required textbook for this course, but I will draw material
heavily from these sources:

A list of other useful books and links to relevant papers will be posted on the
course webpage

Lecture notes and slides will also be posted on the course webpage
Grading
• Pre-test (5%)

• Homework (25%)

• Data challenges (10%)

• Midterm exam (20%)

• Final exam (20%)

• Final project (20%)


Distance learning
Welcome to our online students!

Recorded lectures will be available to all students


(including on-campus students)

I need your help to make this a success

Online resources:
• Course website
• Canvas
• Piazza
A brief interlude

Gradus
Descendo!
Could you learn this trick?
Suppose that
• denotes the color of the card
– 0 = black
– 1 = red
• denotes which card is hidden
– E.g., Ace of Spades, Queen of Hearts, …

You observe me doing this trick many times and form a dataset:

Can you learn a function such that is a reliable predictor of ?


Another approach
You watch me do this trick a couple times and notice I always hand out 5 cards

Suppose you instead consider

Now, can you learn a function such that is a reliable predictor of ?


Is learning even possible?
or: How I learned to stop worrying and love statistics

Supervised learning
Given training data , we would like to learn an (unknown)
function such that for other than

but…

as we have just seen, this is impossible. Without any additional assumptions, we


conclude nothing about except (maybe) for its value on
Probability to the rescue!
Any agreeing with the training data may be possible
but that does not mean that any is equally probable

A short digression
• Suppose that Javier has a biased coin, which lands on heads with some unknown
probability
–
–
• Javier toss the coin times
–

Does tell us anything about ?


What can we learn from ?
Given enough tosses (large ), we expect that

Law of large numbers

as

Clearly, at least in a very limited sense, we can learn something about from
observations

There is always the possibility that we are totally wrong, but given enough data,
the probability should be very small

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy