Statement 1: How to get started in the world of DS/ML/AI

The best way to learn something is, really, a personal thing. The most obvious thing is that it
depends on what you already know. Moreover, you must consider what you want to learn. But,
the most important in my opinion is why you want to learn.

It seems to me that learning is trying to go somewhere. The destination is what you want to
learn. The starting point is your current knowledge. Now, think about why you want to learn. It
will eventually tell you how to learn it. And ultimately show you the way.

Discussion is a good way to unravel what you really want. You can discuss with people in the field
to get their feedback about it, but also close people that knows you well, or even discuss with
yourself by answering your own questions. Let’s see what may be interesting questions to
discuss about.

Set a (soft) goal

First, try to describe your goal, in simple words.

Do you want to learn AI in order to apply such techniques in your life (e.g. for projects, of any
kind)? Do you want to learn it as a career objective (e.g. getting AI related position)? Do you
want to study it (e.g. quite academical studies)? Or are you just curious about it, I mean, you just
want to understand how it works.

These goals are not mutually exclusive. I use the expression soft-goal, inspired by AI jargon, that
mean it’s not only one choice, it’s more like selecting multiple goals and assign it different
importance.

For example, I did start computer science mostly to understand how it works. Then, I wanted to
understand the foundation of it (i.e. theory). Then I found ML/AI magic, which wasn’t satisfying
=> go study it .Therefore, I transformed just curiosity into studies, and eventually job (I’m
discussing it in detail in Paul Tardy's answer to How did you venture into the AI field? What prior
knowledge did you have?)

Abstraction level

Then, think about which abstraction level you want to understand. Complex things has some
abstraction layers.

Think about music. What is music? Is it a melody? Is it just harmony between chords? Is it just
some set of notes? Or just frequencies? You can study music as a composer, an interpreter and
physicist or and audiophile, that’s just different angle to approach the concept.

Going back to AI/ML you also have different layers. The model architecture can be described
with figures, that shows differents boxes and link. You can understand a model, what makes it
interesting, what could be done different, without always deeply understanding the details.
Details are mostly maths. With definitions, equations, optimization problems etc. Then, there’s a
framework layers, that is, designing toolkits that makes it possible (and as simple as possible) to
implements models. One can then use those toolkits to implement new research paper. Or run
experiments. One can also use those toolkit with higher level interface i.e. train the model
without writing a line of code, just pass it a set of parameters.

Every levels has it interest and challenge. As for goals, think about which corresponds to you. You
may want to focus on one part, or on a few, or explore all of them, with or without order.

Re-iterate

Try to keep those questions in mind, and to question yourself on time to time. Goals are set but
not fixed. Learning is discovering, it’s exiting to learn things that you didn’t even know the
existence. Moreover, discoveries may change your objective, your point of view etc.

Get started

Depending on your previous conclusions. You need to pick a starting point. I know that it’s not
that simple, as always there’s different way. Let see some kind of typical learning “route”.

 Online courses: you will find good pointers in other answers. There’s a lot of really
good content from universities/MOOC that can provide you a valuable theoretical
background, if that’s what you need.
 University: Yes. Universities are good way of learning. It may be underrated because
sometime it’s less sexy than alternative approach, but if you can, that is a good way.
Had to mention it.
 Tutorials: can teach you theory without maths. For example, how this or that model is
working. What’s the intuitions behind it. What is it’s strength or weakness etc. Most AI
related frameworks comes with a set of tutorial, that include examples to implement
models using the framework. It’s a good way practically understand what’s going on.
Plus, you can experiment things to challenge your understanding.
 Project based: start a project, or just something you find interesting enough to be your
exercise, a find the right tools, the right information, in order to solve your problem.
It’s an entertaining way of exploring the field, since you may face problems at different
abstraction layer. You will struggle with the python module, the dataset pre-
processing, you will read some papers, watch a video about that model the guy is
talking about, etc. If you have no idea which project to start, let’s combo with the last
point: follow a tutorial, then try to extend it. To get better result, better performance,
to work on different kind of data, etc.
 Statement 2: Discuss a Machine Learning algorithm in detail.
Decision Tree - Classification

Decision tree builds classification or regression models in the form of a tree structure. It breaks
down a dataset into smaller and smaller subsets while at the same time an associated decision
tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes. A
decision node (e.g., Outlook) has two or more branches (e.g., Sunny, Overcast and Rainy). Leaf
node (e.g., Play) represents a classification or decision. The topmost decision node in a tree which
corresponds to the best predictor called root node. Decision trees can handle both categorical and
numerical data.

Algorithm
The core algorithm for building decision trees called ID3 by J. R. Quinlan which employs a top-
down, greedy search through the space of possible branches with no backtracking. ID3
uses Entropy and Information Gain to construct a decision tree. In ZeroR model there is no
predictor, in OneR model we try to find the single best predictor, naive Bayesian includes all
predictors using Bayes' rule and the independence assumptions between predictors but decision
tree includes all predictors with the dependence assumptions between predictors.

Entropy
A decision tree is built top-down from a root node and involves partitioning the data into subsets
that contain instances with similar values (homogenous). ID3 algorithm uses entropy to calculate
the homogeneity of a sample. If the sample is completely homogeneous the entropy is zero and if
the sample is an equally divided it has entropy of one.
To build a decision tree, we need to calculate two types of entropy using frequency tables as
follows:

a) Entropy using the frequency table of one attribute:

b) Entropy using the frequency table of two attributes:

Information Gain
The information gain is based on the decrease in entropy after a dataset is split on an attribute. Constructing a d
attribute that returns the highest information gain (i.e., the most homogeneous branches).

Step 1: Calculate entropy of the target.

Step 2: The dataset is then split on the different attributes. The entropy for each branch is calculated. Then it i
total entropy for the split. The resulting entropy is subtracted from the entropy before the split. The result is the
in entropy.

Step 3: Choose attribute with the largest information gain as the decision node, divide the dataset
by its branches and repeat the same process on every branch.
Step 4a: A branch with entropy of 0 is a leaf node.
Step 4b: A branch with entropy more than 0 needs further splitting.

Step 5: The ID3 algorithm is run recursively on the non-leaf branches, until all data is classified.
Decision Tree to Decision Rules
A decision tree can easily be transformed to a set of rules by mapping from the root node to the
leaf nodes one by one.