Scribd Logo
Upload

Welcome to Scribd!

  • 26 views

    Machine Learning Questions

    Uploaded by

    Priyaprasad Panda
    This summarizes key machine learning concepts in 3 sentences or less: Back propagation, batch normalization, and cross entropy are techniques for training neural networks by propagating errors backwards and reducing internal covariate shift between layers. Common neural network components include activation functions like ReLU and sigmoid, objective functions to minimize loss, and optimizers that use the learning rate to update weights via the gradient. Recurrent models employ LSTM/GRU units to process sequential data by maintaining state between time steps and addressing the vanishing gradient problem.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Machine Learning Questions

    Uploaded by

    Priyaprasad Panda
    26 views2 pages
    This summarizes key machine learning concepts in 3 sentences or less: Back propagation, batch normalization, and cross entropy are techniques for training neural networks by propagating errors backwards and reducing internal covariate shift between layers. Common neural network components include activation functions like ReLU and sigmoid, objective functions to minimize loss, and optimizers that use the learning rate to update weights via the gradient. Recurrent models employ LSTM/GRU units to process sequential data by maintaining state between time steps and addressing the vanishing gradient problem.

    Original Description:

    int

    Original Title

    int9

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This summarizes key machine learning concepts in 3 sentences or less: Back propagation, batch normalization, and cross entropy are techniques for training neural networks by propagating errors backwards and reducing internal covariate shift between layers. Common neural network components include activation functions like ReLU and sigmoid, objective functions to minimize loss, and optimizers that use the learning rate to update weights via the gradient. Recurrent models employ LSTM/GRU units to process sequential data by maintaining state between time steps and addressing the vanishing gradient problem.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    26 views2 pages

    Machine Learning Questions

    Uploaded by

    Priyaprasad Panda
    This summarizes key machine learning concepts in 3 sentences or less: Back propagation, batch normalization, and cross entropy are techniques for training neural networks by propagating errors backwards and reducing internal covariate shift between layers. Common neural network components include activation functions like ReLU and sigmoid, objective functions to minimize loss, and optimizers that use the learning rate to update weights via the gradient. Recurrent models employ LSTM/GRU units to process sequential data by maintaining state between time steps and addressing the vanishing gradient problem.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 2

    Machine Learning questions

    Study online at quizlet.com/_4mb6kl

    1. Back Also known as back prop, this is the process 5. F1/F Score A measure of how accurate a model is by
    Propagation of back tracking errors through the weights of using precision and recall following a
    the network after forward propagating inputs formula of:
    through the network. This is used by applying
    the chain rule in calculus F1 = 2 (Precision Recall) / (Precision + Recall)
    Precise: of every prediction which ones are
    2. Batch [1] When networks have many deep layers
    actually positive?
    Normalization there becomes an issue of internal covariate
    shift. The shift is "the change in the
    Precision = True Positives / (True Positives +
    distribution of network activations due to the
    False Positives)
    change in network parameters during
    Recall: of all that actually have positive
    training." (Szegedy). If we can reduce internal
    predicitions what fraction actually were
    covariate shift we can train faster and better.
    positive?
    Batch Normalization solves this problem by
    normalizing each batch into the network by
    Recall = True Positives / (True Positives +
    both mean and variance.
    False Negatives)
    3. Cross Used to calculate how far off your label
    6. Gradient The gradient is the partial derivative of a
    Entropy prediction is. Some times denoted by CE.
    (Nabla) function that takes in multiple vectors and
    outputs a single value (i.e. our cost functions
    in Neural Networks). The gradient tells us
    Cross entropy is a loss function is related to
    which direction to go on the graph to
    the entropy of thermodynamics concept of
    increase our output if we increase our
    entropy. This is used in multi class
    variable input. We use the gradient and go in
    classification to find the error in the
    the opposite direction since we want to
    predicition.
    decrease our loss.
    4. Drop out
    7. L1 and L2 These regularization methods prevent
    Regularization overfitting by imposing a penalty on the
    coefficients. L1 can yield sparse models while
    L2 cannot. Regularization is used to specify
    model complexity. This is important because
    it allows your model to generalize better and
    not overfit to the training data.
    [1] "It prevents overfitting and provides a way
    of approximately combining exponentially 8. Learning Rate The learning rate is the magnitude at which
    many different neural network architectures [2] you're adjusting your weights of the network
    efficiently"(Hinton). This method randomly during optimization after back propagation.
    picks visible and hidden units to drop from The learning rate is a hyper parameter that
    the network. This is usually determined by will be different for a variety of problems.
    picking a layer percentage dropout. This should be cross validated on.

    # ---- Vanilla Gradient Descent ----


    # W is your matrix of weights
    # dW is the gradient of W
    # Adjust the weights of our layer by
    multiplying our gradient times our
    learning_rate (Which is a scalar value)
    W -= learning_rate * dW
    9. LSTM/GRU 13. Softmax Softmax is a function usually used at the end of a
    Neural Network for classification. This function
    does a multinomial logistic regression and is
    generally used for multi class classification. Usually
    paired with cross entropy as the loss function.
    14. Tanh Tanh is a function used to initialize the weights of
    Typically found in Recurrent Neural Networks your network of [-1, 1]. Assuming your data is
    but are expanding to use in others these are normalized we will have stronger gradients: since
    little "memory units" that keep state between data is centered around 0, the derivatives are
    inputs for training and help solve the vanishing higher. To see this, calculate the derivative of the
    gradient problem where after around 7 time tanh function and notice that input values are in the
    steps an RNN loses context of the input prior. range [0,1].The range of the tanh function is [-1,1]
    10. Objective Also known as loss function, cost function or and that of the sigmoid function is [0,1]. This also
    Functions opimization score function. The goal of a avoids bias in the gradients.
    network is to minimize the loss to maximize the
    accuracy of the network.

    A more technical explanation [3]:

    The loss/cost/optimization/objective function is


    the function that is computed on the prediction
    of your network. Your network will output a
    prediction, y_hat which we will compare to the
    desired output of y. This function then is used in
    back propagation to give us our gradient to
    allow our network to be optimized. Examples of
    these functions are f1/f score, categorical cross
    entropy, mean squared error, mean absolute
    error, hinge loss... etc.
    11. Rectified
    Linear
    Units or
    ReLU

    The sigmoid function has an interval of [0,1],


    while the ReLU has a range from [0, infinity]. This
    means that the sigmoid is better for logistic
    regression and the ReLU is better at
    representing positive numbers. The ReLU do not
    suffer from the vanishing gradient problem.
    12. Sigmoid σ

    A function used to activate weights in our


    network in the interval of [0, 1]. This function
    graphed out looks like an 'S' which is where this
    function gets is name, the s is sigma in greek.
    Also known as the logistic function

    You might also like

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy