Scribd Logo
Upload

Welcome to Scribd!

  • 133 views

    COMP3308/3608 Artificial Intelligence Week 9 Tutorial Exercises Multilayer Neural Networks 2. Deep Learning

    Uploaded by

    hariet
    This document provides exercises for a tutorial on multilayer neural networks and deep learning. It includes 9 exercises covering topics like: - Deriving and writing the weight update rule for the hyperbolic tangent sigmoid activation function - Calculating the number of weights in a sample neural network - Factors that influence the performance of the backpropagation algorithm, like network architecture - The effect of learning rate on convergence in backpropagation - The advantage of using momentum in backpropagation - How too many hidden neurons can lead to poor generalization - An example of convolution in deep neural networks - Working through a detailed backpropagation example to understand the forward and backward passes

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    COMP3308/3608 Artificial Intelligence Week 9 Tutorial Exercises Multilayer Neural Networks 2. Deep Learning

    Uploaded by

    hariet
    133 views2 pages
    This document provides exercises for a tutorial on multilayer neural networks and deep learning. It includes 9 exercises covering topics like: - Deriving and writing the weight update rule for the hyperbolic tangent sigmoid activation function - Calculating the number of weights in a sample neural network - Factors that influence the performance of the backpropagation algorithm, like network architecture - The effect of learning rate on convergence in backpropagation - The advantage of using momentum in backpropagation - How too many hidden neurons can lead to poor generalization - An example of convolution in deep neural networks - Working through a detailed backpropagation example to understand the forward and backward passes

    Original Title

    w9

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides exercises for a tutorial on multilayer neural networks and deep learning. It includes 9 exercises covering topics like: - Deriving and writing the weight update rule for the hyperbolic tangent sigmoid activation function - Calculating the number of weights in a sample neural network - Factors that influence the performance of the backpropagation algorithm, like network architecture - The effect of learning rate on convergence in backpropagation - The advantage of using momentum in backpropagation - How too many hidden neurons can lead to poor generalization - An example of convolution in deep neural networks - Working through a detailed backpropagation example to understand the forward and backward passes

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    133 views2 pages

    COMP3308/3608 Artificial Intelligence Week 9 Tutorial Exercises Multilayer Neural Networks 2. Deep Learning

    Uploaded by

    hariet
    This document provides exercises for a tutorial on multilayer neural networks and deep learning. It includes 9 exercises covering topics like: - Deriving and writing the weight update rule for the hyperbolic tangent sigmoid activation function - Calculating the number of weights in a sample neural network - Factors that influence the performance of the backpropagation algorithm, like network architecture - The effect of learning rate on convergence in backpropagation - The advantage of using momentum in backpropagation - How too many hidden neurons can lead to poor generalization - An example of convolution in deep neural networks - Working through a detailed backpropagation example to understand the forward and backward passes

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 2

    COMP3308/3608 Artificial Intelligence, s1 2021

    COMP3308/3608 Artificial Intelligence

    Week 9 Tutorial exercises


    Multilayer Neural Networks 2. Deep Learning.

    Exercise 1. Backpropagation (Homework)


    e n − e −n
    a=
    a) Show that the derivative of the hyperbolic tangent sigmoid e n + e −n

    is 1-a2.

    b) Write the weight change rule for this function, using the result from a). See slide 26 from the lecture on
    backpropagation.

    for an output neuron i:

    w ji =

    for a hidden neuron j:

    wkj =

    Exercise 2.
    a) A 2-layer feed-forward neural network with 10 input units, 5 hidden units and 3 output units contains
    how many weights? (Include biases.) Show your work.

    b) The backpropagation algorithm iterates until it minimizes what?

    c) Is the backpropagation algorithm guaranteed to achieve 100% correct classification for any linearly-
    separable set of training examples, given a sufficiently small learning rate? Explain briefly.

    Exercise 3. Cybenko’s theorem (slide 39) states that any continuous function can be approximated by a
    backpropagation network with 1 hidden layer. Why do we use networks with more than 1 hidden layer?

    Exercise 4. Using Weka


    1. Load the iris data (iris.arff). Choose 10-fold cross validation. Run the Naïve Bayes and Multilayer
    percepton (trained with the backpropagation algorithm) classifiers with the default parameters and compare
    their performance. Note that multilayer perceptron in Weka is under “Functions”.

    Which classifier was more accurate? Which one was faster to train?

    2. Select “MultilayerPerceptron” as a classifier. In the pane “Choose” (where “MultilayerPerceptron”


    should appear click to open the options window. If you click “More” you can learn what each of them
    means. What does “a” in the number of hidden layers mean? Experiment with 1 hidden layer and different
    number of neurons in it. As you can see the classification performance of the backpropagation network is
    very sensitive to the architecture. Choose “GUI=true” from the window with the options to visualize the
    different networks.
    COMP3308/3608 Artificial Intelligence, s1 2021

    Exercises using Matlab and Matlab’s Neural Network demos

    Exercise 5. Convergence with different learning rates

    In Matlab, type nnd12sd2 to run the demo.

    Try different starting positions and different learning rates. What is the influence of the learning rate? What
    happens if the learning rate is too big or too small?

    Exercise 6. Backpropagation with momentum

    In Matlab, select “Momentum Backpropagation” from Demos -> Toolboxes -> Neural Networks (upper
    left window)

    What is the advantage of using a momentum?

    Exercise 7. Generalization

    In Matlab, type nnd11gn or select “generalization” from Demos -> Toolboxes -> Neural Networks (upper
    left window)

    Try a simple function (i.e. choose a small difficulty index) with too many hidden neurons. What is the
    reason for the poor generalization?

    Exercise 8. Convolution in deep NNs

    Follow the convolution example on this website to see how the convolved features are computed:
    http://deeplearning.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/

    Exercise 9. Backpropagation example

    Follow the backpropagation walkthrough example prepared by Josh Stretton and available from Canvas; it
    is an extended version of the example from the first lecture on slides 33-37. The goal is to understand the
    forward and backward passes of the backpropagation algorithm and how the weights are updated.

    You might also like

    pFad - Phonifier reborn

    Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

    Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


    Alternative Proxies:

    Alternative Proxy

    pFad Proxy

    pFad v3 Proxy

    pFad v4 Proxy