Advanced Mathematics for Engineers Wolfgang Ertel translated by Elias Drotleff and Richard Cubek January 8, 2014 ih Hochschule Ravensburg-Weingarten lUniversity of Applied SciencesPreface Since 2008 this mathematics lecture is offered for the master courses computer science, mecha- tronics and electrical engineering. After a repetition of basic linear algebra, computer algebra and calculus, we will treat numerical calculus, statisties and function approximation, which are the most important basic mathematies topics for engincers. We also provide an introduction to Computer Algebra. Mathematica, Matlab and Octave are powerful tools for the Exercises. Event though we favour the open source tool Octave, the student is free to choose either one of the three. ‘We are looking forward to work with motivated and eager students who want to climb up the steep, hhigh and fascinating mountain of engineering mathematics together with us. I assure you that we ‘will do our best to guide you through the sometimes wild, ough and challenging world of applied mathematics. I also assure you that all your efforts and your endurance in working on the exercises during nights and weekends will pay off as good marks and most importantly as a lot of fun, Even though we repeat some undergraduate linear algebra and calculus, the failure rate in the exams is very high, im particular among the foreign students. As a consequence, we strongly recommend all our students to repeat undergraduate linear algebra such as operation on matrices like solution of linear systems, singularity of matrices, inversion, eigenvalue problems, row-, columm- and nullspaces. You also should bring decent knowledge of one-dimensional and multidimensional calculus, e.g, differentiation and integration in one and many variables, convergence of sequences and series and finding extrema with constraints of multivariate functions. And basic statistics is also required, ‘To summarize: If you are not able to solve problems (not only know the terms) in these fields before you start the course, you have very little chances to successfully finish this course. History of the Course The first version of this script covering mumerics was created in the winter semester 95/96 for computer science students only, It covered the basics of numerical calculus, systems of linear ‘equations, various interpolation methods, ftmetion approximation, and the solution of nonlinear equations. Im summer 1998 a chapter about Statistics was added, because of the weak coverage at our Univer- sity till then, In the winter semester 1999/2000, the layout and structure were improved and some mistakes removed. In the context of changes in the curriculum of Applied Computer science in the summer semester 2002, statistics was shifted, because of the general relevance forall students, into the lecture Math- ‘ematics 2. Instead of Statistics, subjects specifically relevant for computer scientists should be included. The generation and verification of random mumbers is such a topie Since summer 2008, this lecture is offered to Master (Computer Science) students. ‘Therefore the chapter about random numbers was extended. ‘To the winter semester 2010/11 the lecture has beon completely revised, restructured and some important sections added stich as radial basis functions and statistics and probability: These changes. Doecome necessary with the step from Diploma to Master. I want to thank Markus Schneider and Haitham Bou Ammar who helped me improve the lecture. To the winter semester 2010/11 the precourse will be integrated in the Iecture in order to give the students more time to work on the exercises. Thus, the volume of lecture grew from 6 SWS to 8 SWS and was split it into two lectures of 4 SWS each, In the winter semester 2012/13 we switched back to a one semester schedule with 6 hours per week for computer science and mechatronics students. Electrical engineering students will go for four hours, covering chapters one to six only. Wolfgang ExtelContents 1 Linear Algebra 1.1 Video Lectures 12 Exercises 2 Computer Algebra 2.1 Symbol Processing on the Computer 2.2 Gnuplot, a professional Plotting Software 2.3 GNU Octave 24 Exercises 3 Calculus ~ Selected Topics 3.1 Sequences and Convergence 3.2. Series 3.3. Continuity 3.4 Taylor-Series 3.5 Differential Calculus in many Variables 3.6 Exercises 4 Statistics and Probability Basics 4.1 Recording Measurements in Samples 4.2. Statistical Parameters 4.3 Multidimensional Samples 4.4 Probability Theory 4.5 Discrete Distributions 4.6 Continuous Distributions AT Exercises | Mathematics Fundamentals 5.1 Arithmeties on the Computer 5 Numeri 5.2. Numerics of Linear Systems of Equations 5.3 Roots of Nonlinear Equations 5.4. Exercises 6 Function Approximation 6.1 Polynomial Interpolation 6.2. Spline interpolation 6.3 Method of Least Squares and Pseudoinverse 6.4 Exercises we 23 23, 26 29 34 38 62 62 64 65 69 73 7 80 82 82. 86 104 107 107 2 1s 1312 CONTENTS 7 Statistics and Probability 134 7.1 Random Numbers 134 7.2 Exercises 139 7.3. Principal Component Analysis (PCA) ul 74 Estimators ur 7.5 Gaussian Distributions 150 7.6 Exercises 153 8 Function Approximation 155 8.1 Linear Regression - Summary 155 8.2 Radial Basis Function Networks 156 8.3. Singular Value Decomposition and the Pseudo-Inverse 163 8.4 Exercises 168 9 Numerical Integration and Solution of Ordinary Differential Equations 170 9.1 Numerical Integration 170 9.2 Numerical Differentiation 17 9.3 Numerical Solution of Ordinary Differential Equations 77 9.4 Linear Differential Equations with Constant Coefficients 183 9.5 Exercises 191Chapter 1 Linear Algebra 11 ‘Wo use the excellent video lectures from G. Strang, Video Lectures author of 7], available from: http: // ocw. mit, edu/courses/nathematics/18-06-1inear-algebra~spring-2010. In particular we show the following lectures: uetry of linear equations (lecture 01) 2° | Elimination with matrices (lecture 02) 3 | Multiplication and inverse matrices (lecture 03) 4 | Transposes, Permutations, Spaces R" (lecture 05) 5 | Column Space and Nullspace (lecture 06) 6 | Solving Ax = 0: Pivot Variables, Special Solutions (lecture 07) 7 | Solving Ax = b: Row Reduced Form R (lecture 08) 8 | Independence, Basis, and Dimension (lecture 09) 9 | Properties of Determinants (lecture 18) 10 | Determinant Formulas and Cofactors (lecture 19) 11 | Cramer's rule, inverse matrix, and volume (lecture 20, only the part “inverse matrix’) 12 | Eigenvalues and Higenvectors (lecture 21) 13 | Symmetric Matrices and Positive Definiteness (lecture 25) 14 _| Linear Transformations and Their Matrices (lecture 30) 1.2 Exercises Exer 1.1 Solve the nonsingular triangular system utvtwah aay b (2) bs (2.3)4 1 Linear Algebra Show that your solution gives a combination of the columns that equals the column on the right: Exercise 1.2 Explain why the system utvtw= (14) ut 2v+3w=1 (28) vt2u=0 (2.6) js singular, by finding a combination of the three equations that adds up to 0 = 1. What value should replace the last zero on the right side, to allow the equations to have solutions, and what is one of the solutions? Invers: and Transposes Exercise 1.3 Which properties of a matrix A are preserved by its inverse (assuming A! exists)? Ais triangular Ais symmetric ) }) A is tridiagonal ) all entries are integers ) all entries are rationals Exercise 1.4 a) How many entries can be chosen independently, in a symmetric matrix of order n? b) How many entries can be chosen independently, in a skew-symmetric matrix of order n? Permutations and Elimination Exercise 1.5 a) Find a square 3x 3 matrix P, that multiplied from left to any 3 xm matrix A exchanges rows 1 and 2 b) Find a square n xn matrix P, that multiplied from left. to any n xm matrix A exchanges rows i and j Exercise 1.6 A permutation is a bijective mapping from a finite sot onto itself Applied to vectors of length n, a permutation arbitrarily changes the order of the vector compo- nents. The word “ANGSTBUDE” is a permutation of “BUNDESTAG”. An example of a permutation on vectors of length 5 can be described by (3,2,1,5,4) This means component 3 moves to position 1, component 2 stays where it was, component 1 moves to position 3, component 5 moves to position 4 and component 4 moves to position a) Give a 5 x 5 matrix P that implements this permutation b) How can we come from a permutation matrix to its inverse’1.2. Exercises 5 Exercise 1.7 a) Find a 3x3 matrix £, that multiplied from left to any 3 xm matrix A adds 5 times row 2 to row 1 b) Describe an xn matrix E, that multiplied from left to any nx on matrix A adds k times row i to row j ©) Based on the above answers, prove that the elimination of a matrix ean be realized by successive multiplication with matrices from left. Column Spaces and NullSpaces Exercise 1.8 Which of the following subsets of R® are actually subspaces? a) The plane of vectors with first component b; = 0. b) The plane of vectors 6 with by ~ 1 c) The vectors b with bb = 0 (this is the union of two subspaces, the plane 6; = 0 and the plane by = 0). d) The solitary vector b = (0,0, 0) ) All combinations of two given vectors ar = (1,1, 0) and y = (2,0,1) £) The vectors (bi, ba, bs) that satisfy bs — by + 3b, — 0. Exercise 1.9 Let P be the plane in 3-space with equation x ~ 2y + 2 = 6. What is the equation of the plane Py through the origin parallel to P? Are P and Pp subspaces of R®? Exercise 1.10 Which descriptions are correct? The solutions x of 11 4qi% 0 7 af f)- an form a plane, line, point, subspace, nullspace of A, column space of A. Ag = 0 and Pivot Variables Exercise 1.11 For the matrix apisg on determine the to Ar =0. Then apply elimination to Ar = 6, with components b; and by on the right side; find the conditions for Ax = b to be consistent (that is, to have a solution) and find the general solution. What is the rank of A? helon form U, the pivot variables, the free variables, and the general solution Exercise 1.12 Write the general solution to 2af]-t es6 1 Linear Algebra as the sum of a particular solution to Az = 6 and the general solution to Ax = 0. Exercise 1.13 Find the value of ¢ which makes it possible to solve ututaw=2 (1.10) 2u+3v— w= (11) Buddy tw = (1.12) Solving Ax = b Exercise 1.14 Is it true that if vs, vz, us are linearly independent, that also the vectors wy = vy + v2, w2 = vy + Us, ws = v2 + vy are linearly independent? (Hint: Assume some combination cw, + czivz + ests = 0, and find which c; are possible.) Exercise 1.15 Find a counterexample to the following statement: If vy, v2, v3, v4 is a basis, for the vector space IR!, and if W is a subspace, then some subsct of the v's is a basis for w Exercise 1.16 Suppose V is known to have dimension k. Prove that, a) any k independent vectors in V form a basis, b) any k vectors that span V form a basis. In other words, if the number of vectors is known to be right, either of the two properties of a basis implies the other. Exercise 1.17 Prove that if V and W are three-dimensional subspaces of R°, then V and W must have a nonzero vector in common. Hint: Start with bases of the two subspaces, making six vectors in all. The Four Fundamental Subspaces Exercise 1.18 Find the dimension and construct a basis for the four subspaces associated with each of the matrices 4-forae o*|0 28 0 0 140 , Joo food (1.13) Exercise 1.19 If the product of two matrices is the zero matrix, AB = 0, show that the column space of B is contained in the nullspace of A. (Also the row space of A is the left nullspace of 2, since each row of A multiplies B to give a zero row.) Exercise 1.20 Explain why Ar — 6 is solvable if and only if rank A — rank A’, where A” is formed from A by adding 6 as an extra column. Hint: The rank is the dimension of the column space; when does adding an extra column leave the dimension unchanged? Exercise 1.21 Suppose A is an m by n matrix of rank r. Under what conditions on those numbers does a) Ahave a two-sided inverse: AA“? = A“'A = 1? b) Av =b have infinitely many solutions for every 0? Exe some right, ise 1.22 If Ar = 0 has a nonzero solution, show that ATy = f fails to be solvable for des f. Construct an example of A and f.1.2. Exercises 7 Orthogonality Exercise 1.23 In R® find all vectors that are orthogonal to (1, 1, 1) and (1, -1, 0). Produce from these vectors and (1, 1, 1) and (1, -1, 0) a mutually orthogonal system of unit vectors (an orthogonal system) in R® Exercise 1.24 Show that «r— y is orthogonal to x +y if and only if |x\| = |lyl Exercise 1.25 Let P be the plane (not a subspace) in 3-space with equation x | 2y 2 Find the equation of a plane P’ parallel to P but going through the origin. Find also a vector perpendicular to those planes. What matrix has the plane P’ as its nullspace, and. what matrix hast P’ as its row space? Projections Exercise 1.26 Suppose A is the 4 x 4 identity matrix with its last column removed. A is 4x 3. Project b = (1,2,3,4) onto the column space of A, What shape is the projection matrix P and what is P? Determinants Exercise 1.27 How aro d A), and det(A2) related to det A, when A is n by Exercise 1.28 Find the determinants of a) a rank one matrix [2 -1 2 (1.14) b) the upper triangular matrix: 4488 ,_{o122 nas U=|So26 (1.15) 0002 ©) the lower triangular matrix U7, ) the inverse matrix U-; ) the “reverse-triangular” matrix that results from row exchanges, (1.16) Exercise 1.29 If every row of A adds to zero prove that det A =0. If eve prove that det(4 — J) = 0. Show by example that this d vy row adds to 1 es not imply det A= 18 1 Linear Algebra Properties of Determinants Exercise 1.30 Suppose A, is the n by n tridiagonal matrix with 1's everywhere on the three diagonals: 110 A,=[l a=[i ): As- [11a], (1.17) oii Let D, be the determinant of A,; we want to find it. a) Expand in cofactors along the first row of A, to show that Dy = Dy1— Daa b) Starting from D; = 1 and D, = 0 find Ds, Ds,..., Ds. By noticing how these numbers cycle around (with what period?) find D; Exercise 1.31 Explain why a 5 by 5 matrix wi a3 by 3 zero submatrix is sure to be a singular (regardless of the 16 nonzeros marked by 2's): beret bene e the determinant of A= |0 0 0 a =| is zero. (1.18) 000ne 000ne Exercise 1.32 If A is m by n and B is n by m, show that o A [r 9] a nt: Postmultiply by (1.19) a= |, ‘| = det AB (Hise Postmultiply by |] ) (1.19) Do an example with m n, Why does the second example have det AB = 0? Cramers’ rule Exercise 1.33 The determinant is a linear fumction of the column 1. It is zero if two columns are equal. When b= Ax = 21a; +2242 +734 goes into the first column of A, then the determinant of this matrix By is Ib a2 as|=|rray + z2az +303 a2 as] = xia, a2 | = xydetA a) What formula for 27, comes from left side = right side? b) What steps lead to the middle equation? Eigenvalues and Eigenvectors Exercise 1.34 Suppose that is an eigenvalue of A, and x is its eigenvector: Ax = Ax a) Show that this same x is an eigenvector of B = A —7I, and find the eigenvalue. b) Assuming \ #0, show that x is also an eigenvector of A~! and find the eigenvalue.1.2. Exercises 9 Exercise 1.35. Show that the determinant equals the product of the eigenvalues by imagining that the characteristic polynomial is factored into det(A — AZ) = (1 = A)(A2 = A) On = A) (1.20) and making a clever choice of Exercise 1.36 Show that the trace of a square matrix equals the sum of its eigenvalues. Do this in two steps. First, find the coefficient of (—A)"-! on the right side of equation 1.20. Next, look for all the terms in det(A =r) = det | SP a (1.21) Ont On Ayn — which involve (—)"-', Explain why they all come from the product down the main diagonal, and find the coefficient of (—)"-' on the left side of equation 1.20, Compare, Diagonalization of Matrices Exercise 1.37 Factor the following matrices into SAS™ 11 20 / =f] oe a= [2 am) Exercise 1.38 Suppose A = uv" is a column times a row (a rank-one matrix). a) By multiplying A times u show that w is an eigenvector. What is X? b) What are the other eigenvalues (and why)? 7, ¢) Compute trace(A) = vu in two ways, from the sum on the diagonal and the sum of 2's, SAS is the Exercise 1.39 If A is diagonalizable, show that the determinant of A = product of the eigenvalues Symmetric and Positive Semi-Definite Matrices Exercise 1.40 If A = QAQ™ is symmetric positive definite, then R = QVAQT is its symmetric positive definite square root. Why does R have real eigenvalues? Compute R and verify 21 ww 6 1193 a-[f 3] me a= [9 99 am Exercise 1.41 If A is symmetric positive definite and Cis nonsingular, prove that B = CTAC is also symmetric positive definite Exercise 1.42 If A is positive definite and ay, is increased, prove from cofactors that the determinant is increased. Show by example that this can fail if A is indefinite10 1 Linear Algebra Linear Transformation Exercise 1.43 Suppose a linear mapping T’ transforms (1, 1) to (2, 2) and (2, 0) to (0, 0) Find T(v}: (@) v=22) @) v=@1) | v=(-4) @ v=(ab) Exercise 1.44 Suppose Tis reflection across the 45° line, and Sis reflection across the y axis, If'v = (2,1) then T(v) = (1,2). Find S(7(v)) and T(S(w)). This shows that generally ST TS. Exercise 1.45 Suppose we have two bases v4,..,U, and wy,..., wy for R". If a vector has coefficients 6, in one basis and c; in the other basis, what is the change of basis matrix in Me? Start from DyvL bs + Ute = VO = cy +. + Gxt = We (1.24) ‘Your answer represents T(v) = v with input basis of v's and output basis of w’s. Because of different bases, the matrix is not I Exercise 1.46 In many applications it is crucial that a linear system Az = 6 with a square matrix A has a unique solution. Give six conditions on A that are equivalent to Ar = 6 having a unique solution.Chapter 2 Computer Algebra | Definition 2.1 Computer Algebra = Symbol Processing + Numerics + Graphics Definition 2.2 Symbol Processing is calculating with symbols (variables, constants, function symbols), as in Mathematics lectures Symbol Processing Example 2.1 1. symbolic: lim ( n=) =? (asymptotic behavior) (25) zt 2. numeric: Numerics Example 2.2 Numerical solution of x? = 5 Qn = 24 iteration:12 2 Computer Algebra n tn Oe initial value 1 2.25 => V5 = 2.23606798 + 10-* 2 2.236111 ) ‘approximate solution) 3 — 2.23606798 (pp ’ 2.23606798 Advantages of Symbol Processing: sn considerably less computational effort compared to numerics © symbolic results (for further calculations), proofs in the trict manner possib Disadvantages of Symbol Processing: © often there is no symbolic (closed form) solution, then Numerics will be applied, eg = Calculation of Integrals — Solving Nonlinear Equations like: (e* = sinx) 2.1 Symbol Processing on the Computer Example 2.3 Symbolic Computing with natural numbers: Calculation rules, i.e. Axioms necessary. => Peano Axioms e.g, Voyzinty = yt (21) et0=@ (22) (ety)+2 = r++.) (2.3) Out of these rules, e.g. 0-+2 = x can be deduced: Otro x —_ (a) Symbol Processing by Term Rewriting Example 2.4 Chain Rule for Differentiation: [f(g(s))!' + F'(ale) Application: sin(In +2)’ = cos(Inr + 22 Computer: (Pattern matching) sin(Plus(In x, 2))’ = cos(Plus(In x, 2)) - Plus'(In x, 2) cos(Plus(In-r,2)) - Plus(In’«,2') = cos(Plus(In.x, 2)) - Plus (1/2, 0) = cos(Phus(In-r,2)) «Ife cos(Inz + 2) aa2.2 Gnuplot, a professional Plotting Software 13 Computeralgebra Systems @ Mathematica (S. Wolfram & Co.) japle (ETH Zurich + Univ. Waterloo, Kanada) © Octave / Matlab: numerics and graphics only, no symbol processing 2.2 Gnuplot, a professional Plotting Software Gnuplot is a powerful plotting programm with a command line interface and a batch inter- face. Online documentation can be found on http: //sww. gnuplot. info On the command line we can input a plot [0:10] sin(x) Z \ to obtain the graph Almost arbitrary customization of plots is possible via the batch interface. A simple bateh le may contain the lines set terminal postscript eps color enhanced 26 set label "{/Symbol a}=0.01, {/Symbol g}=5" at 0.5,2.2 set output "bucket3.eps" plot [b=0.01:1] a-0.01, c= 5, (a-b-c)/(Log(a) - log(b)) \ title "({/Symbol a}-{/Symbol b}-{/Symbol g})/(1n{/Symbol a} - 1n{/Symbol b})" . TePana For producing a EPS file with the graph ‘ 3-dimensional plotting is also possible, e.g. with the commands set isosamples 50 splot (-pi:pil [-piipi] sin((xe#2 + ys#3) / Geex2 + ye¥2)) which produces the graph14 2 Computer Algebra 2.3 GNU Octave From the Octave homepage: GNU Octave is a high-level interpreted language, primar- ily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and for performing other numerical ex- periments, It also provides extensive graphics capabilities for data visualization and manipulation. Octave is normally used through its interactive command line interface, but it can also be used to write non-interactive programs, The Octave language is quite similar to Matlab so that most programs are easily portable. Downloads, Does, FAQ, ete.: http: //wuw. gnu.org/software/octave/ Nice Introduction/Overview: vp: //nath. jacobs-university .de/oliver/teaching/ iub/resources/octave/octave-intro/octave-intro. html Plotting in Octave: http://www. gnu. org/software/octave/doc/interpreter/Plotting html, Uf > comments Basrcs ectave:d7> 141 ans = 2, octave:d8> x= 243 x= 6 11 suppress output octaverd9> x= 243; octave 50> Uf rede ectave:53> help sin ‘sin’ is a built-in function == Mapping Function: sin 00 Conpute the sine for each element of X in radians. VECTORS AND MATRICES Uf define 2x2 matrix ectave:t> A= [1 2; 34) re 12 34 If define 3x3 natrix ectave:s> A= (123; 456; 789] he2.3 GNU Octave 15 1203 45 6 7 8 8 // accoss single elenen’ octave:4> x = AC2,1) x24 ectave:17> A(3,3) = 17 rv 3 6 7 Uf extract submatrices octave:®> a. rs 12 5 6 7 a7 ectave:9> B= A(1:2,2:3) Be 3 5 6 octave:36> beACt:3,2) be U1 wranspose ectave:25> A? 1407 25 8 306 7 Uf determinant octave:26> det (A) ane = -24,000 UL solve hx U/ inverse ectave:22> anv(A) “1.54167 0.41667 0.12500 1.08333 0.16667 -0.25000 (0.12800 -0.25000 0.12500 I define vector »16 2 Computer Algebra ectave:27> b= (37 12)’ on 3 7 12 Uf solution x ectave:29> x = inv(h) * b 0.20833 1.41667 0.12500 octave:30> A+ x 3.0000 7.0000 12.0000 UL ery No U1 siegal operation octave:31> x +b error: operator *: nonconformant arguments (opl is 3x1, op2 is 3x1) Uf therefore alloved octave:d1> x) * b ane = 10,792 ectave:32> x # BF “0.62500 -1.48833 -2.50000 4.25000 9.91687 17.0000 0.37500 0.87500 1.50000 1 elenentwise operations octaverti> a= (123) 123 octave:10> b= [4 6 6] » 45 6 ectave:12> ax error: operator *: nonconformant argunents (op is 1x3, op2 is 1x3) octave:12> ab 4 10 18 ectave:23> A= [12:3 4] Ae 12 42.3 GNU Octave 7 octave:24> 472 7 10 1 2 octave:25> A.-2 4 a 16 I/ exeate special vectors/natrices ectave:52> x = [0:1:5] o 12345 octave:83> A = zeres(2) rs oo oo ectave:s4> A = zeroa(2,3) he 0 0 0 0 0 0 octave:55> A = ones(?2,5) he ta pad octave:s6> A = aye(4) Ae Diagonal 1 0 0 0 o 10 0 0 0 1 0 oo 0 t octave:s7> Be A eS U/ vector/matrix size octave 43> size(a) 33 octave:44> size(b) a4 octave 45> size(b) (1)18 2_Computer Algebra ans = 3 PLOTTING (2D) ectave:35> x = [-2epi:0.1:2epi]; octave:36> y = sin(x) ectave:37> plot (x,y) ectave:38> 2 = coax); ectave:39> plot (x,2) // wwe curves in one plot octave:40> plot (x,y) octave:41> hold on octave:42> plot (x,2) Uf reset plots octave:S0> close all Uf plot different styl ectave:76> plot(x,2,’r") octave 77> plot(x,2, rx!) ectave:78> plot(x,z,"ge") octave:89> close all Uf wanipulate plot ectave:90> held on ectave:9i> x = [-pi:0.02:pil; Uf another Linesiath ectave:92> plot (x,sin(x),Linewidth’ ,2) ectave:93> plot (x,cos(x) ,’r!, linewidth’ ,2) // define axes range and aspect ratio ectave:94> axis((-pi,pi,-1,1], equal’) => tzy ‘square’ or ‘normal’ instead of ‘equal’ (help axis) 11 regend ectave:95> Legend(’sin’ , "cos"? // set parancters (gea = get current axis) octave:99> setgea, keypos? octave: 103> set (gen, 'xgxia? octave:104> set (gen, 'ygrid’ 2) // legend position (1-4) fon’) // show grid fon’) // show grid in y Uf vitle/labels ectave:102> title (OCTAVE DENG PLOT?) ectave:100> xlabel (unit circle’)2.3 GNU Octave 19 octave 101> ylabel(?trigon. functions’) I/ store as pag octave:105> print ~dpng *deno_plot.png” DEFINE FUNCTIONS signoid.m function $ = signeid(x) tn = size(X; 8 = zeros(an) ; for = Tinmn(1) for j = t:mn(2) SUS) 22 / Ate > KG: signoid(x) +e GM); octave:1> sig + (TAB) signoid signoid.= octave:1> signoia(10) ane = 0.99995 octave:2> signoia([1 10]) error: for x°A, A sust be square // (if not yet implemented elenentwise) error: called fron error: /hone/richaré/faculty/adv.math/octave/signoid.m at line 3, column 4 ectave:2> signoia((t 10]) 0.73108 0.99995 ectave:3> x = [-10:0.01:10]; octave:5> plot (x,signoid(x) , Lis20 2 Computer Algebra I woshgria octave:54> [X,Y] = meshgrid( [1:3], [1:3] 123 1203 1203 114 222 33 3 // weshgrid with higher resol ion (suppress outp) octave:15> [£,Y] = meshgrid([-4:0.2:4) ,[-4:0.2:4]); U/ function over x and y, remember that cos and sin U/ operate on each elenent, result is matrix again ectave:20> Z = cos(x) + sin(1.54¥); Wf prot octave:21> mesh (X,Y,2) octave:22> surs(X,Y,2) octave 44> contour (K,Y,2) octave 45> colerbar octave 46> peolor (X,¥,2) : : tcf aU fe24 Exercises 21 ‘RANDOM NUMBERS / IITSTOGRAMS // equally distributed random nusbers octave:4> xerand(2,5) 0.71696 0.95553 0.17808 0.82110 0.25843, ectave:5> x=rand(1, 1000) ; ectave:6> hist (x); // normally distributed random numbers octave:5> xerandn(1,1000); octave:6> hist (x); Wey ectave:s> xerandn(1,10000) ; octave:6> hist (x, 28); 7 300 1 100 250 © 02 0808 OBL 3220129 4 2.4 Exercises Programming with Octave, Matlab or Mathematica Exercise 2.1 Program the factorial function. a) Write an iterative program that calculates the formula n! =n -(n—1)-...-1 b) Write a recursive program that calculates the formula maf @ VD! ifn>1 ~ 1 ifm=1 a) Write a program that multiplies two arbitrary matrices. Don’t forget to check the di- mensions of the two matrices before multiplying. The formula is Cy > Ain Bes a Do not use built-in functions for matrix manipulation,22 2 Computer Algebra b) Write a program that computes the transpose of a matrix. Exercise 2.3 a) For a finite geometic series we have the formula Sf_gg' = 52". Write a function that takes q and n as inputs and returns the sum. 1 b) For an infinite geometic series we have the formula D%2,q! — if the series converges. Write a function that takes q as input and returns the sum. Your function should produce an error if the series diverges. Exercise 2.4 a) Create a 5 x 10 random Matrix A b) Compute the mean of each column and assign the results to elements of a vector called c) Compute the standard deviation of each column and assign the results to the of a vector called s. Exercise 2.5 Given the row vectors « = [4, 1, 6, 10, —4, 12, Ol] and y= [-1, 4, 3, 10, ~9, 15, ~24) compute the following arrays, a) ai; = 24 b) by; ©) ¢ = xiys, then add up the elements of ¢ using two different programming approaches. a) dy x e) Arrange the elements of and y in ascending order and of the less 1, and y, lculate e:; being the reciprocal £) Reverse the order of elements in x and y in one command. Exercise 2.6 Write a MATLAB function that recursively calculates the square root of a number without using built-in functions like sqrt() Analysis Repetition Exercise 2.7 In a bucket with capacity v there is a poisonous liquid with volume av. The bucket has to be cleaned by repeatedly diluting the liquid with a fixed amount (8 — a)v (0 < 8 < 1~a) of water and then emptying the bucket. After emptying, the bucket always keeps av of its liquid. Cleaning stops when the concentration cy of the poison after n iterations is reduced from 1 to c < € > 0. a) Assume a = 0.01, $ = 1 and ¢ = 10-°. Compute the number of cleaning-iterations. b) Compute the total volume of water required for cleaning ©) Can the total volume be reduced by reducing 8? Ifso, determine the optimal 8 d) Give a formula for the time required for cleaning the bucket. e) How can the time for cleaning the bucket be minimized?Chapter 3 Calculus — Selected Topics 3.1 Sequences and Convergence Consider the following sequences © 1,2,3,5,7,11,18,17,19,23, 1,3,6,10,15,21,28,36,45,55,66, © 1,1,2,3,5,8,13,21,34,55,89,, © 1,2,3,4,6,7,9,10,11,13,14,15,16,17,18,19,21,22,23,24,26,27, 29,30,31,32,33,34,35,36, 37, # 1,3,5,7,8,9,10,11,12,13,14,15,16,17,18 19,20,21,22,23,24,25, 26,27,28,29,31,33, 35,37,38,39,41 43, Find the next 5 elements of each sequence. If you do not get ahead or want to solve other riddles additionaly, have a look at http://www .oeis.org, Definition 3.1 A function N + R,n + a, is called sequence. Notation: (a,)ncx or (a1, aa, 35...) Example 3.1 ¢ (1,2,3,4,...) = (m)nex LE d do) = (nen © (1,2,4,8, 16, ..) = (2°) new24 3. Calculus ~ Selected Topies Definition 3.2 © (a,)nen is called bounded, if there is A,B ER with Vn A Gq (net S On) ® A sequence (a,,),0 SN(c)EN la,—alN(e) Notation: lim a, Ne) | Definition 3.3 A sequence is called divergent if it is not convergent. Example 3.2 ¢ (1,4, 4,...) converges to 0 (zero sequence) # (1,1,1,...) converges to 1 © (1,-1,1,-1,...) is divergent © (1,2,3,...) is divergent ‘Theorem 3.1 Every convergent sequence is bounded. for «= 1: N(1), first 1V(1) terms bounded, the rest bounded through a + .V(1) Not every bounded sequence does converge! (see exercise 3), but:3.1 Sequences and Convergence 25 ‘Theorem 3.2 Every bounded monotonic sequence is convergent B_t 3.1.1 Let (ay) Sequences and Limits (b,) two convergent sequences with: Jim (am, +n) lim (¢- an) Jim (ay ° bn) dm (2) lim a, =a Jim 6, = 6, then it holds: lim oy + lim by ak c- lim ay, b a $F bb40 Aye Theorem 3.3 The sequence ay, (1 PS >) ,n € N converges, nil 2 @, [2 225 The numbers (only) suggest that tl Proof: 1, Boundedness: Vn a, > 0 and 100 1000 10000 59 2705 2.717 2.7181 1 beth26 3. Calculus ~ Selected Topies 2, Monotony: Replacing n by m +1 in (1.) gives a, < dqy1, since in line 3 most summands in a, are bigger! ‘The limit of this sequence is the Buler number: = jim (1+ 2)" = anisosieas 3.2 Series Definition 3.4 Let (a)nex be a sequence of real numbers. The sequence of the partial sums is called (infinite) series and is defined by 5 ax. If (sa)nen converges, a we define » sae 2 o1234 5 6 7 8 9 Sequence a, O123 4 5 6 7 8 9 W Example 3.3 Series S, = }7ae]0 1 3 6 10 15 21 28 36 45 55 n ool 2 3 4 5 6 7 8 9 10 Sequencea fr b 2 2 Roa a a A ven " 2 4 8 16 32, 64 128-256 5181024 3 7 15 3 63 127-255 51110232047 SrisS, {1 5 2 2S = eS Se Ne Te 2 4 5 Ww B Oh We %% Siow (decimal) [1 15 175 1875 1938 1.969 1984 1.992 1.996 1.998 1.9903.2 Series 27 3.2.1 Convergence Criteria for Series Theorem 3.4 (Cauchy) The series }> a, converges iff Ve>0 INEN m> J Proof: Let s):= Yo ax. Then sy = Do a. Therefore (s,) nen is a Cauchy sequence 4 (sn) is convergent. a ‘Theorem 3.5 A series with a, >0 for k > 1 converges iff the sequence of partial sums is bounded. Proof: as exercise ‘Theorem 3.6 (Comparison test) Lot } 7, a convergent series with Wn cq > Oand (@n)nen a sequence with |an|a, converges. ‘Theorem 3.7 (Ratio test) Let ) a, a series with a, # 0 for all m > no. A real number g Ont with 0 no. Then the series }a,, converges. If, from an index no, #21| > 1, then the series is divergent. Proof: Apply the comparison test to} a, and > |ao|g” Example 3.4 == = converges Proof:28 3. Calculus ~ Selected Topies 3.2.2 Power series ‘Theorem 3.8 For each x € R the power series a is convergent Proof: The ratio test gives zn! el 1 5 for n> Gabe| a1 sa nz ala|—1 ao exp(z) is called exponential function Theorem 3.9 (Remainder) exp(e) = 92+ Ryle) N= th approrimation v +i)! with |Ry(x)| < 2: ¢ 3.2.2.1 Practical computation of exp(s) : )) 2 +Ry with Ry SG +R, th Ru Say 15: [Ril << 10-8 2. 71828182815 4£2-10- — (zounding error 5 times 10-1) Theorem 3.10 The functional equation of the exponential function Vz,y € R exp(x-+y) = exp(x) - exp(y),3.3. Continuity 29 Proof: The proof of this theorem is via the series representation (definition 3.5). It is not easy, because it requires another theorem about the product of series (not covered here). Conclusions: L.W2eR exp(—z) = (exp(z))? = PER exr(—z) = (exp(2)) = 5 2.YreR exp(z)>0 3. ¥neZ exp(n) =e" Also for real numbers x € R= ef := exp(z). 1. exp(z) -exp(—2) = exp(x — 2) = exp(0) =1 = exp(—2) xf 2 B20 > esplz)=ltet>+...2150 2<0 & <0 = exp(—2) > 0 = exp(x) = 3. Induction: exp (1) exp (n) = exp(n 1 +1) o exp(h) (for large 2 faster Note: for large r:=n+h n€N — exp(x)=exp(n+h) =e than series expansion) 3.3. Continuity Functions are often characterized in terms of “smoothness”, The weakest form of smoothness is the continuity. Definition 3.6 Let DCR, f DR a function and a € R. We write lim f(x) = C if for cach sequence (2)nen, (tn) € D with lim x, =a we have Jim se)30 3. Calculus ~ Selected Topies Definition 3.7 For x € R the expression [x] denotes the unique integer number n with néz 1. it holds BB F@) = co, if k even and ain. rea={_& » af & odd Proof: for 2 #0 since lim g(x) = 0, it follows lim f(x) = lim x* = oe. o Application: The asymptotic behavior for x -+ oo of polynomials is always determinate by the highest power in x Definition 3.8 (Continuity) Let f : D + R a function and a € D. The function f is called continuous at point a, if lim f( (@). J is called continuous in D, if f is continuous at every point of D.3.3. Continuity 31 For this function lim f(x) # a. f is discontin- uous at the point a x OX a x Example 3.6 ¢ f:x++ c (constant function) is continuous on whole R. # The exponential function is continuous on whole R. @ ‘The identity function f : 2 +4 x is continuous on whole I. Theorem 3.12 Let f,g : D > R functions, that are at a € D continuous and let r € B. Then the functions f+, rf, {+g at point a are continuous, too. If g(a) 4 0, then f is continuous at a, Proof: Lot (2) a sequence with (2,) © D and lim 2, =a Jim (f+ 9) = + ale) “Tim (A) (0) (ANG) Tim (faz) = (F-9)(a) folds because of rules for sequences Lye yi jim(L) = a) fae (a) o Definition 3.9 Let A, B,C subsets of R with the functions f : A+ B and g: BC. ‘Then gof:A—+C, x+49(f(z)) is called the composition of f and g. 1) fog(z) = F(z) Example 3.7. 2.) y—osin(z) = Ysin(a) 3) simoy—(z) = sin(yz) Theorem 3.13 Let f A> B continuous at a € A and g: A> C continuous at = f(a) Then the composition go f is continuous in a, too. Proof: to show: lim, — a =» — lim f(r) — f(a) => lim g( f(z) = aKa),32 3. Calculus ~ Selected Topies Example 3.8 is continuous on whole R, because f(x) = 2°, 9(z) = f(z) +a and +a h(x) = are continuous. g(z) Theorem 3.14 (« J Definition of Continuity) A function f : D + R is continuous at % € D iff: Ve>0 3650 YreD (\x—ml <6 > \f(2) — fle)l 2. The function fR* + R*, + 2* is continuous and strictly increasing, The inverse function f-!: Rt > R',2 ++ YF is continuous and strictly increasing Theorem 3.16 (Intermediate Value) Let f : [a,b] > R continuous with f(a) < 0 and f(b) > 0. Then there exists a p € [a, 6] with f(p) = 0. fx) | ¢iscontinaous, no zero!3.3. Continuity 33 Note: if f(a) > 0, f(b) < take —f instead of f and apply the intermediate value theorem Example 3.10 D=Q: x+>2?~2= f(z) fl) =~1,f(2) =2 there is ap € D with F(p) = 0. Corollar 3.3.1 Is f : [a,b] + R continuous and ¥ is any number between f(a) and f(b), then there is at least one Zé [a,6] with f(z) —¥. fb) fla) Note: Now it is clear that every continuous function on [a,6] assumes every value in the interval (f(a), f(b)! 3.3.1 Discontinuity Definition 3.10 We write lim f(x) =e (lim fla) = @), if for every soquonce (2) with >a (% R (n+ 1)- times continuously differentiable and xo,x € J. Then there is a z between xo and x such that flor) (2) BO) Tay (e—2)"* Example 3.13 f(x) =e Theorems 3.18 and 3.19 yield Convergence: the ratio test implies convergence of 7 bp. = lim), =0 > Jim Ry(z)=0 for all te R Thus the Taylor series for e* converges to f(x) for all x € R! Example 3.14 Evaluation of the integral [a wdz. In As the function f(r) = VI+ 2 has no simple antiderivative (primitive function), it can not he symbolically integrated. We compute an approximation for the integral by integrating the third order Taylor polynomial The exact value of the integral is about 1.11145, i.e. our approximation error is about 1%3.4 Taylor-Series 37 FO Definition 3.11 The series Tj(x) = )> i 2)» — ma) is called Taylor series of f with = 7 expansion point (point of approximation) 29 Note: 1, For © = xp every Taylor series converges, 2. But for x # 2» not all Taylor series converge! 3. A Taylor series converges for exactly these x ¢ I to f(r) for which the remainder term, from theorem 3.18 (3.19) converges to zero. 4, Even if the Taylor series of f converges, it does not necessarily converge tof. (+ example in the exercises.) Example 3.15 (Logarithm series) For 0 < x <2: in) -(e-1) -@ 5, ei Proof: tay ate ee 6 edgy pepe DD! W@)= 2 m@)=-5. m"@)=5, mM) =-F, we) = (ay Induction In (2) = (in(a))! = Expansion at 9 = 1 in (1), A (e-1" , (@-18 (@-1)4 Tinal2) = a@— 1k =(2-1)-% + ) Tale) = SO —*= 1) ; ve = This series converges only for 0 < x < 2 (without proof) Definition 3.12 If a Taylor series converges for all « in an interval I, we call I the convergence area. Is T= [to —r,20 +1] or P= (to- 1,40 +4 series, ris the convergence radius of the Taylor38 3. Calculus ~ Selected Topies Example 3.16 Relativistic ma: Einstein: total energy increase: ? Kinetic energy: Fain = (m — mo) mo m(v) to be shown: for v < ¢ we have kin * Emov” Bua = bm —mg)e = ( for x <1. ) mee = 3 oC = For 3.5 Differential Calculus in many Variables PRR (Hata yan) Y= Flt tay ate) zry=s(2) 3.5.1 The Vector Space R" In order to “compare” vectors, we use a norm: Definition 3.13 Any mapping || || :R” +R, x +> ||2!| is called Norm if and only if 1. |z|| = 0 iff 2. |Az|| = |Al|[z|| VA ER, 2 < RY 3. z+y| R",n a, is called sequence. Notation: (a,)ncx Example 3.17 me Definition 3.16 A sequence (a,)ncy of vectors a, € R" converges to a © R", if YT — shea YS q _— — ee = YS ~~ Ye>0 3N(e)EN |la,—al|N(e) Notation: lim a, =a ‘Theorem 3.21 A (vector) sequence (ay)nex converges to a if and only if all its coordinate sequences converge to the respective coordinates of a. (Proof as exercise.) Notation: a a= (a@z)kcx ax ER" . Note: Theorem 3.21 enables us to lift most properties of sequences of real numbers to sequences of vectors.40 3. Calculus ~ Selected Topies 3.5.3. Functions from R" to R™ Functions f from D CR" to B CR have the form f:DOB , a f(a) Example 3.18 I (x3, 02) = sin(x1 + lnra) m #1: Functions f from Dc R" to BCR” the form f:D>B , f(a) ea falta + yt) ™ Fa(@ay"** 5) Example 3.19 1 fF: RoR no ( van 2) ( cones tiny 2, Weather parameters: temperature, air pressure and humidity at any point on the earth, F : (0°, 360°} x [-90°, 90°] -+ [-270, 20] x [0, 00] x (0, 100% (> (srmecmct humnidity(®, ®) Note: The components f;(z),+-- , f(z) can be viewed (analysed) independently. Thus, in the following we can restrict ourselves to f : R" + R.3.5 Differential Calculus in many Variables aL 3.5.3.1 Contour Plots Definition 3.17 Let D CR?,B CR,c€ B, f: D+ B. The set {(x1,22)|f(x1,22) is called contour of f to the niveau c Example 3.20 f(x1,22) = x2 mm =e for n A0:m == (hyperbolas) c=0 & m=0Vm=0 ContourPlotix y, {x,-3,3}, {y,-3,3}, Contours -> {0,1,2,3,4,5,5,7,89-1, — proeapte y, 0,-3.3), fy.-3,3, -2,-3,-4,-5,-6,-7,-8,-9), PlotPoints -> 30) PlotPoints -> 60] 3.5.4 Continuity in R" analogous to continuity of functions in one variable: Definition 3.18 Let f : D + R” a function and a < R”. If there is a sequence (a,) (maybe more than one sequence) with lim a, = a, we write dim, f(«) if for any sequence (#,),» € D with lim &, = a: Im fla) =e Definition 3.19 (Continuity) Let f : DR” a finction and a € D, ‘The function f is continuous in a, if Jim, f(z) = f(a). f is continuous in D, if f is continuous in all points in D.42 3. Calculus ~ Selected Topies Note: These definitions are analogous to the one-dimensional case. Theorem 3.22 If f: D+ R",g : D> Rh: D > R are continuous in 9 € D, then f+9.f —9.f9 and £ (if h(ao)) £0) are continuous in ep. 3.5.5 Differentiation of Functions in R" 3.5.5.1 Partial Derivatives Partial Derivatives Example 3.21 fs: RoR S122) = 2atay keep a — const., and compute the I-dim. derivative of f wrt. 2; a. Fh cents) = filets) = tore} analogous with 2m = const. second derivatives: oe ate 2 f(z oe} Oo af — a af fy, 2) = L2ey2} Om Dry xz Ix, Example 3.22 (u, v,w) = ww + cosw B,(u,v,w) =v (u,v) =u (u,v, w) = —sinw ilway ++ stm) Definition 3.20 If f(x) = is partially differentiable in & = @o, ic. (iy san) all partial Derivatives 24 (a9)(i ym, k = 1,++> ,n) exist, then the matrix Fie) Flw) He) playa] 20 He) Fle) Be) Be) Be is called Jacobian matrix.3.5 Differential Calculus in many Variables 43 Example 3.23 Lincarisation of a function: f : R? > R¥ in 2» 2x, F(x) = [ sin(xr +2) In(ey) + 2 ° 2 f(2)= ( cos(y +2) cos(ri +22) ) z 1 FexohP GES Fa) I-dimensional J (#0) = Jim Linearisation g of f in 2p = (6) g(21,2) = F( 0 ° > glam)={ 0 J+[ a Inz 2 Note: For « -+ 20 ie, close to ato the linearisation g is a good approximation to f (under which condition?) ction f : R? > R with 2 it (x) 4( flay) 4 Ver if (x,y) # (0,0) 0 if (z,y) = (0,0) Differentiability: {is differentiable on R°\ {(0, 0)} since itis built up and division ifferentiable functions by sum, product a 6 yy sm Ly > tim 22 im 2F (2,0) ding gy (0.9) A lim 5, (0) — the partial derivative 2 is not continuous in (0,0). -> f is in (0,0) not differentiable.44 3. Calculus ~ Selected Topies Symmetries 1. f is symmetric wrt, exchange of x and y, ie, watt, the plane y=: 2. f is symmetric wrt. exchange of x and -y, ive. w.nt, the plane y 3. f(-2,y) = —F (zy), dh. f is symmetric w.r-t. the waxis. 4. f(x,y) =~ flay), Lh. f is symmetric wart. the 2-axis. Contours: Contours: Tate ife>0,r<0(3, Quadr) and ¢<0,x>0 (4, Quadr,) vf ga ife>0,2>0(1 Quadr) and e<0,2<0(2 Quadr,) Signs in the quadrants fayire Continuity: f is continuous on R?\{(0,0)}, since it is built up of continuous functions by sum, product and division. Continuity in (0,0): Let ¢ > 0 such that |x| =¢,ie ¢= Yr +y¥ @ y= vee3.5 Differential Calculus in many Variables 45 = fley=4* Aye str Vl-2/e from jal] < we get [fey] < fe and him f(x,y) Thus f is continuous in (0, 0) 3.5.5.2 The Gradient Definition 3.21 f : D+ R(D CR") H(z) The Vector grad f(x) := f'(x)7 = is called gradient of f, Le) The gradient of f points in the direction of the steepest ascent of /. fayae ry of Slay) =2 Example 3.25 Be) = 28 = uaasten= (2% )=2(3) of Dey) = 2 Jy) = 20 5.5.3 Higher Partial Derivatives of Let f | D> R"(D CR"). Thus 2£(z) is again a function mapping from D to R™ and 2 (4) o> Bons. Oni Dade, is well defined, Theorem 3.23 Let Dc R" open and f : D + R” two times partially differentiable. Then we have for all #9 € D and all ¢,j =1,+-+ ,n46 3. Calculus ~ Selected Topies Consequence: If f : D -+ R™(D © R* open) isk-times continuously partially differentiable, ‘then Or, for any Permutation Il of the numbers 1, 3.5.5.4 The Total Differential If f | R” — Ris differentiable, then the tangential mapping f(x) — f(20) + f'(20)(® — 20) represents a good approximation to the function f im the neighborhood of #5 which can be fila) ~ f (0) = f' (#0) — £0) With (x) and ae de = =a- to diy we get: f(a) = f'(ao)dx (2) = 0 FE (wojden = PF aaj, + 2x," in Definition 3.22 The linear mapping df yeh enlaen is called total differential oa Oe of the function f in ao Note: Since in a neighborhood of aq, fr is a good approximation of the function f, we have for all # close to 0 a(x) = fle) — f(o) Thus df (x) gives the approximate deviation of the function value f(z) from f(2o), when & deviates from ap a little bit 3.5.5.5 Application: The Law of Error Propagation Example 3.26 For a distance of s = 10 kn a runner needs the time of t = 30min yielding an average speed of v= 7 = 204", Let the measurement error for the distance s be As = Lo. and for the time we have At = 1 sec. Give an upper bound on the propagated error Av for the average speed! This can be solved as follows. To the given measurements 1, --- , tq, a funetion f : RX +R has to be applied. The measurement error for 2,--+ tq is given as +Ary,---,+Ar_3.5 Differential Calculus in many Variables az (An > 0. Vi = 1,+-,n). The law of exror propagation gives as a rough upper bound for the error Af (a) of f(y, +++ ,2tq) the assessment a, a, AS (t1,°-+ 20) < Se) An +...4 |e Ax, Definition 3.23 We call Afmaz(t1, the maximum error of f. The ratio “72#') js the relative maximum error. Not only occurs if all measurement errors dr, --- ,dzq add up with should be applied for about n < 5. Afnaz typically gives a too high estimate for the error of f, because this value the same sign. ‘This formula Definition 3.24 When the number of measurements n becomes large, a better estimate for the error Af is given by the formula Afmean(21, for the mean error of f. Example 3.27 Solution for example 3.26, Application of the maximum error formula leads to dv Av(s,t) =| (st) = | 0.001 km | 10 km 1, 05 hk” 025 7 3600 (st) av s+ | (5,0) As+ [Fee 1) 1 =|2[as+ At |; 0.0024 20) & _ oma 3600] 7h as This can be compactly written as the result v = (20 4 0.013)#2 Definition 3.25 Let f : D -> R two times continuously differentiable. The n x n—Matrix Fe) Bate (2) (Hess f)(2) := wrbg() -- FA(@) is the Hessian—Matrix of f in x, Note: Hess is symmetric, since ef os Onde; IxjOx,48 3. Calculus ~ Selected Topies 3.5.6 Extrema without Constraints Again we appeal to your memories of one-dimensional analysis: How do you determine extrema of a function f : R + R? This is just a special case of what we do now. Definition 3.26 Let D c R" and f : D + Ra function. A point x € D is a local maximum (minimum) of f, if there is a neighborhood U CD of x such that f(x) > fly) (Fe) < Fly) Wy eu. Analogously, we have an isolated local Maximum (Minimum) in 2, if there is a neighborhood U CD of x such that F(z) > Fly) (baw. f(z) < fly) WyeU, vAw Al these points are called extrema, If the mentioned neighborhood U of an extremum is the whole domain, ie. U = D, then the extremum is global. Give all local, global, isolated and non-isolated maxima and minima of the function shown in the following graphs: PlotaD(t [x,y], {x,-8,5.4y,-8,5}, PlotPoints -> 30] ContourPlot{tix.y], tx,-8,5).{y,-5,5}, PlotPoints -> 60, ContourSmoothing -> True, ContourShading-> False] Theorem 3.24 Let D CR" be open and f : D + R partially differentiable. If f has a local extremum in x € D, then grad f(«) = 0. Proof: Reduction on I-dim. case: For i= 1,---,n define gi(h) == ftay-- jt thy--- tm) If f has a local extremum in 2, then all g; have a local extremum in 0. Thus we have for all3.5 Differential Calculus in many Variables 0 is gf(0) =0. Since g{(0) = 2) we get aa) srad/(2) = =0 Ble) Note: © Theorem 3.24 represents a necessary condition for local extrema, « Why is the proposition of Theorem 3.24 false if D CR" is no open set? Linear Algebra Reminder: Definition 3.27 Let A a symmetric n x n-Matrix of real numbers. As positive (negative) definite, if all cigenvalues of A are positive (negative) As positive (negative) semidefinite, if all cigonvalues are > 0 (< 0) Ais indefinite, if all cigenvalues are # 0 and there exist positive as well as negative eigenvalues ‘Theorem 3.25 Criterium of Hurwitz Let A real valued symmetric matrix. A ist positive definite, if and only if for k = 1,---,n ay a >0 au One Ais negative definite if and only if-A is positive definite Theorem 3.26 For D < R" open and two times continuously differentiable f : D -+ R with grad f(x) =0 for @ € D the following holds: a) (Hess f)(x) positive definite f has in z an isolated minimum b) (Hlessf)(x) nogative definite + f has in & an isolated maximum ©) (Hessf)(#) indefinite + f has in # no local extremum. Note: ‘Theorem 3.26 is void if (Hessf)(z) is positive oder negative semidefinite. Procedure for the application of theorems 3.24 and 3.25 to search local extrema of a funetion f : (DC R") +R: 1. Computation of grad f50 3. Calculus ~ Selected Topies 2. Computation of the zeros of grad f 3. Computation of the Hessian matrix Hess/ 4, Evaluation of Hess (z) for all zeros a of grad f Example 3.28 Some simple functions f : IR? > R: L f@yaPty¥te srad/(e,y) = ( 3) = grad/(0,0) = ( ° ) =0 wer (3 8) is positive definite on all? = _f has an isolated local minimum in 0 (paraboloid) 2. fl,y)--e +e gradf(0,0) =0 Hess/ (¢ ’) = isolated local maximum in 0 (paraboloid). 3. f(xy) artby+e ab40 saty=(4) 0 Ve €R? => no local extremum, tess — (5 4) = Hessf indefinite = f has no local extremum, A Seu) Py! gradf = ( i) = grad (0,0) Hess (0,0) = G a) = Hess positive smidefinite, but f has in 0 an isolated minimum.ential Calculus in many Variables BL ered = ( ° ) > grad/(0,y) =0 (0.0) 20 joo (2 2) > Hessf positive semidefinite, but f has a (non isolated) local minimum. All points on the y-axis (x = 0) are local minima, sit 7 fayaery grad f(x,y) = ( Ft ) = gradf(0,0) = 0 Hess (0, 0) ( ‘4 ) = Hess positive semidefinite, but f has no local extremum, 3.5.7 Extrema with Constraints Example 3.29 Which rectangle (length «r, width y) has maximal area, given the perimeter v. Area f(x,y) = 2y. ‘The function f(x,y) has no local maximum on B®! Constraint: U = 2(2 + y) or x+y = ¥ substituted im f(x,y) = xy = = y=U/4 ist (the unique) maximum of the area for constant perimeter U! In many cases substitution of constraints is not feasible! Wanted: Extremum of a function f(z1,-- , 2) under the p constraints y(zis--- p49) =0 hepa, Tq) =O52 3. Calculus ~ Selected Topies Theorem 3.27 Let f: D> R and h: D~> RP be continuously differentiable functions on an open set DC R",n > p and the matrix h/(x) has rank p for all x € D. If zy © D is an extremum of f under the constraint(s) h(x) = 0, there exist real numbers Ay, ++ Ap with shea Da F le) =0 Wile yn and h(a) = 0 Vk =1,--p Illustration: For p= 1, i. only one given constraint, the theorem implies that for an extremum z of f under the constraint A(z) = 0 we have grad f(x0) + Agradh(2o) = 0 @ gradf and gradh are parallel in the extremum o! ¢ > Contours of f and A for h(x) = 0 are parallel in xo ¢ The numbers Ay, +++ ,\, are the Lagrange multipliers. Note: We have to solve n +p equations with n +p unknowns. Among the solutions of this (possibly nonlinear) system the extrema have to be determined. Not all solutions need to be extrema of f under the constraint(s) h(2r0) = 0 (necessary but not sufficient condition for extrema.) Definition 3.28 Let f,h be given as in theorem 3.27. The function L: D> R. Lx. S(@1y--> 420) + YO daha er,--- Zn) = is called Lagrange function. Conclusion: The equations to be solved in theorem 3.27 can be represented as: aL Ox, g(a) =0. (k= 1,---\p)3.5 Differential Calculus in many Variables 53 Example 3.30 Extroma of f(x,y) +3 under the constraint h(2, Contours of iy and const xy.2-0 Ley) = yy 434 Aes y—2) aL (x,y) = 20+ 2d0 Be aL Sey) = Bye = Arr gradL(x,y)=0 , A(z,y)=0 Qe+2e = 0 (I) Qtr = 0 (2) Pey-2=0 0) (2)in (1): 22—4zy — 0 (A) y = 2-22 (3a) (Sa) in (4): 2x—42(2—22) — 0 first soluti : 2 rst solution: ¢; = (9 ) is a maximum 2-8 det = 4a? = raf i mast Ms 5 Sel er w= (fF) oats (-y54 3. Calculus ~ Selected Topics Example 3.31 Extrema of the function f(x,y) = 4x? — 3xry on the disc Ko. = (eye + si} 7 | uN 1. local extrema inside the dise Dg ,: wads(en)- (852) 0 a ( t ) is the unique zero of the gradient. tres = ( Is] |-0-9- = Hess/ is neither positive nor negative definite. Eigenvalues of Hess f =: A At-e @ (A-\)z-0 B-A 3 B-A -3 o (8) B)s-vem(%) 2) @ (8B-A(-A)-9=0 & W-BA-9 Mia =44VI679 A=9 = Hess is indefinite = f has no local extremum on any open set D. => in particular f has on Dg , no extremum!3.5 Differential Calculus in many Variables 55 2. Local extrema on the margin, ic. on IDo,: local extrema von f(x,y) = 4x? — 3xy under the constraint «? + y? —1=0: Lagrangefunction L = 42? — 3xy + A(z? + y? — 1) % 8x — Sy + 2dr = (24+ 8)e ~ 3y aL ay 7 ty Equations for x,y, d: (1) 8r-3y+2\¢ =0 (2) 32+ Dy =0 @) @+yt-1 =0 (Dy -(@)x= (4) Sry —3y? +32? =0 first solution: (3) > (32) (3a)in(4) : £82vT—# ~ 3(1 ~ 24) +327 =0 Subst: 2? =u: 48VuvT—w = 3(1—u) -3u=3-6u squaring: O4u(l—u) = 9-36u+36u? Gh + 64u — 36u? +36u-9 = 0 00u? + 100u-9 = 0 Contours: > f(x,y) has on Rog im a ) isolated local maxima = f(x,y) has on Rog in 23 = ( ca isolated local minima.36 3. Calculus ~ Selected Topies 3.5.7.1 The Bordered Hessian In order to check whether a candidate point for a constrained extremum is a maximum or rinimum, we need a sufficient condition, similarly to the definiteness of the Hessian in the unconstrained case, Here we need the Bordered Hessian a ae mo 2 ® Hess hy hy ah, a ad ah Bhp aL eL Seo Be Sdn aE ‘This matrix can be used to check on local minima and maxima by computing certain subde- terminants. Here we show this only for the two dimensional case with one constraint where ‘the bordered Hessian has the form a i and the sufficient criterion for local extrema is (in contrast to the unconstrained case!) the following simple determinant condition: Under the constraint h(x, y) = 0 the function f has in (z,y) a © local maximum, if |Hess(1r, y)| > 0 © local minimum, if [Hess(x, y)| < 0. If |Fess(sx, y)| = 0, we can not decide on the properties of the stationary point (x,y). Appli- cation to example 3.30 yields eradZ(c,y) = ( 2x(1 +2) ) By +r 0m 1 Fess(x,y) = | 2e 21+) 0 1 0 2 Substitution of the first solution of gradZ = 0 which is x = gives y= 2, \= 4 into this matrix ol |Fress(0, 2)| = 60 2 ‘which proves that we indeed have a maximum in (0,2) 3.5.7.2 Extrema under Inequality Constraints ‘We recall that for finding an extremum of a function f(«) under the constraint g(x) = 0, we have to find a stationary point of the Lagrange function L(x, d) = f(z) + Ag(z)3.6 Exercises 57 by solving gradZ(w, A) = 0 Tn example 3.31 we had to find an extremum under an inequality constraint. We now want to develop a general method for finding maxinzum (minimum) of a function f(a) under a constraint (2) > 0 as shown in figure 3.1. rad JX) 2(x)=0 Figure 3.1 As in the example we have to consider two cases. Either g(x) > 0, ie. the constraint is inactive, or g(«) = 0 and the constraint is active. If g(x) > 0 the condition for an extremum, is simply grad f(z) = 0. This is cquivalent to solving grad L(x, A) = 0 with = 0 If the solution lies on the margin, i.e. g(x) = 0, we can apply the Lagrange formalism as shown above and get 40. Now f(z) will only have its maximum on the margin if \ > 0. If <0, f will be at a minimum on the margin (c.f. exercise 3.32). Thus, for finding a maximum of {(z) under the constraint g(2) > 0, in both cases we have Ag(a) = 0. Thus, to find a maximum of f(x) under the constraint g(x) > 0, we have to maximize the Lagrange function with respect to z and \ under the so called Karush-Kuhn-Tucker conditions a(z) >0 AzO Ag(a) =0. 3.6 Exercises Sequences, Series, Continuity Exercise 3.1 Prove (e.g. with complete induction) that for p € R it holds: (n+ (2+ n) p+k) = Step tn) +%) z Exercise 3.238 3. Calculus ~ Selected Topies a) Calculate i.e, the limit of the sequence (a,)new With ay = Land aay, = VIF aq. Give an exact solution as well as an approximation with a precision of 10 decimal places. b) Prove that the sequence (a,)nex converges, Exercise 3.3. Calculate i.e, the limit of the sequence (a,)nen with ao = 1 and any: = 1+1/ay. Give an exact solution as well as an approximation with a precision of 10 decimal places. Exercise 3.4 Calculate the number of possible draws in the German lottery, which result in having three correct numbers. In German lottery, 6 balls are drawn out of 49. ‘The 49 balls are numbered from 1-49. A drawn ball is not put back into the pot. In cach lottery ticket field, the player chooses 6 numbers out of 49. Then, what is the probability to have three correct numbers? Exercise 3.5 Investigate the sequen regarding convergence. (an)nen With ay 1= 1 Exercise 3.6 Calculate the infinite sum J" == Exercise 3.7 Prove: A series Df2g x with Vk - ax > 0 converges if and only if the sequence of the partial sums is limited. Exercise 3.8 Calculate an approximation (if possible) for the following series and investigate their convergence a) Psi" ™ 6) SO anit + (1/n)] * Exercise 3.9 Investigate the following functions f : R + R regarding contimuity (give an outline for each graph): 9 S@)= » s={ 2 fen a y= { ETP A272 9 serie 9 se Exercise 3.10 Show that f:R— R with f(a) ~ {9 fx zational ‘ 1 if ¢ irrational is not continuous in any point3.6 Exercises 59 Taylor-Series Exercise 3.11 Calculate the Taylor series of sine and cosine with zo = 0. Prove that the Paylor series of sine converges towards the sine function. VE at x = 0 and x Exercise 3.12 Try to expand the function f(x) series, Report about possible problems. 1 into a Taylor Exercise 3.13 Let f be expandable into a Taylor series on the interval (—r,r) around 0 ((r > 0). Prove: a) If f is an even function (f(z) = f(-z)) for all x € (-r,r), then only even exponents appear in the Taylor series of f, it has the form 7 ax,2** = b) If f is an odd function (f(x) = —f(—z)) for all x € (—r,r), then only odd exponents appear in the Taylor series of f, it has the form aaxyiz%* Exercise 3.14 Calculate the Taylor series of the function iy -[ o# itz 40 ne~{ 0 if2=0 t zo = 0 and analyse the series for convergence. Justify the result! Exercise 3.15 Calculate the Taylor series o for the approximate calculation of x. (Use for thi function arctan in arp = 0. Use the result for example tan(/4) = 1.) Functions from &" to R™ Exercise 3.16 Prove that the dot product of a vector « with itself is equal to the square, s length (norm) Exercise 3.17 a) Give a formal definition of the function f : R + B+ U {0} with f(x) = (2) b) Prove that for all real numbers x, y |x + yl < [2] + \y/ Exercise 3.18 a) In industrial production in the quality control, components are measured and the values 1, Ey determinated. The vector d = ¢—s indicates the deviation of the measurements to the nominal values s1,...,s,- Now define a norm on R® such that ||d]| < ¢ holds, iff all deviations from the nominal value are less than a given tolerance © b) Prove that the in a) defined norm satisfies all axioms of a norm. Exercise 3.19 Draw the graph of the following functions f : R? — R (first manually and then by the computer!) Aile.y) =? +98, falayy) =a? +O"? Fla, y) Exercise 3.20 Calculate the partial derivatives 74, #4, 24 of the following functions > Ban? Bar sR oR a) f(z) =|h a) flz)=s b) f(z) =a7 +f! ©) fle) =a" (x +22) e) f(z) =sin(z;+az2)60 3. Calculus ~ Selected Topies Exercise 3.21 Build a function f : R? + R, which generates roughly the following graph: ContourPlot [f [x,y], {x,-5,8},{y,-5,5}, PlotPoints PLotsDLE [x,y], {x,-5,5),4¥,-5,57, => 60, ContourSnoothing -> PlotPoints -> 30] True, ContourShading-> False] Exercise 3.22, Calculate the Jacobian matrix of the function f (x3, 2, 23) = ( sGnnen) ) 123 VEG Exercise 3.23 For f(x,y) = ( sin +e?) ): find the tangent plane at 9 = ( 3 ) Exercise 3.24 Draw the graph of the function y(1 cos) for |y| > || Heya { "rey Show that f is continuous and partially differentiable in R?, but not in 0 i , Pay Exercise 3.25 Caleulate the gradient ofthe function f(2,y) = -= > and draw it as Fy an arrow at different places in a contour lines image of f. Exercise 3.26 The viscosity 1) of a liquid is to be determinated with the formula K = 6rnor. Measured: r = 3em, v = Sem/sec, K = 1000dyn. Measurement error: |Ar| < O.lem, |Av| <0.003em/sec, |AK| <0.ldyn. Determine the viscosity 1) and its error An. Extrema Exercise 3.27 Examine the following fumction for extrema and specify whether it is a local, global, or an isolated extremum: a) fey) = yY-2-y) b) g(ey) = aks (ety)? (k=0,3,4) Exercise 3.28 Given the function f : R? > B, f(x,y) — (y — 2)(y — 32°). a) Calculate grad f and show: grad f(z, y) = 04 2 = b) Show that (Hess )(0) is semi-definite and that f has a isolated minimum on each straight line through 0. ¢) Nevertheless, f has not an local extremum at 0 (to be shown!) Exercise 3.29 Given the fimetions ®(x,y) = y2x— 25, f(x,y) = 2? +9? —3.6 Exercises 61 a) Examine & for extrema b) Sketch all contour lines A= 0 of & ¢) Examine @ for local extrema under the constraint {(c, y Exercise 3.30 The function sin(22? + 39°) Pry F(a,y) = has at (0,0) a definition gap. This ean be remedied easily by defining e.g. (0,0) a) Show that f is continuous on all R? except at (0,0). Is it possible to define the function at the origin so that it is continuous? b) Calculate all local extrema of the function f and draw (sketch) a contour Tine image (not easy). ©) Determine the local extrema under the constraint (not easy) i) e=01 ii) y=01 iii) Pry? Exercise 3.31 Show that grad(f g) = ggradf + f gradg. Exercise 3.32 Prove: When searching for an extremum of a function f(e) under am in- equality constraint g(a) > 0, the function f(x) will only have its maximum on the margin g() = 0 if A> 0 in the equation gradZ(x,d) =0. If <0, f will be at a minimum on the margin.Chapter 4 Statistics and Probability Basics Based on samples, statistics deals with the derivation of general statements on certain features, 4.1 Recording Measurements in Samples Discrete feature: finite amount of values, Continuous feature: values in an interval of real numbers. Definition 4.1 Let X be a feature (or random variable). A series of measurements &,...,% for X is called a sample of the length n. Example 4.1 For the feature X (grades of the exam Mathematics I in WS 97/98) following sample has been recorded: 1.013 2.22.22.225 2929 2.92.9 2.929 293.0 3.03.0 3333343.73.9394147 Let g(z) be the absolute frequency of the value x. ‘Then tz) = 2o(e) is called relative frequency or empirical density of X. Grade X | Absolute frequency g(a) | Relative frequency A(z) | TO T 0.042 13 1 0.042 22 3 0.13 25 1 0.042 29 7 0.29 30 3 0.13 33 2 0.083 34 1 0.042 37 1 0.042 39 2 0.083 41 1 0.042 47 1 0.042 The content of this chapter is strongly leaned on [2]. Therefore, [2] isthe ideal book to read4.1 Recording Measurements in Samples 63 Izy < 22 2 variables, the data But one can determine the covariances between two Thus all eigenvalues are If dependencies among different variables are to be compared, a4.3. Multidimensional Samples 67 Initial Math-Test Results of this Lecture aS ee Correlation of Pre-Test and Initial Math-Test CORRELATIONS, MM: 0.38 (blue), EEM: 0.15 (rec!) a £ ° = 7 2 ak 4 ¢ Bell Peal eal Bh ee lh, tle Ponts in presence tast Appendicitis database of 473 patients Example 4.3 In a medical database of 473 patients*with a surgical removal of their ap- pendix, 15 different symptoms as well as the diagnosis (appendicitis negative/positive) have "Tike data was obtained fom the hospital 14 Nothelfer in Weingarten with the friendly assistance of Dr. Ramp. Mr. Kuchelmeister used the data for the development of an expert system in his diploma thesis68 4. Statisties and Probability Basies been recorded, age gender_( D pain quadranti_(Omnein__ pain_quadrant?_(O-nein, pain_quadrant3_(O-nein, pain_quadrant4_(O=nein. guarding. (O=nein..1=ja) rebound tenderness_(0=nein_1=}a) pein.on_tapping__(O=nein_.1=}a) vibration_(O-nein_1=ja) rectal_pain_(Onein__1=ja) ‘tonp_ax tonp_re: Leukocytes: anitus_( appendicivis_ ‘The first 3 data records w% 10010101 w20010101 wm io0010000 10 10 00 Correlation matrix for the data of all 473 patients: continuous continuous continuous continuous ot on 37.9 38.8 23100 0 1 36.9 37.4 8100 0 0 36.7 36.9 9600 0 1 Loon 01 bows 034 0x7 os Loos 37 TON OT Toe TORO OUS Uo OUT OU OMT OOS OT OOO 1. 00074-0019 Dos O.968-017 O.0084-0.17 “O14 018 DOT DOM Oe O01 02 008-0081 -024 “1. 0059 DA OOM 0.14 G49 OORT O54 DOS OL OOIT “O14 00s D2 055 OoHL GO 019 088 O15 OUI O11 O12 OOES O21 D058 17 013 O08 O44 OoTL 1. O16 04 O28 02 O24 036 0.20 -OoDD1a O88 oe 021 025 Oomals O16 1 O47 O2% O21 O49 O24 O27 O08 OOM Day doi oor O44 O08 04 O27 1. O53 O35 O19 ORT BaF Boa O88 O13 002% OUST OUsTOOI 02 021 025 O24 1 GIT DIT 022 OOas OT ‘0017 002 O11 O084011 02 019 019 O15 OAT 1. O72 O28 O03 O15 0084 DOM 012 OomHO12 036 O21 O27 O19 OAT O72 1. O58 OOM OAL OM 00% O11 O41 0083 029 027 0.27 023 022 O24 038 1 GOs Odd 0.0 B11 024 OolTo21 -DOVI8 Hass 0.026 02 OMS O35 OOIL OosL 1. -LoDss The matrix structure is more apparent if the numbers are illustrated as density plot In ‘the left diagram, bright stands for positive and dark for negative, The right plot shows the absolute values, Here, white stands for a strong correlation between two variables and black for no correlation *The frst t0 images have been votated by 90°. Therefore, the fields in the density plot correspond to the matrix elements44 Probability Theory 69 It is clearly apparent that most of the variable pairs have no or only a very low correlation, ‘whereas the two temperature variables are highly correlated, 4.4 Probability Theory The purpose of probability theory is to determine the probability of certain possible events within an experiment, Example 4.4 When throwing a dic once, the probability for the event throwing a six” is 1/6, whereas the probability for the event throwing an odd number” is 1/2. Definition 4.4 Let © be the set of possible outcomes of an experiment. Each w < 9 stands for a possible outcome of the experiment. If the w, € M exclude each other, but cover all possible outcomes, they are called elementary events Example 4.5 When throwing a die once, = {1,2,3,4,5, 6}, because no two of these events can occur at the same time, Throwing an even number {2, 4,6} is not an elementary event, as well as throwing a number lower than 5 {1, 2,3, 4}, because (2,4, 6}/{1, 2,3, 4} = {2,4} # 0 Definition 4.5 Let be a set of elementary events, A= — A = {w € Mw ¢ A} is called the complementary event to A. A subset A of 2" is called event algebra over 9, if 1LMEA 2. With A, Ais also in A. 3. If (Ap)new is a sequence A , then US An is also in A. Every event algebra contains the sure event 9 as well as the impossible event 0. At coin toss, one could choose A = 2 and = {1,2,3,4,5,6}. Thus A contains any possible event by a toss,70 4. Statisties and Probability Basies If one is only interested in throwing a six, one would consider A = {6} and only, where the algebra results in A= {@, A, A, 9} The term of the probability should give us an as far as possible objective description of our ,believe” or conviction” about the outcome of an experiment. As numeric values, all real numbers in the interval [0, 1] shall be possible, whereby 0 is the probability for the impossible event and 1 the probability for the sure event. {1,2,3,4,5} 4.4.1 The Classical Probability Definition Lot $1 = {u1,W2,-.. un} be finite. No elementary event is preferred, that means we assume a symmetry regarding the frequency of occurence of all clementary events. The probability P(A) of the event A is defined by P(A) = [Al Amount of outcomes favourable to A jay ‘Amount of possible outcomes It is obvious that any elementary event has the probability 1/n. The assumption of the same probability for all elementary events is called the Laplace assumption. Any elementary event has the probability 1/n Laplace assumption Example 4.6 Throwing a die, the probability for an even number is 12,4.6}1 P24) = 345.01 4.4.2. The Axiomatic Probability Definition The classical definition is suitable for a finite set of elementary events only. For infinite sets a more general definition is required. Definition 4.6 Let be a set and A an event algebra on 9. A mapping P:A>(0!) is called probability measure if: 1 PO 2. If the events A, of the sequence (A,)nen are paitwise inconsistent, ie, for i,j € N itholds 4,9.A; = 0, then ( “ )-¥ For A € A, P(A) is called probability of the event A. From this definition, some rules follow directly44 Probability Theory m1 Theorem 4.1 1. P(@) = 0, ic. the impossible event has the probability 0. 2, For pairwise inconsistent events A and B it holds P(AU B) = P(A) + P(B) 3. For a finite amount of pairwise inconsistent events Aj, Ap, ... Ax it holds P (u ) SPA) a a 4. For two complementary events A and A it holds P(A) + PC 5. For any event A and B it holds P(AU B) = P(A) + P(B) — P(A B). 6. For AC B it holds P(A) < P(B) Proof: as exercise 4.4.3 Conditional Probabilities Example 4.7 In the Doggenriedstraije in Weingarten the speed of 100 vehicles is mea- sured. At each measurement it is recorded if the driver was a student or not. The results are as follows [Brat Frequency | Relative frequency Vehicle observed 100 T Driver is a student (S) 30 03 Speed too high (@) 10 on Driver is a student and speeding (SG) 5 0.05 We now ask the following question: Do students speed more frequently than the average person, or than non-students ‘The answer is given by the probability P(G|S) for speeding under the condition that the driver is a student Driver is a student and specding| 1 ‘Driver is a student] ~ 30° 6 P(G\S) = Definition 4.7 For two events A and B, the probability for A under the condition B (conditional probability) is defined by P(ANB) PAB) = At example 4.7 one can recognize that in the case of a finite event set the conditional probability P(A|B) can be treated as the probability of A, when regarding only the event 7 The determined probabilities can only be used for further statements ifthe sample (100 vehicles) is representative. Otherwise, one can only make a statement about the observed 100 vehicles.72 4. Statisties and Probability Basies B, i.e. as puja) = 408) Definition 4.8 If two events A and B behave as P(A|B) = P(A), then these events are called independent. A and B are independent, if the probability of the event A is not influenced by the event B. ‘Theorem 4.2 From this definition, for the independent events A and B follows P(ANB) = P(A) - P(B) Proof: P(ANB) P(AIB) PAI) = Bay =P(A) = P(ANB)= P(A): P(B) Example 4.8 The probability for throwing two sixes with two dice is 1/36 if the dice are independent, because 1 an 6 36 = P(die 1 = six die P(die 1 ix) - P(ai six), whereby the last equation applies only if the two dice are independent. If for example by magic power die 2 always falls like die 1, it holds P(die 1 = six die 2 = six) = z 4.4.4 The Bayes Formula Since equation (4.7) is symmetric in A and B, one can also write P(ANB) P(A\B) = w P(B|A) = P(A\B) aswell as P(BIA) = 57, Rearranging by P(A B) and equating results in the Bayes formula P(BIA)- P(A) P(A|B) = 4 P(B)4.5 Discrete Distributions 73 Bayes Formula, Example A very reliable alarm system wams at burglary with a certainty of 99%. So, can we infer from an alarm to burglary with high certainty? No, because if for example P(A|B) = 0.99, P(A) =0.1, P(B) = 0.001 holds, then the Bayes formula returns P(AIB)P(B) _ 0.990.001 P(B\A) = (lA) P(A) or =0.01 4.5 Discrete Distributions Definition 4.9 A random variable with finite or countably infinite range of values is called discrete random variable. Example 4.9 Throwing a die, the number X is a discrete random variable with the values {1,2,3,4,5,6}, this means in the cxample it holds 1 = 1,...,r = 6. If the dic does not prefer any number, then p= P(X =n) = 1/6, this means the numbers are uniformly distributed. The probability to throw a number <5 P(X <8) = Dw =5/6 In general, one defines Definition 4.10 The function, which assigns a probability p; to each x, of the random variable X is called the discrete density function of X. Definition 4.11 For any real number x, a defined function > rH P(X <2) is called distribution function of X. Such as the empirical distribution funetion, P(X < 2) is a monotonically increasing step function. Analogous to the mean value and variance of samples are the following definitions.74. 4 Statisties and Probability Basics Definition 4.12 The number E(X) = Soap. is called expected value. The variance is given by Var(X) = E((X ~ E(X))?) = )3(a: — B(X))?p: whereby \/Var(z) is called standard deviation. It is easy to see that Var(X) := E(X?) — E(X)* (exercise) 4.5.1 Binomial Distribution Let a soccer player's scoring probability at penalty kicking be = 0.9. The probability always to score at 10 independent kicks is Byooa(10) = 0.9" = 0.35. It is very unlikely that the player scores only once, the probability is Byoa9(1) = 10.0.1" - 0.9 = 0.000000009 We might ask the question, which amount of scores is the most frequent at 10 kicks Definition 4.13 The distribution with the density funetion Bue(s) = (2) p*( = py" is called binomial distribution. ‘Thus, the binomial distribution indicates the probability that with m independent tries of a binary event of the probability p the result will be x times positive, Therefore, we obtain Broo o(k) = (a) 0.1.0.9" The following histograms show the densities for our example for p = 0.9 as well as for p=05,4.6 Continuous Distributions 75 For the binomial distribution it holds B(X) = So. () p= 2)* = np BS and Var(X) = np(l ~ p). 4.5.2 Hypergeometric Distribution Let NV’ small balls be placed in a box. A of them are black and n balls, the probability to draw x black is Ky (® ~ Ky @) The left of the following graphs shows Hyooso,.o(z), the right one Hy.osyao(2). This cor responds to N balls in the box and 30% black balls. It is apparent, that for V = 10 the density has a sharp maximum, which becomes flatter with VV > 10. KC white. When drawing Ay,xn(2) = 4.6 Continuous Distributions76. 4. Statisties and Probability Basies Definition 4.14 A random variable X is called continuous, if its value range is a subset of the real numbers and if for the density function f and the distribution function F it holds F(z) = P(X <2) = [ F(t With the requirements P() = 1 and P() = 0 (see def. 4.6) we obtain tim, F(z) =0. sowie lim F(2) 4.6.1 Normal Distribution The most important continuous distribution for real applications is the normal distribu- tion with the density : yt (=n) Pual2) exp ( oe) Theorem 4.3 For a normally distributed variable X with the density y,,. it holds E(X) = pand Var(X) =o? For = 0 and @ = 1 one obtains the standard normal distribu- tion yoa. With ¢ = 2 one ob- tains the flatter and broader den- sity yo. Example 4.10 Let the waiting times at a traffic light on a country road at lower traffic be uniformly distributed. We now want to estimate the mean waiting time by measuring the waiting time T 200 times. aeutigke Wartezeiten (49 Klassen) 8 The empirical frequency of the waiting times is shown opposite in the image. The mean value ( lies at 60.165 s quencies and the mean value i econds. The free dicate a uniform distribution times between 0 und 120 sec. Tz t ( sec] Due to the finiteness of the sample, the mean value does not lie exactly at the expected value of 60 seconds. We now might ask the question, if the mean value is reliable, more precise with what probability such a measured mean differs from the expected value by