Linear Least Squares Approximation性最小二乘逼近.pptVIP

Linear Least SquaresApproximation By Kristen Bauer, Renee Metzger, Holly Soper, Amanda Unklesbay Linear Least Squares Is the line of best fit for a group of points It seeks to minimize the sum of all data points of the square differences between the function value and data value. It is the earliest form of linear regression Gauss and Legendre The method of least squares was first published by Legendre in 1805 and by Gauss in 1809. Although Legendre’s work was published earlier, Gauss claims he had the method since 1795. Both mathematicians applied the method to determine the orbits of bodies about the sun. Gauss went on to publish further development of the method in 1821. Example Consider the points (1,2.1), (2,2.9), (5,6.1), and (7,8.3) with the best fit line f(x) = 0.9x + 1.4 The squared errors are: x1=1 f(1)=2.3 y1=2.1 e1= (2.3 – 2.1)2 = .04 x2=2 f(2)=3.2 y2=2.9 e2= (3.2 – 2.9)2 =. 09 x3=5 f(5)=5.9 y3=6.1 e3= (5.9 – 6.1)2 = .04 x4=7 f(7)=7.7 y4=8.3 e4= (7.7 – 8.3)2 = .36 So the total squared error is .04 + .09 + .04 + .36 = .53 By finding better coefficients of the best fit line, we can make this error smaller… E must be MINIMIZED! How do we do this? E = ∑(mxi+ b – yi )2 Treat x and y as constants, since we are trying to find m and b. So…PARTIALS! ?E/?m = 0 and ?E/?b = 0 But how do we know if this will yield maximums, minimums, or saddle points? Minimum! Since the expression E is a sum of squares and is therefore positive (i.e. it looks like an upward paraboloid), we know the solution must be a minimum. We can prove this by using the 2nd Partials Derivative Test. 2nd Partials Test And form the discriminant D = AC – B2 If D 0, then (x0,y0) is a saddle point. If D 0, then f takes on A local minimum at (x0,y0) if A 0 A local maximum at (x0,y0) if A 0 Calculating the Discriminant If D 0, then (x0,y0) is a saddle point. If D 0, then f takes on A local minimum at (x0,y0) if A 0 A local maximum at (x0