人工智能Nilson版-英文课件-Chap03.pptVIP

(c) 2000, 2001 SNU CSE Biointelligence Lab  Neural Networks Chapter 3 3.1 Introduction TLU (threshold logic unit): Basic units for neural networks Based on some properties of biological neurons Training set Input: real value, boolean value, … Output: di: associated actions (Label, Class …) Target of training Finding f(X) corresponds “acceptably” to the members of the training set. Supervised learning: Labels are given along with the input vectors. 3.2.1 TLU Geometry Training TLU: Adjusting variable weights A single TLU: Perceptron, Adaline (adaptive linear element) [Rosenblatt 1962, Widrow 1962] Elements of TLU Weight: W =(w1, …, wn) Threshold: ? Output of TLU: Using weighted sum s = W?X 1 if s ? ? 0 0 if s ? ? 0 Hyperplane W?X ? ? = 0 3.2.2 Augmented Vectors Adopting the convention that threshold is fixed to 0. Arbitrary thresholds: (n + 1)-dimensional vector W = (w1, …, wn, 1) Output of TLU 1 if W?X ? 0 0 if W?X 0 3.2.3 Gradient Decent Methods Training TLU: minimizing the error function by adjusting weight values. Two ways: Batch learning v.s. incremental learning Commonly used error function: squared error Gradient : Chain rule: Solution of nonlinearity of ?f / ?s : Ignoring threshod function: f = s Replacing threshold function with differentiable nonlinear function 3.2.4 The Widrow-Hoff Procedure Weight update procedure: Using f = s = W?X Data labeled 1 ? 1, Data labeled 0 ? ?1 Gradient: New weight vector Widrow-Hoff (delta) rule (d ? f) 0 ? increasing s ? decreasing (d ? f) (d ? f) 0 ? decreasing s ? increasing (d ? f) The Generalized Delta Procedure Sigmoid function (differentiable): [Rumelhart, et al. 1986] The Generalized Delta Procedure (II) Gradient: Generalized delta procedure: Target output: 1, 0 Output f = output of sigmoid function f(1– f) = 0, where f = 0 or 1 Weight change can occur only within ‘fuzzy’ region surrounding the hyperplane (near the point f(s) = ?). The Error-Correction Procedure Using threshold unit: (