FAFU机器学习07-1 kNN and Bayes课件.pptxVIP

Foundations of Machine Learning kNN and Bayes Main Classification MethodsLogistic RegressionLinear Discriminant AnalysisDecision Tree InductionNearest Neighbor Bayes Classification MethodsClassification by BackpropagationSupport Vector MachinesEnsemble Methods… Nearest Neighbor Classifiers Nearest Neighbor ClassifiersRequires three things The set of stored records Distance Metric to compute distance between records The value of k, the number of nearest neighbors to retrieve To classify an unknown record:Compute distance to other training records Identify k nearest neighbors Use class labels of nearest neighbors to determine the class label of unknown record (e.g., by taking majority vote) Definition of Nearest Neighbor 1 nearest-neighborVoronoi图，又叫泰森多边形或Dirichlet图，它是由一组由连接两邻点直线的垂直平分线组成的连续多边形组成。N个在平面上有区别的点，按照最邻近原则划分平面；每个点与它的最近邻区域相关联。 Nearest Neighbor ClassifiersCompute distance between two points: Euclidean distance Determine the class from nearest neighbor listTake the majority vote of class labels among the k-nearest neighbors Weight the vote according to distanceweight factor, w = 1/d2 Nearest Neighbor ClassifiersCompute distance between two points: Euclidean distance Minkowski distance Manhattan Distance for nominal attributes Distance for binary attributesDistance for ordinal variables Distance for mixed types Nearest Neighbor Classifiersclass sklearn.neighbors.DistanceMetricThis class provides a uniform interface to fast distance metric functions. The various metrics can be accessed via the get_metric class method and the metric string identifier from sklearn.neighbors import DistanceMetric dist = DistanceMetric.get_metric(euclidean) X = [[0, 1, 2], [3, 4, 5]] dist.pairwise(X) array([[ 0. , 5, [ 5 0. ]]) Nearest Neighbor Classifiersclass sklearn.neighbors.DistanceMetricMetrics intended for real-valued vector spaces: Nearest Neighbor ClassifiersChoosing the value of k: If k is too small, sensitive to noise points If k is too large,