THE HKPU, DEPARTMENT OF EIE Lecture Notes of Machine Learning Lecture 1 55y January 12, 2015 1 SUPERVISED LEARNING ˜k˜‰´§x(i ) L«\C‰\A§y (i ) L«C‰ 8I" (x(i ), y (i ) ¡Ø§|{(x(i ), y (i ));i = 1,2,...,m} ¡8§X L «\C§Y L«C§5‘§•˜¢Œ" The goal of supervised learning can be described as follows: given a training set, to learn a function h : X → Y so that h(x) can predict the output value as close as the corre- sponding value of y. The function h is called hypothesis. ˜u8I/“§Supervised Learning '⁄a. Regression Problem££8flK⁄: C·ºY. Classification Problem£'aflK⁄: C·l. 1.1 5£8 •k˜{5£8flK§•b\Cx ·3 m2§•k–eL“ hθ(x) = θ0 + θ1x1 + θ2x2 1 

θi ·ºŒ (k¡)§øºŒ–\X 5NY. •\ Vg§-x0 = 1, ªL“–U⁄ hθ(x) = n θi xi = θT x i=0 y3§|km 8§•F"Ø•ºŒθ§ƒhθ(x) 8IƒUC"OºŒθ§•‰´Xecost function: m i=1 J(θ) = 1 2 (hθ(xi )− y i )2 (1) ø…Œ…ŒØ§e5§•?Xƒ)ºŒθ" 1.2 LMS ALGORITHM ƒ‘ºŒθ ƒ…Œ§•–^Fe{? 1ƒ)§ºŒθ #–L«⁄Xe θj := θj − α ∂ ∂θj J(θ) ¥§α ¡Learning rate. •k…ŒŒ§{B§k˜k Ø„(x, y). ∂ ∂θj J(θ) = ∂ ∂θj = 2· 1 2 (hθ(x)− y)2 1 2 (hθ(x)− y) · (hθ(x)− y)· ∂ n ∂θj θi xi − y) ( = (hθ(x)− y)· ∂ ∂θj = (hθ(x)− y)x j i=0 ?§•–kØ#OK θj := θj + α(y i − hθ(xi ))xi j (2) ø·k¶LMS#K§Widrow-Hoff˘SOK§ºŒθ #ßu " lØ„§•ºŒθ X#ƒ…Œ–´æ" fl¢§Øu„kı„§k{–غŒθ ?1#§ ·batch model§,·stochastic model" 2 

batch model#K·zg#H{⁄k§Xe⁄« Repeat until convergence θj := θj − α (y i − hθ(xi ))xi j m i=1 { for every j } stochastic modelz‹?1g#§Xe⁄« Loop { { for i = 1 to m, θj := θj − α(y i − hθ(xi ))xi } j for every j } l¡«“–w§batch model zg#H{⁄k§ø· ~§AO·3Œ~ı§stochastic model ·=#§z #g§ˇ~„e§stochastic model ‹e/’batch model fl§ Lstochastic model"·kUˆ{´æ§·3NC5 £6˜§øV·¡stochastic model ˇ§“´æk‰¯ 5§,,·´æNC§Øıø«Cqfi†~C §stochastic model ˙p§⁄–stochastic model‹‘k˜ {" ˇ~«{·(«“§8'⁄Øıbatch§, |^stochastic model?1#§zbatch§?1g#§ø| ^stochastic modelp§3‰§~3NC6˜" REFERENCES [1] Andrew Ng, "Machine Learning", Stanford University. 3