wo (WNIT. 4 ComPutational Learnigg Theory + Sample complerity for finite nypotnunts Sample compbxity for unfinite Yypotneris » Mustake bourd. vrodel of dteorntay. Terence BASED LEARNING:- “K nearest Nerghbour Leorni ay ghted. Regrerrion Radial Bais function. nefworks, a leery, + Comf udational Leoni pg Then ny 1 - ~ GomBusationas Lerrrning Theory da a ab: field af AT concenred with mathematical methods or ote design abplied Jo Shu macRine leorning problems. for the purpose of quont fying Leorning taaks ond Algerian -ths fea mega mony dincipines, auch as profability theory, stahahes , infor ration theory ete: - CLT provides a formal Promework , in which Jt possibile to Precigey formulat and addreoa queatiom mending Birformann of axfeurt Jeoming algoritme: Thekey aofiects tras runt be Prralized wu cae dn whith Leoynur imterads with Hs envi vonmunt 1. what hypothenis sface ohould be needed and th Hee ot doth set 2 3% How can we te wh a modo ue a geet approxi mahibr. for thu target funchon 2 4 A fowral dapihihon of effitienty of toth dot, Sorage and. Procening time C(Hme Comjauxily )-COT focus on th fotinoing goals {aang ie “3, ? 22 Rambles Cornbtnity- Sean Hoo many Avuhi: examples are needed, Pr a Seamer to learn a day! Qrctionet auceemfubly. « QD _LomPdadional ComBure by:- The computational sombirdly seoretrpord tp de computational effort C such an CPU speeds memory sine, duadraction Wh) ‘required. by a bearing algorithnn to converge: B) Migtake Bound :. St refew to che tolat numBer of mixclaniPication by dearnen “before convergence. te. eonvenging to & Succersfur Aypothesis - —Thore ore many sub: fields of shidy, put two widely ured au i PAC ea 9. ve Dimension. * PAC teaming { Probably Approximately Correct Learning ) = 9+ isa theorehcal machine Leorning Sromecoork created. by Leslie Vatbiortt: un 1984. — oH alms to quantify athe dif picuty thvolved Jn Learnipg & doak and might be conridured sub: Pied of CLT: = fh Poblem cn PAC Leomable, Jf aww ub an abgoriteen(®) for ony aubtribwier (D) and concept clars (C) which produces a Fypothuads (A) with small ever.@ lets unduistard ctu defiiition of PAC + — Subbore theve ih an uhbut space = {0,14 Le. Boolean /nfud Shoe. ~ PB comebt ¢:%—» {0,14 a booleon furction defined on a trp 4 SPoce [target farction frat clamniPy sth gin input to 0 or 43 ~ A concept clua t= §G, G4 wh set of corcelsts- Ca a —~ Im PAC teornh Targek un de fird ov born 0. fixed unkinown Concept with rape to dilobibwion D ovr ah inp shace XG cemoI, Ba, couy). =... drawn Jindspendtty ob yardem from dilotrtuion D. Groat of Sh deomun is fo awl 0 Aujbethesis REC UC _Dimensions; — H stands for Vabnik - cheruonckis Dirnennions. - H dh a meanuv of te capauity of a +t of Yurctions Hat son $e deornt by a clami Pication algovi thre ~ he capability of a furchon un given fm terms of orm Pbruily. ssive power, richress and Plexi bitihy. - 3+ cb aloo a dheoretical model /frome ork of racking teorn wshich dh afmed Um quantify ae sapabidity of a domincg atm = 3 con Be ccensicunied an sn sub fold of shasisHeal leaming Theo,A Somfie ComBtexity fox _fibile Hybotheata: &) > PAC Learrobility ub largely determined $y te nir of “training examBles required. ‘ey the deorner- &, athe youth uh the num Run of requsred- ‘training examples with problem sine) ur on SmnPortant factor ted affects ute succern of learner. We considon only esnisistentileornoy ( Bounding core clans of « Hamov to Lonyistert ) - A Learn uh consistent JP Jt ouput Aypothals wRy porfectey Sel the dot. we. hlx)= Che) > Next wn to bound Bar nor of traning exomples required 4 Conristent earn - (independant of trating Og oxi teers uned to. derive typothut, 1. tonaiotent typoHwots ). This 1 done using oncept af Upnion shou * Sevrion fou uh dufined an thy se of all ~fybothents Che 1D whi co celowni'Pay core training exompis D. Geuion spa (V§,,5) = {hex | @x Cx, caJen) hx) @))) Bo Pere, eveuy consistent Learner oubpuds cm By pothenis sore: to wien specs, Dregandion of Mn apou. X, typoites spate H basing sabe. DJ we Evy utsnaistont fy tothens Belongs te _wuion spor U8u,0. So te bound we rw of drain: exompler needed comaitstor Leanne , : > twon(e)necPauctermvowioSpoc : St rrrors a vewsin apace from whicR alt error (E) pave Bern ex Rounted (or emo.9)) A 9S. dh whi® a tupothunt Ran error Belew oe err tnverRold (6) ur called try &) exRouated vewriory .® Let us comidun a Huns § foace CH) + hanget concept: ©. data, diishybuhin (D) and act of tratneng exam ples. Bowen spou V8, 9 ur sold oy err @) exhauated wrt to Conuph Card data dist Bution DO, sf every Aypothunta ‘fan brror len Ham a Hyrpfeld valu &) GA € YSi4,0) CMTE (A) < enor) F Genewd Bourd on dhe nov of Fraininey exomBile suPficrent Py ony Consintont Jeornen ub an by _~ai ™= a (Jom + an (+) whens mda mur of training exam Bies & ub Enor bourd (E) Int Nedunal Logarithm. (doge) H ub Hybothats sce S ub Probabictily Jeveh volun Tow or tuo drayrdock of cin bound. -{sombienty tone) © % can tod tw quite urak Bounds © Qn come of vinfinite hypothe we ar not abby this© | A Simp Corpus hits byottene ty Of Kybothuris afou H ~ The VC dimemion measwus vthy comb rot by ctr not of chistinct Fypotoen's |b] Bud dwateod ty the nor of distinct Jnotorcer (examples) from xX tat con te tom pletely dineminaded. uring H: To Undin thin Yirst we define the notion of Shatteruing_o- Sot of snstorces:-and_UC dimenians.... * Stateripgef-oract -ofesiwshonecet: {shatougemaammpantivionty ~ GCnsidir some subsd of unstorces SOX ond 6 be de conabe Com cen an unpub sbouX% ( C:% fo, 13) (‘“ denote CCS) an re of ak tobels ] COS) = ff cea oo, Clam) 43 £6 C3 [ Bach ty botuats from H simbosen some label on S yd A partthen $ uhto two subsets up Les | weet ond LES |AA)=0 — Gein oa Crdone ad S, Arywsaren 2 posiblentabale , thang & H may not Ge abe ot rePresent some of tren - : a os GytotonreH that m-neprmee ci Os of Se Seq ue (A 8) | es pe ae a a 84 | Bea ie — = - ~T es © @® i Se gt = Ha 8-2, % no'oP possible Lads = aS= 2794 Also for evyy Label crus ext a Ay bottusin Frm H S=0 Are lobdls thy. rnstancenSimilan you con-ry fr $23. oe nob be NOK () santa tH FL nh pre t abot. . Ad fix ‘ © dh rot mienoey Od 9 srunathtin fr srrboning Se Pottibim on label it can he nen: inear a ou) * OC Dinmerisions:- — the vc dimemion (VC) 6 0 Ay pothenis space H un denoted by UCCH)- we sich thi ound on somBle com testy rod wow VC (H) yatheou than |H|. This allews do cPanacten'ze ou Sample cemplrcity of rrany unfenite Aypotheria- 4 A tee, grew bo gitr by: mz a [* Wh ()+ 8 UC(N) «Loge 8) en Gh Tunibrr of draining example whwu;- ™M= ne of draining exomples:- 8 = Rotability tanafort UC(H)= VC dimennion of fypothun's H &= error found. Mustake Bourd ud abarogy durcuned’—INSTANCE BASED LE, = Instance-based learning is a family of learning algorithms that, instead of performing explicit generalization (when training examples are provided), compares new problem instances with instances seen in training, which have been stored in memory. * Instance-based methods are sometimes referred to as "lazy" learning methods because they delay processing until a new instance must be classified.No model is learned = The stored training instances themselves represent the knowledge = Training instances are searched for instance that most closely resembles new instance * tis a supervised learning algorithm. Instead of estimating the target function once for the entire instance space, these approaches can estimate it locally and differently for each new case to be categorized, which is a significant benefit of delayed, or lazy, learning. Performance Time complexity of instance based learning algorithms depends upon the size of training data. Time complexity of this algorithm in the worst case is 0 (n), where n is the number of training items to be used to classify a single new instance. Functions of instance-based learning Instance-based learning refers to a family of techni which produce a class label/prediction based on the similarity of the query to its nearest ues for classification and regression, neighbor(s) in the training set. Functions are as follows: 1. Similarity: Similarity is a machine learning method that uses @ nearest neighbor approach to identify the similarity of two or more objects to each other based on algorithmic distance functions. 2. Classification: Process of categorizing a given set of data into classes, It can be performed on both structured and unstructured data. The process starts with predicting the class of given data points. The classes are often referred to as target, label or categories. 3. Concept Description: Much of human learning involves acquiring general concepts from past experiences. This description can then be used to predict the class labels of unlabeled cases. Advantages of instance-based learning: ly adaptable to new data, which is collected as we Bo + This technique is ea: = fast: it only involves database writes and reads, and + simple: no interesting learning takes place.STE a a a a @ Disadventagés of instatice-based Iéatninik: + Classification costs are high. + Large amount of memory required storing the data, and each query involves starting, the identification of a local model from scratch + No generalization: data are stored explicitly, * Duplicate values likely to cause a problem. ‘Some of the Instance-based Learning algorithms are: 1) Lazy Learners (KNN algorithm) 2) Radial Based Functions (RBF) 3) Case-Based Reasoning (CBR) 4) Locally Weighted Learning (LWL) Lazy Learners (KNN algorithm) = Uses k “closest” points (nearest neighbours) for performing classification. ‘The nearest neighbors of an instance are defined in terms of the standard Euclidean distance. Let an arbitrary instance x be described by the feature vector {ai(x), a2(x), --.dn(*)) where ar (x) denotes the value of the rth attribute of instance x. Then the distance between two instances xi and xj is defined to be d(xi, xj), where Vee) = ayy \ rol * K-Nearest Neighbor algorithms classify a new example by comparing it to all previously d(xi.x)) seen examples The assumption K-NN uses is that the new cases are similar to the previous already trained and classified cases, and hence classifies the new data in the category it is most similar to. + Itcan be used for both classification and regression problems. Example: Suppose, we have an image of a creature that looks similar to cat and dog, but we want to know either it is a cat or dog. So for this identification, we can use the KNN algorithm, as it works on a similarity measure. Our KNN model will find the similar featuresof the new data set to the cats and dogs images and based on the most similar features it will put it in either cat or dog category KNN Algorithm Steps: The following are the steps for classification task by using KNN Algorithm. Step 1 Load the Data Set Step-2 Choose any valve of nearest neighbor data points (K). K can be any integer value. (Say K=5) Step 3 Calculate the Euclidean distance between test data (Queries instance x) and each rows of training data. Note down the value of the Euclidean distance in an ordered table. ‘Step 4 Arrange the Euclidean distance table in ascending order. Choose top K rows (say K=5) trom this table. Step 5 Now assign a class to new instance point based on the most nearest class of rows Step6 = End Category B * cereaorye New data point ye [NN * New data point assigned to Category ? Category A Adva: of KNN Algorithm: © [tis simple to implement © Itis robust to the noisy training data © It can be more effective if the training data is large. Disadvantages of KNN Algorithm: © Always needs to determine the value of K which may be complex some time. The computation cost is high because of calculating the distance between the data ° points for all the training samplesw Beample: The result marks of Five student arr given rn Below Table . Now a neco quayy 4 orrived obowt a studs mands. Maths seat a] Come Os Now, Bi dank dr to det whether othis new Student (x) un pas oF iL. Solution: ® Gret Data ond chore K.. (Le K=5) @ Caleutah Euclidean divtoncr behveen mew shuchne (x) mraaks wie GL students of dre table - de (OQ H+ ee Ma)? QD Cuelidson diistoncr talewladion i. For pornt of row 1 Iai +6-3)% = 5-38 For point of vow 2 for point of row 3 6 3 + (8-8) = 336 For point of roo 4: for point of vow 5 2Qu ® Oreten vt table axeonting So its euclicear drtonee dry ancendirg) order. | dnereaning ordre Euclidean Table © now Pick ToB Krows CHeu Ks 3) from se Euclidean’ Tab! Je ¢ check AM Bhear points of row 2, 8 and 5 teonge #e Pass 80, we decid Hak the new student manks (x) do a “Pa” cadegery Hence classification in done -| ) + Locatl Lueignted Segrenrion - 3H ud a emonapanametricwalgoritirsnuniike a Spbical Diner regremion Akgoritimn which un a panomednic_algoritrn:. [A parometric algorithms s> an algevitirn tras dostit need to retour ( trauhing data) when we need sts rake Prectictien. 7] ‘thy Locally Gughted Reqremer ? + for a Wincor dataset Wuineay rygresnion sam Guntrali ce Be earning modet to fit de datr- ji ets | bX it Locatlyy eesghtod fe rae a Pentland (@ Linear Dato cet (8) Non. Lineor data 2k * Lneor Teremion un not suitable for non. lineow data net + Bo we introduce Locatty wergdhed Reqrtmmion algorithm to Prekict an valu “wg lens fe The artual vale of a gion dole yet- (By ApLE Hey the cate rney ito grater, dota rer ard Powe reuth'ple vmrall Liner tha ean fe dtr data) — Buh memory Booed non metric apbroack that expla! Y oe ees retain hw fg dott, and urnrit wack time a ychory (ie $+ > also a sauifaowvrtsedLeor ning Algoritinen Baie Tewninolagien — Be Regression: reans abbfrroni nnosiog a real valued tanger function.2 Agcaly: The funchin (f) Ue abproximated haned on tee ats. points ushicR arr Local oy mah he queay port Coa) [ newrimlana 3 Weighted; The och trashing SomBLe vn waighted ty Hs distorce - From quay pont Gq). i Wy Wwaghted civabwnce uth giver by kennel airoothieg function. AWE KEtoords,-Kernel.furchion-Ke-detowenin Pes in wor 40g, then wpa amigrd be BucBihas a * “0 - Tub fan to Xq Hun W Lr arratl. ? Given a new snvstonce Xq,, LWR censbruch an inmabion 2 thot fils the trainig eromple in the neigh Boshood searourding <4. g ey te This abprotimadion in dram und to cealculate che £.(xq) which wr enfin value oh dager os welarted Linear Kon faget fumction ub abbprozimdtid neor qe point (xa) wing a dinear Rumchion ant Abproximahion of fumction P)= Wo + WAG + WO)... +Wrank) O Whur a ; (x) valur of th abhibudt of uimtona (x) coefficients of cevaite, we wae Gradient peas nue (to ee [00 = NE (e)- foo) +4 C&) who 1p Heornieg Gmasanh, Rover wy = To Prd ctw wy - AbbroKimmadion for yth ath butt of mle ee nS.aE Canc Baned loarning = Uned for clasai fication © Regrerion Cremory Parts Mpa), = Stig an odvanced tixhona Bane? Learning metho A whicw ar ured to solve mort comflx problems - — Doesnt tincluck euclidean d&'stonc. —“process of solving new proBlemn Bared on Mu solutions of Similan Last problem - “Chen a new case ovrfven to classify. then Pirsh am fdorteat 94 Hou fund chon che reafeehive tring set us checked - of tt uh not fund , then COR sdution we vetiwined WIL cReck the UNsterncer Hat au airmilan / neighbor to tne ed care ond ft, -reafiective solution up retunred. ? TEnstonce Boned Learning | Ane Bared. Leamingy Lazy Learner te: defer Lox Learners BL deulvion making Focus us on Similan Siintlan. Sindoncen 4 dincarding roworcen aPpount frorn 2 a vrePrenerstunsionces a> sty picanty 2ePresenta uring more Teal valued ponds ire toh symbolic erdadions - ab munrionca cushion mer SY aioe whace lo{thn hey Funetior normalizing the range of 1, Preprocessor: This prepares the input for processing ¢ numeric value features to ensure that they are treated with equal importance by the similarity function formatting the raw input into a set of cases ete Similarity: This function assesses the similarities of a given case with the previously stored cases in the concept description Assessment may involve explicit encoding and/or dynamic computation. CBL similarity functions find a compromise along the continuuum between these extremes. 3. Prediction: This function inputs the similarity assessments and generates a prediction for the value of the given cases goal feature i.e. a classification when it is symbolic values. 4. Memory Updating: This updates the stored case base such as by modifying or abstracting previously stored cases forgetting cases presumed to be noisy or updating a features relevance weight setting, Case-based learning cycle: : After the problem situation has been assessed, the best matching case is 1. Case retriev: searched in the case base and an approximate solution is retrieved. 2. Case adaptation: The retrieved solution is adapted to fit better the new problem. olution evaluation: The adapted solution can be evaluated either before the solution is 3. applied to the problem or after the solution has been applied. In any case, if the 2 adapted again or accomplished result is not satisfactory. the retrieved solution must | more cases should be retrieved. 4: Case-base updating: If the solution was verified as correct, the new case may be added to the case base A new problem is matched against the cases furnishing the case base and one or more similar cases are Retrieved. A solution suggested by the matching cases is then Reused.Ge Problem ere | The benefits of CBR are: + Ease of knowledge elicitation + Absence of problem-solving bias + Incremental learning © Suitability for complex and not-fully formalised solution spaces + Suitability for sequential problem solving + Ease of explanation + Ease of maintenance Limitations of Case Based Learning are as follows: + Handling large case bases + Dynamic problem domains + Handling noisy data + Fully automatic operation * Customer Service Helpdesk Engineering and Law for Technical Design and Legal Rules Medical science for Patient case histories and treatment.18 Ab Radial Ban's function (RGF): - a | — A Rodiat Bar's function us a real valid furchon whoo, value deberds only ont. Unbut and Sts distance from some fixed Print (c) oy BRL origin + — $+ we euclidean distona Jd eakeetatr distance Setveantive ports. — © The RGF dn ured for otha abproxi mabion of multivaniats tang er fonctions . The Tanget famehion abproximration da giver a ’ Fe) = + Ewe Ky (Cru, €)) wha fey = Abmopieation of ruth vendiats Jarges furchion We = Iihat Was Was Ost wer, Kad Gu, 0d) = Fema Function AC XU, x) = distana Belweer Ky and %- — Gish used din ANN which Kove only ons Hidden Nedt- { Basi rule Serge Jay ow perceptron tut wed fer solving non-linear Soho sets. 2 » Widdunit o. - rAlsdilad Xa Z © 7 ™ Om Foe) = Ew; Rs q o \ Ot, 4 bat oe “pakios fone) \ fershion . we wip ~ + Gnvtead of nen-Lineor activation farchim REF aw unk -fer providing iA Non. Linearity vin tha neleo = Commonly ured for Klarsifteadion Janks in ANN:= RBF a used for ane nen: lincor datase (by tonerting dh data its Linear, doko. st ) example : OEE OP wo clans of dasa KN ete Hay Fey ars * Fay ant 9088 O00 204 ay A > Shor 500590002204 P04 # . 50 O ~ Grcle & Yor -Lineor olocta. ~ 50. drowing a tine belween data points th not possible there fre. “We toret ote giver date. into Wheor deta using 2 Steps: X- Yncreane athe dimensionally (ad = 3d) ~ ophoral step (hot monadiatory , onvy bared on requirernent 22 bxBord the direction CHorizontat) ComBrem cthe Qrechon (Vouical ) dul = 3 be (one Bo, after arr 9 stopsdati udu become %- Awd aD po Hw Be ee 09 COSTZ0Q00000°0 ©00°°20000 0000How +0 temverdthn data into Lingo :- > Lonridin one centre vondomly for a dade, point > draw a woncntic dirde. () ©) DHA uh the vodsinn (distance form ctr Center to Aitic port ) TS comb ress | exBord ue wre 3 functions: W Mutkiqvadric :- $0) = (x24 cr) Yo. C>O : seme Constant i. e. ae @ drvewe Huth: Quadne. > Cr) = yn © Gawsfan Funchon:- (most ueemmonly ured) = =v? O(r)= exh [ = j 50% some wenatamt - *ovortgges: @ Trainining Un RBF retwork ue fact @ basily sntrpret ustas up ctu meaning of furchon(radiat) m ~ wn Yiddon nod. Dey dunign @ goo Goreraizahon © throng Soleron dp uinpuk Nolar-