What is pattern recognition (lecture 4 of 6)

What is pattern recognition (lecture 4 of 6)

• 1.
  ERI SUMMER TRAINING COMPUTERS& SYSTEMS DEPT. Dr. Randa ElanwarLecture 4
• 2.
  Content Feature types Feature selectionmethods Classifier types Template Matching Statistical methods 2 ERI Summer training (C&S) Dr. Randa Elanwar Statistical methods Stochastic Processes Structural Matching Neural Networks Rule-based methods The real pattern recognition system
• 3.
  Feature types 3 After havingthe big picture about how classification is done, we need to discuss some points in more details to have deeper understanding Features Types Three different groups of feature extraction techniques exist: templates, structural decomposition and series expansion. ERI Summer training (C&S) Dr. Randa Elanwar
• 4.
  Feature types 1. TemplateMatching, i.e. using a template (as a mask) to compute certain features. For example: 4 ERI Summer training (C&S) Dr. Randa Elanwar using a grid to compute the ratio of foreground black pixels to the back ground white pixels, or the count of black pixels per column, or the number of connected pixels. The template can be circular mask with sectors and the foreground black pixels area could be the feature, and so on.
• 5.
  Feature types 5 Template Matchingis very sensitive to pattern/sample size, rotation and translation variations as well as to noise. I.e., two typical patterns may have different feature vectors if one of them is bigger in size, rotated, or translated a little. Thus, it has limited application. 2. Structural Decomposition i.e., describing the geometry of the pattern shape. For example (1) knowledge about the contour of the object like height contour and chain code features, (2) knowledge about what sort of components make up that object like end points, T-joints and X-joints, loops, curves, etc. ERI Summer training (C&S) Dr. Randa Elanwar
• 6.
  Feature types 3. SeriesExpansion i.e., representing a signal by a linear combination of a series of simpler well-defined functions, where the coefficients of the linear combination are considered as features. If these functions are orthogonal to each other, then the 6 ERI Summer training (C&S) Dr. Randa Elanwar If these functions are orthogonal to each other, then the coefficients provide non-redundant (distinct) information (features) like: Moments, Fourier transform, Gabor transform and wavelets. There is an infinite number of potential features that one can extract from a finite 2D pattern. However, best features are those relevant to the problem domain.
• 7.
  Feature types Usually, featuresused are selected and tested in groups till the best group of features leading to the best classification results are selected eventually. As we mentioned before, increasing the number of features to over fit the data classification in the feature space is a 7 ERI Summer training (C&S) Dr. Randa Elanwar to over fit the data classification in the feature space is a wrong practice because once we have a new set of data the decision boundary will not be able to classify them correctly. Moreover, as we illustrated, increasing features more than needed will lead to introducing noise to the system and cause samples overlap in the feature space.
• 8.
  Feature selection methods 8 Atthis stage, we are not going to go into deep details describing the features selection techniques but for knowledge these are the techniques names only: Suboptimal solution sets , Sequential forward selection (SFS), Generalized SFS, Sequential backward selection (SBS), Generalized SBS, etc. ERI Summer training (C&S) Dr. Randa Elanwar
• 9.
  Feature selection methods 9 Allof them go around the same idea either initialized by a small set of features and adding to it gradually or initializing by all features in one set and taking away from it gradually according to the calculation of probability of correct classification.calculation of probability of correct classification. Now it's time to have a closer look to other classifiers than NN for broader vision of machine learning. ERI Summer training (C&S) Dr. Randa Elanwar
• 10.
  Classifier types As wementioned before, recognition involves classifying each unknown object into one of a finite number of categories (or classes). There are different methods of classification to implement the above recognition approaches: Template Matching. 10 ERI Summer training (C&S) Dr. Randa Elanwar Template Matching. Statistical methods. Stochastic Processes (Markov Chains). Structural Matching (Trees, Chains, etc.). Neural Networks. Rule-based methods. Many recent methods mix several techniques together in order to obtain improved reliability.
• 11.
  Template Matching 11 determine thedegree of similarity between two vectors (Groups of pixels, shapes, curvatures, etc) in the feature space using some minimum distance approach (e.g. Euclidean distance). Examples: Direct Template Matching, String Matching, Elastic Template Matching. ERI Summer training (C&S) Dr. Randa Elanwar
• 12.
  Statistical methods 12 concerned withstatistical decision functions and a set of optimal criteria, which determine the probability of the observed pattern belonging to a certain class. Examples: k-Nearest-Neighbor (k-NN), and Bayesian Classifier (Maximum likelihood). ERI Summer training (C&S) Dr. Randa Elanwar
• 13.
  Stochastic Processes is adoubly stochastic process (probabilistic models) offered for modeling patterns. One of the most important properties is the existence of efficient algorithms to automatically train the models without any need of segmenting patterns. 13 ERI Summer training (C&S) Dr. Randa Elanwar any need of segmenting patterns. Example: Hidden Markov Model (HMM)
• 14.
  Structural Matching 14 patterns arerepresented as unions of smaller sub- patterns (primitives). It can find the relationship among them. Examples: grammatical methods and graphical methods. ERI Summer training (C&S) Dr. Randa Elanwar
• 15.
  Rule-based methods Sometimes called"Ad-hoc rules", they use abstract descriptions of patterns to recognize them. For example, a rule of recognizing the letter ‘x’ might be ‘two lines that cross over and are at approximately 45 and 135 degrees’. 15 ERI Summer training (C&S) Dr. Randa Elanwar The problem with these methods is that it is not possible to design an exhaustive set of rules that model all possible ways of forming a letter. They are still not as good as the performance of statistical methods. They are however very useful in disambiguating between certain class pairs using very few parameters. One example is the disambiguation between a ‘v’ and ‘u’ based on the presence or absence of a hook at the end
• 16.
  The real patternrecognition system 16 ERI Summer training (C&S) Dr. Randa Elanwar
• 17.
  The real patternrecognition system 1. Data acquisition and sensing: Measurements of physical variables (capturing image, recording speech or video etc.) Important factors (parameters) in this step that affects the patterns due to data acquisition: bandwidth, resolution, 17 ERI Summer training (C&S) Dr. Randa Elanwar patterns due to data acquisition: bandwidth, resolution, sensitivity, distortion, SNR, latency, etc. Data collected is divided to 3 sections (train, test, validation). The training data is that used for learning models. From time to time you need to make a test to adjust the system parameters (thresholds) or rules. In such case, validation data is used. When the system performs quite well (without over fitting), test section is used for final evaluation.
• 18.
  The real patternrecognition system 18 2. Pre-processing: an essential task that precedes the tasks of image representation and recognition. Its importance is derived from the fact that the discrimination power is directly proportional to the digital image quality, in the sense that, the higher the image quality the less confusions we have and thus more powerful classification we can make.make. Some common operations performed as preprocessing are: binarization, the task of converting gray-scale image into a binary black-white image; noise removal, the extraction of the foreground textual matter by removing textured background, salt and pepper noise or interfering strokes; image enhancement and restoration, the task of converting the image to be more suitable than the original image for a specific application ; morphological image processing, the task of extracting image components that are useful in the representation and description of region shape. ERI Summer training (C&S) Dr. Randa Elanwar
• 19.
  The real patternrecognition system The basic morphological algorithms are: boundary extraction; region filling; extraction of connected components; thinning and thickening. And the most common morphological operations used are 19 ERI Summer training (C&S) Dr. Randa Elanwar And the most common morphological operations used are dilation, where the value of the output pixel is the maximum value of all the pixels in the input pixel's neighborhood; erosion, where the value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood; opening, which smoothes contours and eliminates small islands and sharp peaks; and closing, which smoothes contours, fuses narrow breaks and eliminates small holes.
• 20.
  The real patternrecognition system 20 ERI Summer training (C&S) Dr. Randa Elanwar
• 21.
  The real patternrecognition system Segmentation of pattern image is sometimes considered as preprocessing operation and sometimes is a separate stage. 21 ERI Summer training (C&S) Dr. Randa Elanwar separate stage. 3. Feature extraction: Finding a new representation in terms of features
• 22.
  The real patternrecognition system 22 4. Model learning and estimation: Learning a mapping between features and pattern groups and categories. 5. Classification: Using features and learned models to assign a pattern to a class. 6. Post-processing (extra optional stage to improve performance): Evaluation of confidence in decisions. Exploitation of context to improve performance. Combination of multiple classifiers decisions. ERI Summer training (C&S) Dr. Randa Elanwar