Search Results for « Automated Text Categorization »

The automated categorization (or classification) of texts into predefined categories has witnessed a booming interest in the last 10 years, due to the increased availability of documents in digital form and the ensuing need to organize them. In the research community the dominant approach to this problem is based on machine learning techniques: a general inductive process automatically builds a classifier by learning, from a set of preclassified documents, the characteristics of the categories. The advantages of this approach over the knowledge engineering approach (consisting in the manual definition of a classifier by domain experts) are a very good effectiveness, considerable savings in terms of expert labor power, and straightforward portability to different domains. This survey discusses the main approaches to text categorization that fall within the machine learning paradigm. We will discuss in detail issues pertaining to three different problems, namely, document representation, classifier construction, and classifier evaluation.

The automated categorization (classification) of texts into predefined categories is one of the widely explored fields of research in text mining.Now-a-days, availability of digital data is very high, and to manage them in predefined categories has become a challenging task.Machine learning technique is an approach by which we can train automated classifier to classify the documents with minimum human assistance.This paper discusses the Naïve Bayes, Rocchio, k-Nearest Neighborhood and Support Vector Machine methods within machine learning paradigm for automated text categorization of given documents in predefined categories.

Recent popularity of collaborative tagging as a component of a retrieval system has lead us to study such a system. Similar to text categorization, albeit in a less centralized fashion, collaborative tagging relies on humans to annotate documents with metadata descriptions, i.e. tags. For that reason, this thesis attempts to extend the tagging process to include a more consistent non-human annotations in the form of automatic text categorization. In applying automatic text categorization to collaboratively tagged data, we have created two sets of experiment. The first experiment compares two classification methods, Naive Bayes and Support Vector Machine (SVM) in a straightforward 1-vs. all classification. The results of the comparison allow us to make important observations such as the benefit of using a maximum margin classifiers (SVM) in annotating concepts with skewed document distributions as well as establishing a baseline result. For the second experiment, we have found that the lack of structure in tagging has limited our learning approach to the simple 1-vs. all setting. Inspired by the application of hierarchical categorization in web directories[15], we introduce in our second experiment a categorization approach that automatically builds a hierarchy from the tag space and incorporates it to the training and classification process. Unlike previous hierarchical categorizations that rely on human-generated hierarchies, our hierarchical approach relies on an artificial hierarchy that is created from tag usage analysis. After the method was applied to the dataset, we compared the result of the new methods with the baseline results from the first experiment. Based on that comparison, we observed that our hierarchical approach improves not only on the quality of predictions, but also the efficiency (total training and classification time) of our automatic text categorization system.

Manual Analysis of huge amount of textual data requires a tremendous amount of processing time and effort in reading the text and organizing them in required format.In the current scenario, the major problem is with text categorization because of the high dimensionality of feature space.Now-a-days there are many methods available to deal with text feature selection.This paper aims at such semi automated text categorization feature selection methodology to deal with a massive data using one of the phases of David Merrill's First principles of instruction (FPI).It uses a pre-defined category group by providing them with the proper training set based on the demonstration phase of FPI.The methodology involves the text tokenization, text categorization and text analysis..

The goal of this research is to develop an algorithm to automatically retrieve critical information from raw data files in NASA's airborne measurement data archive.The product has to meet specific metrics in term of accuracy, robustness and usability, as the initial decision-tree based development has shown limited applicability due to its resource intensive characteristics.We have developed an innovative solution that is much less resource intensive while offering comparable performance.As with many practical applications, the data available are noisy and correlated; and there is a wide range of features that are associated with the information to be retrieved.The proposed algorithm uses a decision tree to select features and determine their weights.A weighted Naive Bayes is used due to the presence of highly correlated inputs.The development has been successfully deployed in an industrial scale, and the results show that the development is well-balanced in term of performance and resource requirements K EY

The goal of this research is to develop an algorithm to automatically classify measurement types from NASA’s airborne measurement data archive. The product has to meet specific metrics in term of accuracy, robustness and usability, as the initial decision-tree based development has shown limited applicability due to its resource intensive characteristics. We have developed an innovative solution that is much more efficient while offering comparable performance. Similar to many industrial applications, the data available are noisy and correlated; and there is a wide range of features that are associated with the type of measurement to be identified. The proposed algorithm uses a decision tree to select features and determine their weights.A weighted Naive Bayes is used due to the presence of highly correlated inputs. The development has been successfully deployed in an industrial scale, and the results show that the development is well-balanced in term of performance and resource requirements.

The goal of this research is to develop an algorithm to automatically classify measurement types from NASA's airborne measurement data archive.The product has to meet specific metrics in term of accuracy, robustness and usability, as the initial decision-tree based development has shown limited applicability due to its resource intensive characteristics.We have developed an innovative solution that is much more efficient while offering comparable performance.Similar to many industrial applications, the data available are noisy and correlated; and there is a wide range of features that are associated with the type of measurement to be identified.The proposed algorithm uses a decision tree to select features and determine their weights.A weighted Naive Bayes is used due to the presence of highly correlated inputs.The development has been successfully deployed in an industrial scale, and the results show that the development is well-balanced in term of performance and resource requirements.