We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Taming Wild High Dimensional Text Data with a Fuzzy Lash

Authors: Amir Karami
Abstract: The bag of words (BOW) represents a corpus in a matrix whose elements are the frequency of words. However, each row in the matrix is a very high-dimensional sparse vector. Dimension reduction (DR) is a popular method to address sparsity and high-dimensionality issues. Among different strategies to develop DR method, Unsupervised Feature Transformation (UFT) is a popular strategy to map all words on a new basis to represent BOW. The recent increase of text data and its challenges imply that DR area still needs new perspectives. Although a wide range of methods based on the UFT strategy has been developed, the fuzzy approach has not been considered for DR based on this strategy. This research investigates the application of fuzzy clustering as a DR method based on the UFT strategy to collapse BOW matrix to provide a lower-dimensional representation of documents instead of the words in a corpus. The quantitative evaluation shows that fuzzy clustering produces superior performance and features to Principal Components Analysis (PCA) and Singular Value Decomposition (SVD), two popular DR methods based on the UFT strategy.
Subjects: Machine Learning (stat.ML); Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG); Applications (stat.AP)
Cite as: arXiv:1712.05997 [stat.ML]
  (or arXiv:1712.05997v1 [stat.ML] for this version)

Submission history

From: Amir Karami [view email]
[v1] Sat, 16 Dec 2017 17:57:57 GMT (45kb,D)

Link back to: arXiv, form interface, contact.