The impact of pre-processing and feature selection on text classification

Nowadays text classification dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platform. However, those textual data are hard to managed and processed for classification purposes. Pre-processing activities and feature select...

Full description

Bibliographic Details
Main Authors:	Nur Syafiqah, Mohd Nafis, Suryanti, Awang
Format:	Conference or Workshop Item
Language:	English English
Published:	2019
Subjects:	QA76 Computer software
Online Access:	http://umpir.ump.edu.my/id/eprint/25395/ http://umpir.ump.edu.my/id/eprint/25395/1/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf http://umpir.ump.edu.my/id/eprint/25395/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf

id	ump-25395
recordtype	eprints
spelling	ump-253952019-11-12T07:19:10Z http://umpir.ump.edu.my/id/eprint/25395/ The impact of pre-processing and feature selection on text classification Nur Syafiqah, Mohd Nafis Suryanti, Awang QA76 Computer software Nowadays text classification dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platform. However, those textual data are hard to managed and processed for classification purposes. Pre-processing activities and feature selection are two methods to process the text document. Therefore, this paper is presented to evaluate the effect of pre-processing and feature selection on the text classification performance. A tweet dataset is utilized and pre-processed using several combinations of pre-processing activities (tokenization, removing stopwords and stemming). Later, two feature selection techniques (Bag-of-Words and Term FrequencyInverse Document Frequency) are applied on the pre-processed text. Finally, Support Vector Machine classifier are used to test the classification performances. The experimental results reveal that the combination of pre-processing technique and TF-IDF approach achieved greater classification performances compared to BoW approach. Better classification performances hit when the number of features is decreased. However, it is depending on the number of features obtained from the preprocessing activities and feature selection technique chose. 2019 Conference or Workshop Item PeerReviewed pdf en http://umpir.ump.edu.my/id/eprint/25395/1/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf pdf en http://umpir.ump.edu.my/id/eprint/25395/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf Nur Syafiqah, Mohd Nafis and Suryanti, Awang (2019) The impact of pre-processing and feature selection on text classification. In: International Conference On Computer Science, Electrical And Electronic Engineering., 29 - 30 April 2019 , Kuala Lumpur, Malaysia. pp. 1-9.. (Unpublished)
repository_type	Digital Repository
institution_category	Local University
institution	Universiti Malaysia Pahang
building	UMP Institutional Repository
collection	Online Access
language	English English
topic	QA76 Computer software
spellingShingle	QA76 Computer software Nur Syafiqah, Mohd Nafis Suryanti, Awang The impact of pre-processing and feature selection on text classification
description	Nowadays text classification dealing with unstructured and high-dimensionality text document. These textual data can be easily retrieved from social media platform. However, those textual data are hard to managed and processed for classification purposes. Pre-processing activities and feature selection are two methods to process the text document. Therefore, this paper is presented to evaluate the effect of pre-processing and feature selection on the text classification performance. A tweet dataset is utilized and pre-processed using several combinations of pre-processing activities (tokenization, removing stopwords and stemming). Later, two feature selection techniques (Bag-of-Words and Term FrequencyInverse Document Frequency) are applied on the pre-processed text. Finally, Support Vector Machine classifier are used to test the classification performances. The experimental results reveal that the combination of pre-processing technique and TF-IDF approach achieved greater classification performances compared to BoW approach. Better classification performances hit when the number of features is decreased. However, it is depending on the number of features obtained from the preprocessing activities and feature selection technique chose.
format	Conference or Workshop Item
author	Nur Syafiqah, Mohd Nafis Suryanti, Awang
author_facet	Nur Syafiqah, Mohd Nafis Suryanti, Awang
author_sort	Nur Syafiqah, Mohd Nafis
title	The impact of pre-processing and feature selection on text classification
title_short	The impact of pre-processing and feature selection on text classification
title_full	The impact of pre-processing and feature selection on text classification
title_fullStr	The impact of pre-processing and feature selection on text classification
title_full_unstemmed	The impact of pre-processing and feature selection on text classification
title_sort	impact of pre-processing and feature selection on text classification
publishDate	2019
url	http://umpir.ump.edu.my/id/eprint/25395/ http://umpir.ump.edu.my/id/eprint/25395/1/17.%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf http://umpir.ump.edu.my/id/eprint/25395/2/17.1%20The%20impact%20of%20pre-processing%20and%20feature%20selection.pdf
first_indexed	2023-09-18T22:38:58Z
last_indexed	2023-09-18T22:38:58Z
_version_	1777416786287788032

The impact of pre-processing and feature selection on text classification

Similar Items