Lucky, Henry and Suhartono, Derwin (2022) Investigation of Pre-Trained Bidirectional Encoder Representations from Transformers Checkpoints for Indonesian Abstractive Text Summarization. Journal of Information and Communication Technology, 21 (01). pp. 71-94. ISSN 2180-3862

Preview

PDF - Published Version Available under License Attribution 4.0 International (CC BY 4.0). Download (902kB) | Preview

Abstract

Text summarization aims to reduce text by removing less useful information to obtain information quickly and precisely. In Indonesian abstractive text summarization, the research mostly focuses on multi-document summarization which methods will not work optimally in single-document summarization. As the public summarization datasets and works in English are focusing on single-document summarization, this study emphasized on Indonesian single-document summarization. Abstractive text summarization studies in English frequently use Bidirectional Encoder Representations from Transformers (BERT), and since Indonesian BERT checkpoint is available, it was employed in this study. This study investigated the use of Indonesian BERT in abstractive text summarization on the IndoSum dataset using the BERTSum model. The investigation proceeded by using various combinations of model encoders, model embedding sizes, and model decoders. Evaluation results showed that models with more embedding size and used Generative Pre-Training (GPT)-like decoder could improve the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score and BERTScore of the model results.

Item Type:	Article
Uncontrolled Keywords:	Abstractive text summarization, BERTSum model, BERT Score, GPT-like decoder, ROUGE score
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	College of Arts and Sciences
Depositing User:	Mrs Nurin Jazlina Hamid
Date Deposited:	26 Jul 2022 07:39
Last Modified:	09 Feb 2023 03:05
URI:	https://repo.uum.edu.my/id/eprint/28753

Actions (login required)

View Item