AnchiBERT: A Pre-Trained Model for Ancient Chinese Language Understanding and Generation.

Image credit: Unsplash

Abstract

Ancient Chinese is the essence of Chinese culture. There are several natural language processing tasks of ancient Chinese domain, such as ancient-modern Chinese translation, poem generation, and couplet generation. Previous studies usually use the supervised models which deeply rely on parallel data. However, it is difficult to obtain large-scale parallel data of ancient Chinese. In order to make full use of the more easily available monolingual ancient Chinese corpora, we release An-chiBERT, a pre-trained language model based on the architecture of BERT, which is trained on large-scale ancient Chinese corpora. We evaluate AnchiBERT on both language understanding and generation tasks, including poem classification, ancient-modern Chinese translation, poem generation, and couplet generation. The experimental results show that AnchiBERT outperforms BERT as well as the non-pretrained models and achieves state-of - the-art results in all cases.

Huishuang Tian
Huishuang Tian
M.Sc Student

My research interests include NLP

Kexin Yang
Kexin Yang
Ph.D. Student

My research interests include Neural language generation and Non-autoregressive translation

Dayiheng Liu
Dayiheng Liu
Ph.D. Student

My name is Dayiheng Liu (刘大一恒).

Jiancheng Lv
Jiancheng Lv
Dean and professor of Computer Science of Sichuan University

My research interests include natural language processing, computer vision, industrial intelligence, smart medicine and smart cultural creation.

Related