ArTST: Arabic Text and Speech Transformer

Hawau Olamide Toyin, Amirbek Djanibekov, Ajinkya Kulkarni, Hanan Aldarmaki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in these tasks, ArTST performs on a par with or exceeding the current state-of-the-art in all three tasks. Moreover, we find that our pre-training is conducive for generalization, which is particularly evident in the low-resource TTS task. The pre-trained model as well as the fine-tuned ASR and TTS models are released for research use.

Original languageEnglish
Title of host publicationArabicNLP 2023 - 1st Arabic Natural Language Processing Conference, Porceedings
EditorsHassan Sawaf, Samhaa El-Beltagy, Wajdi Zaghouani, Walid Magdy, Nadi Tomeh, Ibrahim Abu Farha, Nizar Habash, Salam Khalifa, Amr Keleg, Hatem Haddad, Imed Zitouni, Ahmed Abdelali, Khalil Mrini, Rawan Almatham
PublisherAssociation for Computational Linguistics (ACL)
Pages41-51
Number of pages11
ISBN (Electronic)9781959429272
Publication statusPublished - 2023
Event1st Arabic Natural Language Processing Conference, ArabicNLP 2023 - Hybrid, Singapore, Singapore
Duration: Dec 7 2023 → …

Publication series

NameArabicNLP 2023 - 1st Arabic Natural Language Processing Conference, Proceedings

Conference

Conference1st Arabic Natural Language Processing Conference, ArabicNLP 2023
Country/TerritorySingapore
CityHybrid, Singapore
Period12/7/23 → …

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'ArTST: Arabic Text and Speech Transformer'. Together they form a unique fingerprint.

Cite this