Corpora and Repository

Training Datasets

WG2 / Datasets from the COST members

Dataset name and brief description, including purposeAuthors/creatorsLink
MSVD-Turkish: The first large scale video captioning dataset for Turkish languages, obtained by carefully translating the English descriptions of the videos in the MSVD (Microsoft Research Video Description Corpus) dataset into Turkish.Begum Citamak, Ozan Caglayan, Menekse Kuyu, Erkut Erdem, Aykut Erdem, Pranava Madhyastha, and Lucia Specia.MSVD-Turkish

WG4 / Data-to-text NLG training datasets

Data-to-text NLG systems require training data. Here we provide a list of freely available datasets that have been created with different methodologies (automatically, crowdsourcing etc.) and for different NLG sub-tasks.

WebNLG 2017Gardent, C., Shimorina, A., Narayan, S., & Perez-Beltrachini, L. (2017). Creating Training Corpora for NLG Micro-Planners. ACL.2017
WebNLG 2020Gardent, C., Shimorina, A., Narayan, S., & Perez-Beltrachini, L. (2017). Creating Training Corpora for NLG Micro-Planners. ACL.2020
KBGenBanik, E., Gardent, C., & Kow, E. (2013). The KBGen Challenge. ENLG.2013
E2E NLG ChallengeDusek, O., Novikova, J., & Rieser, V. (2020). Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge. Comput. Speech Lang., 59, 123-156.2017
MultiWOZ 2.2Zang, X., Rastogi, A., Zhang, J., & Chen, J. (2020). MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines. ArXiv, abs/2007.12720.2020
ToTToParikh, Ankur P., et al. “Totto: A controlled table-to-text generation dataset.” arXiv preprint arXiv:2004.14373 (2020).2020 
RotoWireWiseman, Sam, Stuart M. Shieber, and Alexander M. Rush. “Challenges in Data-to-Document Generation.” Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017.2017
WikiBioLebret, Rémi, David Grangier, and Michael Auli. “Neural text generation from structured data with application to the biography domain.” arXiv preprint arXiv:1603.07771 (2016).2016 
Logic2TextChen, Zhiyu, et al. “Logic2Text: High-Fidelity Natural Language Generation from Logical Forms.” arXiv preprint arXiv:2004.14579 (2020).2020 
DARTNan, Linyong, et al. “Dart: Open-domain structured data record to text generation.” arXiv preprint arXiv:2007.02871 (2020).2020 
ENT-DESCCheng, Liying, et al. “ENT-DESC: Entity Description Generation by Exploring Knowledge Graph.” arXiv preprint arXiv:2004.14813 (2020).2020 
GEM (Generation, Evaluation, and Metrics)Gehrmann, Sebastian, et al. “The gem benchmark: Natural language generation, its evaluation and metrics.” arXiv preprint arXiv:2102.01672 (2021).2021 

Skip to content