Multi-task, Multilingual, Multi-modal Language Generation
Multi3Generation

WG 2 – Efficient Machine Learning algorithms, methods, and applications to language generation

WG2 focuses on efficient ML machinery behind state-of-the-art Language Generation (LG) models, since much of the improvements we observe across different LG tasks nowadays are due to the application of varied deep neural network architectures. This involves, among others:

  • multi-task learning
  • transfer learning
  • representation learning
  • structured prediction
  • generative models

Moreover, WG2 investigates integration strategies for multi-modal data, which is of critical importance for Multi3Generation.

This working group will be working on:

  • a repository of open source software for processing language and visual content, inc. a directory of sources of materials or components
  • a survey on efficient use of ML for LG
  • the organization of training schools

If you would like to join this working group, please contact the WG Leader and coleader: Aykut Erdem (aykut-at-cs.hacettepe.edu.tr) and Elena Lloret (elloret-at-dlsi.ua.es)

Publications and preprints

Journal Articles

Conference Papers

  • Lloret, E., Barreiro, A., Bhatt, M., Bugarín-Diz, A., Modoni, G. E., Silberztein, M., … & Erdem, A. (2023). Multi3Generation: Multitask, Multilingual, and Multimodal Language Generation. _Open Research Europe_, _3_. https://open-research-europe.ec.europa.eu/articles/3-176
  • Anabela Barreiro, Elena Lloret, Oleksii Turuta. An Outlook on Natural Language Generation. Position Papers of the 18 th Conference on Computer Science and Intelligence Systems pp. 27–34. DOI: 10.15439/2023F3591, ISSN 2300-5963 ACSIS, Vol. 36
  • Martínez-Murillo, I., Sepúlveda-Torres, R., Saquete Boró, E., LLoret, E., & Palomar, M. (2023). Team GPLSI at AuTexTification Shared Task: Determining the Authorship of a Text. Iberlef, 2023, SEPLN 2023.
  • E. Lloret, B. Méndez, P. Moreda, M. Palomar. ADAPT@: A MULTILINGUAL AND MULTIMODAL ONLINE WEB APPLICATION TO PERSONALIZE TEXTUAL INFORMATION PROCESSING AND VISUALIZATION. 17th annual International Technology, Education and Development Conference (INTED). Valencia (Spain) – 6th, 7th and 8th of March, 2023.
  • Grazina Korvel, Alkiviadis Katsalis, Konstantinos Diamantaras, Elena Lloret. Enrich Knowledge Graphs and Test Pre-trained Language Models in Graph2seq Tasks. The 13th Conference “Data Analysis Methods for Software Systems” December 1 – 3, 2022, in Druskininkai, Lithuania
  • M. Sercan Amac , Semih Yagcioglu , Aykut Erdem, and Erkut Erdem. “Procedural Reasoning Networks for Understanding Multimodal Procedures“, In 23rd Conference on Computational Natural Language Learning (CoNLL).
  • J. Phang, I. Calixto, PM. Htut, Y. Pruksachatkun, H. Liu, C. Vania, K. Kann, SR. Bowman (2020). English intermediate-task training improves zero-shot cross-lingual transfer too. In AACL 2020.
  • V. Milewski , MF. Moens and I. Calixto (2020). Are scene graphs good enough to improve image captioning? In: AACL 2020.
  • Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Madhyastha, Erkut Erdem, Aykut Erdem and Lucia Specia. Cross-lingual Visual Pre-training for Multimodal Machine Translation, Conference of the European Chapter of the Association for Computational Linguistics (EACL 2021), Short Papers, 2021.
  • Rob van der Goot, Ahmet Üstün, Alan Ramponi, Ibrahim Sharaf and Barbara Plank. Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP. In EACL 2021. Received EACL 2021 oustanding paper award (demo track)
  • Ilker Kesen, Ozan Arkan Can, Erkut Erdem, Aykut Erdem, Deniz Yuret, “Modulating Bottom-Up and Top-Down Visual Processing via Language-Conditional Filters”, 5th Multimodal Learning and Applications Workshop (MULA 2022) – in conjunction with CVPR 2022 (Best Paper Award), New Orleans, USA, June 2022.
  • Anabela Barreiro, José GC de Souza, Albert Gatt, Mehul Bhatt, Elena Lloret, Aykut Erdem, Dimitra Gkatzia, Helena Moniz, Irene Russo, Fabio Kepler, Iacer Calixto, Marcin Paprzycki, François Portet, Isabelle Augenstein, Mirela Alhasani, “Multi3Generation: Multitask, Multilingual, Multimodal Language Generation”, 23rd Annual Conference of the European Association for Machine Translation (EAMT 2022), Ghent, Belgium, June 2022.
  • Tayfun Ates, Muhammed Samil Atesoglu, Cagatay Yigit, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, Deniz Yuret, “CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions“, Findings of the Association for Computational Linguistics (ACL 2022), May 2022.
  • Parcalabescu, L., Cafagna, M., Muradjan, L., Frank, A., Calixto, I. and Gatt, A., 2022, May. VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 8253-8280).
  • Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, Andre Martins. Quality-Aware Decoding for Neural Machine Translation  Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL). July 2022.
  • Ricardo Rei, Ana C Farinha, José G.C. de Souza, Pedro G. Ramos, André F.T. Martins, Luisa Coheur, Alon Lavie. Searching for COMETINHO: The Little Metric That Could. ​​In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pages 61–70, Ghent, Belgium. European Association for Machine Translation.
  • Ilker Kesen, Aykut Erdem, Erkut Erdem, Iacer Calixto. “Detecting Euphemisms with Literal Descriptions and Visual Imagery”, The Third Workshop on Figurative Language Processing (FigLang 2022), in conjunction with EMNLP 2022.
  • D. Dashenkov, K. Smelyakov and O. Turuta, “Methods of Multilanguage Question Answering,” 2021 IEEE 8th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T), 2021, pp. 251-255.
  • Selim Fekih, Nicolo’ Tamagnone, Benjamin Minixhofer, Ranjan Shrestha, Ximena Contla, Ewan Oglethorpe and Navid Rekabsaz, “HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crises Response” In Findings of the 2022 Conference on Empirical Methods in Natural Language Processing (Findings of EMNLP), December 2022.
  • Benjamin Minixhofer, Fabian Paischer, Navid Rekabsaz, “WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models” In proceeding of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), July 2022.

Datasets

Dataset name and brief description, including purposeAuthors/creatorsLink
MSVD-Turkish: The first large scale video captioning dataset for Turkish languages, obtained by carefully translating the English descriptions of the videos in the MSVD (Microsoft Research Video Description Corpus) dataset into Turkish.Begum Citamak, Ozan Caglayan, Menekse Kuyu, Erkut Erdem, Aykut Erdem, Pranava Madhyastha, and Lucia Specia.MSVD-Turkish

Code 

Name and/or brief descriptionAuthors/creatorsLink
Procedural Reasoning NetworksAmac, M. Sercan, Yagcioglu, Semih, Erdem, Aykut, and Erdem, Erkuthttps://github.com/hucvl/prn

Open source repository

Neural Natural Language Generation (GitHub)

Other outputs

Add anything here that doesn’t clearly fall under the other headings.

CA18231 Meeting

WG2-progress-video-presentationDownload
WG2 Meeting Summary_ Lisbon (Virtual), December 2020Download

Skip to content