Multi3Generation aims to foster an interdisciplinary network of research groups working on different aspects of language generation (LG). We frame LG broadly as the set of tasks where the ultimate goal involves generating language. In contrast to the more classical definition of NLG, this also includes tasks not concerned with LG in an immediate sense, but that could obviously inform or improve LG models.
The Action will focus on the four core challenges:
- Data and information representations: in modern applications, inputs can be different sources such as images, videos, KBs and graphs.
- Machine Learning (ML): modern ML approaches face additional challenges when applied to LG; inputs should be mapped to different correct outputs, i.e. challenges involve e.g. structured prediction and representation learning.
- Interaction: Applications of LG, e.g. Dialogue Systems, Conversational Search Interfaces and Human-Robot Interaction pose additional challenges to LG due to uncertainty derived from the changing environment and the non-deterministic fashion of interaction.
- KB exploitation: structured knowledge is key to many NLP tasks, including NLG, supporting ML methods that require expansion, filtering, disambiguation or user adaptation of generated content. This Action aims to address these challenges by answering the following questions:
- How can we efficiently exploit common-sense, world knowledge and multi-modal information from various inputs such as knowledge bases, images and videos to address LG tasks such as multi-modal machine translation (MT), video description and summarisation?
- How can modern ML methods such as multi-task learning (MTL), representation learning and structured prediction be leveraged for LG?
- How can the models from (1) and (2) be exploited to develop dialogue-based, conversational Human-Computer and Human-Robot interaction methods?
The EU has recently published its Digital Single Market priority Language Technologies and Big Data, which aims to promote the intelligent use and management of data sources in Europe to facilitate the provision of innovative, commercial and social solutions. Our proposed action will look into how different data sources can be aggregated and exploited and how novel ML methods can assist in automating the management of data sources as well as the extraction of important information. In addition to this, Multi3Generation is related to the Automated Translation policy which aims to provide innovative solutions for cross-lingual access to digital services by addressing the translation workload.
Multi3Generation also speaks to the concerns raised in the recent European Parliament resolution on Language Equality in the Digital Age (2018/2028(INI)), which is premised on the necessity of a multilingual orientation in the development of language technologies. NLP, NLG and HCI are also included as part of key strategies for R&D plans in other countries outside the EU. In this manner, The National Artificial Intelligence (AI) Research and Development Strategic Plan, defined in 2016, sees AI as an urgent priority, establishing as its second main strategy the development of effective human-AI collaboration methods, among which the need for NLP systems capable of engaging in real-time dialogue with humans are emphasised. These types of systems may make the interactions between humans and AI systems more natural and intuitive. In China or the United Arab Emirates, the AI Research and Development Strategic Plans, both launched in 2017, provide goals for setting up the next generation key AI systems, including NLP and autonomous technology to apply to a wide range of sectors (education, health, transport, etc.). The Digital Transformation Monitor report about USA-China-EU plans for AI concludes that 70% of the global economic impact of AI will be concentrated in North America and China. Europe cannot lag behind, and Multi3Generation will set up a network of experienced researchers from different European academic institutions, research centres, and companies, who will share their knowledge and foster cutting-edge research and development in the field of AI, able to compete with the US or China. Crucially, this Action incorporate members from leading research-heavy multinational companies in Asia, which can help bridge the knowledge gap between Europe and the rest of the world and also between academia and industry. Several European industries and markets will benefit from the outcomes of this action, including e-commerce (ability to translate multi-modal content and, as a result, reach wider audiences), smart vehicles (through vision and language interfaces), customer service (through automated assistants), media and broadcasting.
Finally, the robotics industry will benefit from this action through the development of innovative Human-Robot Interactions. In fact, the International Federation of Robotics reports that the sales for personal/domestic robots assistants increased by 31% in 2017 (~6.1 million units/year) and this number could reach almost 39.5 million units within the period 2019-2021. The trend for this market is positive, is expected to increase substantially in the future, creating new opportunities and challenges for Human-Robot communication, which will be addressed by this Action.