Broadening SIGIR. Development of successful conversational IR systems will require significant expertise in eliciting, finding, and delivering information, which are core strengths of the information retrieval research community. It will also require user modeling, dialog systems, speech interfaces, and HCI skills that provide opportunities for collaboration with colleagues in other areas of computer science. People tackling this research problem will need to work across disciplines.
Broadening SIGIR. Many questions pertaining to responsible and accountable technology originate in other scientific communities. Often, they are social, ethical, or legal in nature rather than purely technical. We need technical skills to solve them but we should collaborate with social scientists, psychologists, economists and lawyers, e.g., to understand the impact of using FACT IR systems in society, to be exposed to suitable ethical frameworks, and to anchor the definitions of the core concepts in FACT IR, such as what is an explanation in scientific discourses that have considered such notions for decades.
Broadening SIGIR. There is potential for cross-disciplinary collaboration and impact with a number of scientific fields, including psychology, economics, learning sciences, and robotics. In fact, some IR advances described in this report will require interdisciplinary solutions that draw from paradigms and methods in multiple areas.
Broadening SIGIR. Papers on offline evaluation through test collections dominate SIGIR evaluation papers. While such work is important, there are other research challenges to address. Venues like KDD, NIPS, WSDM, and ICML are publishing much work in online evaluation, and SIGIR-focused researchers should stake more of a claim. We have the expertise in large-scale reusable experimental design that will be necessary to harness the full power of these methods for retrieval systems. If we can encourage more IR focused online evaluation research, we hope this will create a bridge between SIGIR and the other more ML focused conferences as well as attracting new SIGIR participants from those communities.
Broadening SIGIR. There are neighbourhood areas, such as Natural Language Processing and Recommender Systems, which suffer from similar issues in terms of explanation and prediction of the performance of their systems. These areas could benefit from an advancement within SIGIR and, at the same time, SIGIR could benefit from teaming up with these areas to jointly address these issues and come to more general and robust solutions.
Broadening SIGIR. Accomplishing the research program will require collaboration among researchers from different disciplinary, theoretical and methodological traditions, e.g. computer scientists, information sci- entists, human-computer interaction researchers, cognitive and experimental psychologists. The SIGIR community needs to ensure that its core venues support the growth of research bridging interactive IR and test collection-based experimentation. There is a great deal of foundational work on methodologies, and that work is best conducted where research ideas are taken note of, in the conferences of record for the community.
Broadening SIGIR. This area draws heavily in the first instance on work done in machine/deep learning and statistical natural language processing, and as such, any activity in this space will naturally lead to stronger connections with these fields through cross-fertilization of ideas and greater visibility for SIGIR research. Beyond this, there are unique characteristics/challenges in IR that we can expect to give rise to methodological breakthroughs with broader implications including: • IR has a very mature understanding of what types of document/collection representation are needed for retrieval (e.g. inverted file indexing, positional indexing, document zoning, document graphs), more so than fields such as speech, NLP and computer vision, and de- veloping representation learning methods that are able to capture these rich data structures will have implications well beyond IR. • IR has decades of experience in assigning, interpreting and learning from document-level relevance judgments; there is considerable scope to transfer this expertise beyond the bounds of IR. • IR has a rich history of multimodality (including images, speech, video, and (semi-)structured data) with well-established datasets, and a relatively mature understanding of how to har- ness that multimodality to draw inspiration from when developing new models. • There is deep knowledge of methods for attaining run-time and storage efficiency in IR, both of which are critical issues in machine/deep learning research at present, and any advances on the part of the IR community would have far-reaching implications beyond SIGIR. • IR expertise in evaluation, especially focusing on the user experience, has the potential to significantly shape research on tasks such as question-answering, summarization and machine reading, where current evaluation practices are narrowly focused on string matching with a gold standard.
Broadening SIGIR. An explicit goal of research with generated information objects (GIOs) is to broaden the scope of the SIGIR discipline. To address open domain information needs, GIOs will be combined from rich and varied generations of new information objects. In the long term, GIOs are a stepping stone towards the synthesis of new information from existing sources. This research effort will fuel a wide range of obvious cross-field collaborations: 1) NLP to understand linguistic representations, summarization, and discourse and dialog, 2) HCI to create user-information interactions that are natural and help the user accomplish her task effectively, and 3) vision and language to enable efficient presentation and interaction when multi-modal information is used as information units. We believe that creating challenge tasks with GIOs in mind will clear up some risks and concerns regarding feasibility and evaluation, and spur increased collaboration within and across the community.
Broadening SIGIR. Several opportunities exist to broaden the IR Community and interact with other communities on emerging efficiency challenges. Specific examples include the embedded / distributed computing community for cross-device search and machine-driven search in IoT devices, the NLP community for conversational IR, the ML community for complex ranking function optimization, the CPM community for future index structures, and the database community for combining structured and unstructured resources and for query optimization ideas.
Broadening SIGIR. There is a lot of related work in other research communities, and opportunities for collaboration as well as starting points for IR research. This includes work on dataspaces, the EU NEPOMUK project on semantic infrastructure for desktop retrieval,3 and work on semantics and retrieval in the lifelogging community, for example in the Lifelogging Tools and Applications workshops. Research in privacy-preserving data linkage and data mining will also be relevant to problems in linking and sharing data.
