Obstacles and Risks Clause Samples

Obstacles and Risks. For academics seeking to undertake research in large-scale IR systems there are obvious risks, primarily in regard to achieving genuine scale. Many of the research questions that offer the greatest potential for improvement – and the greatest possibilities for economic savings – involve working with large volumes of data, and hence significant computational investment. Finding ways of collaborating across groups, for example, to share hardware and software resources, and to amortize development costs, is a clear area for improvement. Current practice in academic research in this area tends to revolve around one-off software developments, often by graduate students who are not necessarily software engineers, as convoluted extensions to previous code bases. At the end of each student’s project, their software artifacts may in turn be published to GitHub or the like, but be no less a combination of string and glue (and awk and sed perhaps) than what they started with. Agreeing across research groups on some common data formats, and some common starting implementations, would be an investment that should pay off relatively quickly. If nothing else, it would avoid the ever-increasing burden for every starting graduate student to spend multiple months acquiring, modifying, and extending a code base that will provide baseline outcomes for their experimentation. Harder to address is the question of data scale and hardware scale. Large compute instal- lations are expensive, and while it remains possible, to at least some extent, for a single server to be regarded as a micro-unit of a large server farm, there are also interactions that cannot be adequately handled in this way, including issues associated with the interactions between different parts of what is overall a very complex system. Acquiring a large hardware resource that can be shared across groups might prove difficult. Perhaps a combined approach to, for example, Amazon Web Services might be successful in being granted a large slab of storage and compute time to a genuinely collaborative and international research group. Harder still is to arrange access to large-scale data. Public web crawls such as the Common Crawl can be used as a source of input data, but query logs are inherently proprietary and difficult to share. Whether public logs can be used in a sensible way is an ongoing question. Several prior attempts to build large logs have not been successful: the logs of CiteSeer and DBLP are heavily sk...

Obstacles and Risks. To enable this research we need broad collaborations between IR researchers and communities outside IR. Finding effective ways of collaborating and finding a shared language requires con- siderable effort and investment that may not be properly “rewarded” by funding bodies and evaluation committees. An important risk concerns the diversity of perspectives on the definition of core concepts such as fairness, ethics, explanation or bias across scientific and engineering disciplines, gov- ernments or regulating bodies. Having more transparent IR systems could make systems more vulnerable for adversaries as knowledge about the internals of systems need to be shared through explanations. A potential obstacle is initial resistance from system developers and engineers, who might have to change their workflows in order for systems to be more transparent. Another possible obstacle is the tension between transparency and fairness, and an enterprise’s commercial goals. An inadvertent risk is introducing a new type of bias into our systems about which we are unaware. 4 IR for Supporting Knowledge Goals and Decision-Making 4.1 Description IR systems should support complex, evolving, or long-term information seeking goals such as acquiring broad knowledge either for its own sake or to make an informed decision. Such support will require understanding what information is needed to accomplish the goal, scaffolding search sessions toward the goal, providing broader context as information is gathered, identifying and flagging misleading or confusing information, and compensating for bias in both information and users. It requires advances in algorithms, interfaces, and evaluation methods that support these goals. It will be most successful if it incorporates growing understanding of cognitive processes: how do people conceptualize their information need, how can contrasting information be most effectively portrayed, how do people react to information that flies in the face of their own biases, and so on.