Common use of Annotation Process Clause in Contracts

Annotation Process. To carry out the annotation work, we recruited five Dutch archaeology students at the Bachelor level. We specifically selected students in their second and third year, as some basic knowledge of archaeology is extremely helpful in determining whether a word is a specific entity or not. The students were asked to annotate a total of 16 hours each, over a two week period, during which they could come and work at times that suited them, a few hours at a time. We opted not to have the students work a whole day on this task, as the annotation process is tedious and monotonous, which makes it hard to keep concentration. Loss in concentration can cause mislabelling, and so having them work for only small amounts of time might help prevent this. The students were first asked to thoughtfully read the guidelines and ask any questions. During annotation, we were always present to resolve difficult sentences and entities and explain to the students how to handle these. The students reported this to be very helpful, and learned from each other’s prob- lems. Most of these issues were relatively rare edge case though, and the original annotation guidelines covered most encountered entities sufficiently.