Token Span Prediction Sample Clauses

Token Span Prediction q For every Ei, the pair (Eqj , Eij) is fed into the multi-head attention layer, MHA, where Eqj = Eq \ {ec} and Eij = Ei \ {ec}. MHA [27] then generates the attended m i i i1 in i embedding sequences, Ta, . . . , Ta , where Ta = {ts, tw , .., tw }. Finally, each Ta i i is fed into two softmax layers, SL and SR, that generate oA ∈ Rn+1 and or ∈ Rn+1 to predict the leftmost and the rightmost tokens in Ui respectively, that yield the answer span for Q. It is possible that the answer spans are predicted in multiple utterances, in which case, the span from the utterance that has the highest score for the utterance ID prediction is selected, which is more efficient than the typical dynamic programming approach.