SQuAD - Refined Implementation of Contextually Enriching Passage Sequences (SQUAD-RICEPS)

img
Our default project took on the task of SQuAD 2.0 Question Answering using inspiration from an approach described in Christopher Clark's 2017 paper, "Simple and Effective Multi-Paragraph Reading Comprehension". We combine the embedding, encoding, and bi-attention of BiDAF with an additional two layers of self attention. Our findings see an improvement when using a TriLinear attention layer on top of a Multiheaded Scaled Dot Product Self Attention layer. While we had promising results with character embeddings on the dev set, we were unable to refine our implementation of character embeddings to improve our model. We were able to produce an EM score of 59.5 and an F1 score of 62.7 which improved on the BiDAF baseline's score of 56.3 and 59.4.