Our task is to design a machine reading comprehension (MRC) model that can
accurately solve question answering problems from the Stanford Question Answering Dataset (SQuAD). For our model, we aimed to 1) implement the QANet
model, which is one of the highest performing non-pretrained models, and 2)
extend QANet with a verification module inspired by Zhang et al. (2020) to better
identify unanswerable questions and improve performance on SQuAD 2.0. We
explored variants on both the QANet architecture as well as the Retro-Reader
Architecture experimenting with different values for hyperparameters and our best single model achieved an F1/EM score of 66.10/62.28 on the development set and 64.422/60.659 on the test set. We explored a variant on the Retro Reader architecture that involved training one model to always predict an answer and training a separate model that does all the answerability prediction. Despite not significantly improving the performance of the model, through our error analysis, we gained deep insights into what components degraded model performance and developed potential hypotheses for future improvements. In particular when testing the Retro QANet model, we discovered that the Intensive QANet model was prone to false negatives and false positives thus we hypothesize that the main shortcoming of our model is its reading comprehension ability. Overall, we explored the application of retro reader and verification techniques to one of the highest performing non-PCE models and experimented with parameters and the architecture.
