Introduction
This demonstration uses SQuAD (Stanford Question-Answering Dataset). In SQuAD, an input consists of a question, and a paragraph for context. The goal is to find the span of text in the paragraph that answers the question. We evaluate our performance on this data with the "Exact Match" metric, which measures the percentage of predictions that exactly match any one of the ground-truth answers.
We fine-tune a BERT model to perform this task as follows:
Feed the context and the question as inputs to BERT.
Take two vectors S and T with dimensions equal to that of hidden states in BERT.
Compute the probability of each token being the start and end of the answer span. The probability of a token being the start of the answer is given by a dot product between S and the representation of the token in the last layer of BERT, followed by a softmax over all tokens. The probability of a token being the end of the answer is computed similarly with the vector T.
Fine-tune BERT and learn S and T along the way.
References:
链接地址：https://keras.io/examples/nlp/text_extraction_with_bert/