StructBERT
Youtube search... | ...Google search
- Natural Language Processing (NLP)
- StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding | W. Wang, B. Bi, M. Yan, C. Wu, J. Xia, Z. Bao, L. Peng and L. Si - Alibaba Group Inc.
incorporating language structures into pre-training. Specifically, we pre-train StructBERT with two auxiliary tasks to make the most of the sequential order of words and sentences, which leverage language structures at the word and sentence levels, respectively. As a result, the new model is adapted to different levels of language understanding required by downstream tasks. StructBERT: Incorporating Language Structures Into Pretraining For Deep Language Understanding | W. Wang, B. Bi, M. Yan, C. Wu, J. Xia, Z. Bao, L. Peng and L. Si - Alibaba Group Inc.