Smart Review
SMART Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization
Recent posts
Review “Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing”
SAINT: Separated Self-AttentIve Neural Knowledge Tracing
Review “ELECTRA: PRE-TRAINING TEXT ENCODERS AS DISCRIMINATORS RATHER THAN GENERATORS”
ELECTRA: PRE-TRAINING TEXT ENCODERS AS DISCRIMINATORS RATHER THAN GENERATORS
Review “Attention Is All You Need”
Attention Is All You Need Attention Is All You Need 서론 BERT, ELECTRA를 접하면서 Transformer라는 단어를 많이 접했다. Transformer를 여러번 배웠지만 정확하게 알고 있지 않은 느낌이 계속 들어서 Transform...
Seq2Seq with attention
Sequence to Sequence (Seq2Seq) Sequence를 Encoding와 Decoding하는 작업 Sequence to sequence는 encoder와 decoder로 이루어져 있는 framework으로 대표적인 자연어 처리 architecture 중 하나