Artificial Intelligence
Transformer
A deep learning model architecture primarily used in the field of Natural Language Processing (NLP) that relies entirely on attention mechanisms to process data sequences without using recurrence or convolution. / 一种主要在自然语言处理（NLP）领域使用的深度学习模型架构，完全依赖注意力机制来处理数据序列，不使用循环或卷积。
- Knowledge type
- 数学理论
- First proposed or discovered
- Proposer
- Turkey Farm member
- Creator
- 农场主
- Last updated
- 2026-10-22 16:03:30
Sub-knowledge points
Encoder-Decoder ArchitectureFeed-Forward NetworksLayer Normalization
Higher-order knowledge points
Main parameters and performance indicators
Self-Attention: Mechanism to weigh the importance of different parts of the input dataMulti-Head Attention: Parallel attention mechanisms to capture different types of relationshipsPositional Encoding: Information added to input embeddings to indicate the position of tokens in a sequence
Open problems
Computational complexity for very long sequencesInterpretability of attention weightsHandling of out-of-vocabulary tokens
Related literature
ranked by community starsSALiT: A structure-aware lightweight Transformer for multi-task remote sensing interpretationOpen source
Related products and links
BERTGPTVision Transformer (ViT)
Creation and modification trail
1 recordsRecord title: Transformer.
Node update feedback
review required