Tag: RoPE

item-image
Sep, 24 2026

Positional Encodings in Transformers: How LLMs Understand Word Order

Discover how positional encodings enable Transformers to understand word order. Learn the differences between sinusoidal, learned, and RoPE methods.

item-image
Sep, 24 2026

Positional Encodings in Transformers: Why Word Order Matters

Discover how positional encodings enable Transformers to understand word order. Learn the math behind sinusoidal functions, learned embeddings, and RoPE.

item-image
Mar, 6 2026

Architectural Innovations That Improved Transformer-Based Large Language Models Since 2017

Since 2017, transformer-based language models have evolved through key architectural changes like RoPE, SwiGLU, and pre-normalization. These innovations improved context handling, training stability, and efficiency-making modern AI models faster, smarter, and more scalable.