参考

• CUDA Programming Guide

• PTX memory consistency model

• A Formal Analysis of the NVIDIA PTX Memory Consistency Model by Lustig et al., 2019

• CUDA global memory caching documentation

• Dissecting the Turing GPU Architecture through Microbenchmarking (Jia et al., 2019)

• PTX Documentation: Link

• SASS Instruction List: Link

• Roofline Model

• Cache coherence

• PTX memory consistency model

• CUDA global memory caching documentation

• Multiple buffering

• https://mlc.ai/modern-gpu-programming-for-mlsys/chapter_background/index.html

• Introduction to TIRx

• TIRx Layout API

• Building a Tiled GEMM

• Pipelining GEMM with TMA

• Scaling GEMM with Warp Specialization and Clusters

• mlc-ai/modern-gpu-programming-for-mlsys

• MLC.AI Modern GPU Programming for MLSys - Flash Attention 4

• FlashAttention-4: Faster Attention with Asynchrony and Low-precision

• Triton Tutorial - Vector Addition；triton-lang/triton

• Triton Tutorial - Fused Softmax；triton-lang/triton

• Triton Tutorial - Matrix Multiplication；triton-lang/triton