Profiling in PyTorch (Part 2): From nn.Linear to a Fused MLP

This BuzzRadr summary, "Profiling in PyTorch (Part 2)," details how nn.Linear replaces a hand-written matmul-add pair. It explains that nn.Linear folds bias addition into the matrix multiplication kernel using an epilogue, avoiding separate add operations. The summary also notes that torch.compile offers minimal fusion benefits for a single nn.Linear operation because the underlying cuBLAS GEMM kernel already handles bias addition efficiently.