DDR4/PCIe4 vs DDR5/PCIe5 for LLMs- I benchmarked them for pre-training. What are your thoughts?

A user benchmarked DDR4/PCIe4 versus DDR5/PCIe5 for pre-training Large Language Models (LLMs) and is seeking community input. The dilemma revolves around whether to invest in DDR5/PCIe5 now, despite potential upgrade complexities with RAM channels and DIMM counts, or to opt for the seemingly more reasonable DDR4/PCIe4. The user is interested in various use cases and opinions regarding recent workstation purchases or upgrades for pre-training, fine-tuning, RL, or inference.

This report is unique as it directly benchmarks DDR4/PCIe4 against DDR5/PCIe5 specifically for LLM pre-training, unlike general hardware comparisons.