A linter for PyTorch 'torch-preflight' [P]
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
这条记录涉及编程工具或代码能力更新，适合开发者评估工作流变化和可复用价值。
一款名为“torch-preflight”的新型linter工具已开发完成，旨在识别PyTorch代码中常见的错误，这些错误常导致GPU资源浪费。该工具无需执行代码，也无需GPU或PyTorch安装即可分析PyTorch代码。目前，它能检测13种问题，例如不正确的损失累积、遗漏的zero_grad()调用以及分布式数据并行（DDP）配置错误。…
Been working on this for the last few months. I've been working on PyTorch for the past few years and I always felt, many a times my work went into dump, because of some mistakes I made in the code. torch-preflight reads your PyTorch code and catches the bugs costing you GPU hours.
Things like losses.append(loss), which holds the autograd graph from every step until CUDA dies on you or no zero_grad() in the loop or gradient accumulation without dividing the loss or DDP with no DistributedSampler, so every rank trains on the same batches. I've been able to get 13 rules so far. Your code never gets imported or executed, so you need no GPU and no torch install.
There's another part to this that estimates VRAM. Point the tool at a training script and a GPU, and you learn whether the run fits before you pay for the instance. You also get the list of changes to make the run fit, with the GiB each one saves.
pip install torch-preflight
https://github.com/highwaterlabs/torch-preflight
https://pypi.org/project/torch-preflight/
Please try this out, and I would like to get your feedback! It's still a work in progeress.
Would like to know what breaks on your code. False positives kill a linter, and my only large test target so far has been the PyTorch source tree. Same for the memory numbers. Mine land within 4% of measured peaks, but from four models on one T4.
PS: open to contributions, and issues are already open on the repo. Soon I'm going to add a few "Good first issues" as well. Feel free to ping me if you have any questions!