Reconstructing 3D bone geometry from 2 X-ray silhouettes using a statistical shape model + differentiable rendering [P]
A developer is working on a pipeline to reconstruct 3D distal femur geometry from two orthogonal X-ray views (PA + lateral) without using CT scans, neural networks, or large training sets. The most challenging aspect was achieving correspondence, with various methods like KD-tree nearest neighbor (50.7x roughness), CPD (28.2x), BCPD (47.5x), and FilterReg failing to meet the 5x acceptance gate. ShapeWorks was the only successful method, achieving 3.3x roughness. The developer is currently working on real X-ray validation and automatic segmentation.
Working on a pipeline that recovers a patient specific 3D distal femur from two orthogonal X-ray views (PA + lateral). No CT, no neural network, no massive training set.
approach: build a PCA shape model from 50 CT-derived femur meshes (MedShapeNet), then fit it to two silhouettes using PyTorch3D's soft rasterizer with sigma annealing. 10 shape coefficients, Mahalanobis prior to keep things plausible, Adam optimizer, ~1000 iterations.
The part that took the longest (and made me suffer the most too): correspondence. Tried KD-tree nearest neighbor (50.7x roughness vs CT surface), CPD (28.2x), BCPD (47.5x), and FilterReg (couldn't even run). Finally got ShapeWorks working at 3.3x. only method that passed the 5x acceptance gate I set before testing.
LOO validation on 5 held out femurs: 0.86-1.43mm on within range targets. Two extreme cases failed because they sat outside the 49-mesh model's coverage on mode 1, the optimizer can't recover a coefficient the model doesn't support. Bridge ICP alignment was also poor on those cases (0.6 inlier fraction), which accounted for more error than the shape fitting itself.
Interesting finding: the sigma anneal endpoint has to match the reference render's sigma exactly. Hardcoding a constant tuned on one SSM caused an 87x accuracy degradation on another. Tying it to camera_extent × 1e-4 fixed it.
Still working on real X-ray validation (need paired CT data) and automatic segmentation. Happy to answer questions