Strand-based Hairstyle Generation via Large Reconstruction and Multimodal Models
Conghui Hao, Tao Huang, Yuefan Shen, Tongtong Wang, Zhongtian Zheng, Kui Wu
cs.GR
2026-08-14
By combining an off-the-shelf Large Reconstruction Model and Large Multimodal Model with classical geometry diffusion, this pipeline reconstructs strand-based 3D hair from one photo in 20 minutes with no training, reaching 0.75 IoU versus 0.64 for the best prior method.
Strand-based 3D hair modeling is expensive and hard to automate. Production pipelines still lean on artists hand-authoring strand flow, which doesn't scale. Learning-based reconstruction methods can go from a single photo to hair geometry, but they typically need large training sets to learn a volumetric orientation-field predictor, then trace strands from that field as post-processing. These methods generalize poorly to structurally complex styles such as ponytails and buns that are rare in training data, and many output implicit representations like neural fields or VAE latents that don't plug directly into strand-level editing or physics simulation. This paper skips training a dedicated hair network, combining an off-the-shelf Large Reconstruction Model (LRM) and Large Multimodal Model (LMM) into a pipeline that needs neither task-specific training nor data collection.
Five stages. An LRM (Tripo 3D in the paper) reconstructs a coarse mesh from a single portrait, aligned to a template bust with a scalp mask; subtracting the bust yields a watertight hair mesh. That mesh is rendered untextured from six viewpoints (front, back, left, right, two top-down angles), and an LMM (Nano Banana 2) generates a grayscale guidance image per view emphasizing local strand flow. A 2D structure tensor on that guidance extracts per-pixel direction, which is projected back onto the mesh surface. Separately, the LMM identifies parting lines and constrained regions such as ponytails and buns in the same rendered views, which segment the surface into loose, gathering, constrained, and scalp regions, semantic cues that constrain global structure without prescribing individual strand geometry.
A three-stage diffuse-correct-diffuse process then propagates direction from the visible surface into the full volume. Stage one fills occluded surface regions by smoothing a sign-invariant tensor field. Stage two resolves the sign ambiguity, since a tensor knows the line but not which way along it, using the semantic regions as priors: hair flows toward the constrained region from gathering areas, away from it inside constrained regions, and along gravity in loose areas. Stage three diffuses the now-oriented surface field into the volume interior using scalp normals as boundary conditions. Strands are traced bidirectionally through the resulting field, with additional strands seeded wherever forward tracing leaves the volume sparsely covered. No network is trained anywhere; LRM and LMM are both interchangeable off-the-shelf models.
Against four open-source baselines (NeuralHDHair, HairStep, DiffLocks, Im2Haircut), measured by silhouette IoU against the input view and orientation error (Ori-Err) against HairStep-extracted directions:
| Method | IoU↑ | Ori-Err↓ |
| NeuralHDHair | 0.52 | 68.4° |
| HairStep | 0.64 | 55.5° |
| DiffLocks | 0.61 | 55.6° |
| Im2Haircut | 0.64 | 45.1° |
| This method | 0.75 | 32.6° |
Against the unreleased HairLRM and HairGPT, the paper compares rendered results on the same input images used in those papers, qualitatively preserving hairstyle shape and local detail better. Ablations show that swapping in HairStep-predicted orientation before volume diffusion fails badly on unseen hairstyles, particularly at gathering and parting regions. Skipping sign disambiguation and diffusing unoriented directions directly causes strands to loop inward toward the scalp. A sampling-density ablation shows 5k volumetric samples make tracing fail outright, while 100k samples show no meaningful gain over the default 10k. The full pipeline runs about 20 minutes per portrait (3 min LRM meshing, 3 min LMM multi-view queries, 3 min semantic processing, 3 min orientation diffusion, 8 min tracing 50k strands), well under Im2Haircut's hour-scale differentiable optimization.
The result argues for an engineering path rather than a modeling one: stitching together general-purpose LRMs, LMMs, and classical geometry processing beats a dedicated trained network on strand-level reconstruction, especially for complex styles underrepresented in training data. For digital-human pipelines, this cuts out collecting a hair dataset and training a bespoke network, replacing it with off-the-shelf reconstruction and vision-language APIs. The paper also shows the output imports directly into Unreal Engine for simulation and stays compatible with standard grooming workflows for artist touch-up. The tradeoff is a hard dependency on two black-box foundation models; the paper doesn't quantify how much quality would swing with weaker or stronger backbones.
The authors state two limitations explicitly. Highly intricate curly or kinky hairstyles remain hard because the interwoven strand topology is difficult to infer reliably from a single image, producing wrong strand flow, missing fine detail, and locally inconsistent geometry; failure cases are shown. Fine-scale strand detail that's only weakly represented on the reconstructed surface is hard to recover faithfully. Proposed next steps are supporting braids and other complex topologies, tighter coupling between geometric reconstruction and semantic understanding, and interactive editing tools. Beyond what's stated, the semantic-partition pipeline (spline merging, region cleanup) relies on several hand-tuned thresholds that the paper only claims are robust across the examples shown, without a sensitivity analysis. The quantitative comparison also covers only four open-source baselines; HairLRM and HairGPT, being unreleased, get only a qualitative single-image comparison rather than a controlled numeric one.