{"@context":"https://schema.org","@type":"CreativeWork","@id":"https://forgecascade.org/public/capsules/3f4a48f3-2371-4101-ae65-8a3207a95e52","identifier":"3f4a48f3-2371-4101-ae65-8a3207a95e52","url":"https://forgecascade.org/public/capsules/3f4a48f3-2371-4101-ae65-8a3207a95e52","name":"Variance Reduction for Expectations with Diffusion Teachers","text":"# Variance Reduction for Expectations with Diffusion Teachers\n\n**Authors:** Jesse Bettencourt, Xindi Wu, Matan Atzmon, James Lucas, Jonathan Lorraine\n**arXiv:** https://arxiv.org/abs/2605.21489v1\n**Published:** 2026-05-20T17:59:52Z\n\n## Abstract\nPretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most from amortized reuse; ~25% additional from IS+stratification) without changing the objective; in single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck.","keywords":["cs.LG","cs.AI","cs.CV","stat.CO","stat.ML"],"about":[{"@type":"Thing","name":"Bronchial diverticula"},{"@type":"Thing","name":"nuclear-transcribed mRNA catabolic process, meiosis-specific transcripts"},{"@type":"Thing","name":"spinal muscular atrophy, infantile, James type"},{"@type":"Thing","name":"Terminal Services DLL"},{"@type":"Thing","name":"vSphere Installation Bundles"},{"@type":"Thing","name":"Office Template Macros"}],"citation":[],"isPartOf":{"@type":"Dataset","name":"Forge Cascade Knowledge Graph","url":"https://forgecascade.org"},"publisher":{"@type":"Organization","name":"Forge Cascade","url":"https://forgecascade.org"},"dateCreated":"2026-05-21T06:00:06.165000Z","dateModified":"2026-05-21T06:00:06.165000Z","isBasedOn":"https://arxiv.org/abs/2605.21489v1","additionalProperty":[{"@type":"PropertyValue","name":"trust_level","value":65},{"@type":"PropertyValue","name":"verification_status","value":"source_linked"},{"@type":"PropertyValue","name":"evidence_level","value":"primary_source"}]}