Efficient Diffusion Models for High-Fidelity Generation under Resource Constraints

Abstract

Diffusion models have rapidly emerged as state-of-the-art generative frameworks for producing high-fidelity images, audio, and multimodal content. However, their practical deployment in resource-constrained environments—such as edge devices, mobile platforms, embedded systems, and low-latency industrial applications—remains challenging due to their significant computational demands, extensive sampling steps, high memory overhead, and energy consumption. This research paper presents a comprehensive investigation into designing efficient diffusion models that deliver competitive generative quality while operating under stringent resource limitations. The work identifies key bottlenecks in traditional diffusion pipelines, including large-scale noise scheduling, iterative denoising complexity, and expensive backbone architectures, and explores algorithmic and architectural innovations to mitigate these constraints. 

The proposed framework integrates three major contributions. First, we introduce a lightweight noise scheduler based on adaptive time-step pruning, which dynamically adjusts the diffusion trajectory to reduce the number of denoising steps without degrading sample quality. This scheduler leverages information-theoretic metrics to maintain model stability, enabling up to 70% reduction in sampling iterations. Second, we design a compact U-Net backbone optimized through depthwise separable convolutions, cross-layer feature reuse, and parameter-efficient attention mechanisms. This architecture achieves substantial reductions in parameter count and memory footprint while preserving the expressive power required for high-fidelity generation. Third, we propose an end-to-end distillation and quantization pipeline that transfers knowledge from a large teacher diffusion model to a smaller student model via consistency distillation, and subsequently applies post-training 8-bit and 4-bit quantization to minimize runtime cost. This two-stage compression strategy proves effective for deployment on edge-class GPUs and modern mobile SoCs.