To Overlay or to Customize? Revisiting Architectural Choices in Heterogeneous Systems (🔥📣New Paper & Project🔥📣! )

Abstract

Autonomous Driving Systems (ADS) increasingly rely on diverse deep neural networks to support perception, prediction, planning, and control under strict real-time constraints. FPGA-based heterogeneous computing provides an attractive platform for DNN workloads, but it raises a fundamental deployment question: should the system rely on a flexible overlay architecture, or repeatedly load customized bitstreams optimized for dedicated models, which should be treated as a first-class systems problem rather than a purely architectural one? Overlay-based execution offers fast model switching and better adaptability relying on lightweight instruction or parameter updates, while customized architectures can provide higher model-wise efficiency at the cost of reconfiguration latency and reduced flexibility. However, the boundary between these two design choices remains unclear in realistic ADS scenarios. In this work, we present a systematic study of this trade-off from a deployment-centric perspective, focusing on an autonomous driving scenario. Instead of treating overlay and customized acceleration as isolated design points, we analyze when each approach is preferable under practical conditions, including workload variation, architectural design, reconfiguration latency, and switching frequency. Our analysis shows that overlay-based architecture is more suitable for highly frequent model switching under the state-of-the-art architecture. However, as bitstream reload overhead continues to reduce, customized architectures may become increasingly attractive, especially for workloads with efficiency requirements. Conversely, if overlay architectures become more capable and flexible, they may further expand their advantage over customized architectures. These observations provide design insights for future architectural design, and the optimal deployment strategy will be flipped according to the technique development.

Peipei Zhou

Assistant Professor of Engineering

My research interests include Customized Computer Architecture and Programming Abstraction for Health & AI Applications