Traditional Vision Pipeline
Near-sensor Vision Processing
Vision processing on traditional architectures is inefficient due to energy-expensive off-chip data movement. Many researchers advocate pushing processing close to the sensor to substantially reduce data movement, promoting energy efficiency. However, it generates heat increasing sensor temperature, thereby resulting in noisy images potentially degrading task accuracy. The graph below shows the relationship between temperature and image noise.
OUR CHARACTERIZATION
To better appreciate the insights offered by nearsensor processing, we characterize the thermal implications of using 3D stacked image sensors with near-sensor vision processing units. Our characterization reveals that near-sensor processing reduces system power but degrades image quality. For reasonable image fidelity, the sensor temperature needs to stay below a threshold, situationally determined by application needs. Fortunately, our characterization also identifies opportunities – unique to the needs of near-sensor processing – to regulate temperature based on dynamic visual task requirements and rapidly increase capture quality on demand.
The goal of this project is to leverage near-sensor processing to create efficiency benefits while maintaining sufficient image fidelity for vision and imaging tasks. We develop novel mechanisms that can efficiently regulate sensor temperature for continuous and on demand image fidelity needs. In our design, these mechanisms are governed by a runtime controller, which we call Stagioni.
EXECUTIVE FLOW
Stagioni primarily takes the fidelity needs and the ambient environment settings as inputs. While the fidelity needs will be specified by the application developer through our API, Stagioni leverages on-board sensors to derive the ambient settings. For example, Stagoni uses an ambient temperature sensor typically available on mobile phones to derive its value. Along similar lines, it obtains the ambient lighting situation by reading the exposure and ISO values from automatic exposure controller module available in phone cameras.
Adaptive to temperatire
Adaptive to lighting
In addition to fidelity and ambient parameters, Stagioni also leverages on-system performance counters to estimate the application activity information such as the number of memory loads and stores and number of arithmetic instructions. This information is processed against the characterized models, stored in the system memory as look-up tables, to derive power, temperature, and noise trends. These trends along with the fidelity and ambient constraints will constitute different thermal boundaries of the system.
Based on these thermal boundaries, Stagioni analytically determines different policy parameters such as duty cycle and migration frequency. Finally, these policy parameters are fed to appropriate policy controllers – 1) gating controller for stop-capture-go and 2) migration controller for seasonal migration – to put the thermal management mechanisms into action. These two mechanisms quell image quality concerns, while striving to optimize for system power and performance.
Our evaluation shows that our novel dynamic thermal management strategies can unlock the energy-efficiency potential of near-sensor processing. For our evaluated tasks, our strategies save up to 53% of system power with negligible performance impact and sustained image fidelity.