Most vision-based sensing and localization infras- tructure today employ conventional area scanning cameras due to the high information density and cost efficiency offered by them. While the information-rich two-dimensional images provided by such sensors make it easier to detect and classify traffic objects with the help of deep neural networks, their accurate localization in the three-dimensional real world also calls for a reliable calibration methodology, that maintains accuracy not just during installation, but also under continuous operation over time. In this paper, we propose a camera calibration architecture that extracts and uses corresponding targets from high definition maps, augment it with an efficient stabilization mechanism in order to compensate for the errors arising out of fast transient vibrations and slow orientational drifts. Finally, we evaluate its performance on a real-world test site.