Summary The circadian clock drives rhythmic physiological variations, making accurate clock-phase measurement essential. Single-sample phase inference from bulk RNA is an exciting alternative to time-series methods, but its applicability to single cells remains unclear. Using multiplexed smFISH to quantify up to six core-clock genes in mouse fibroblasts, combined with a Gaussian process-based algorithm, we show that even with minimal technical dropouts, transcriptional noise in core-clock genes prevents reliable single-cell phase estimation. Simulations predict that measuring ∼50 low-noise, clock-like oscillatory genes is required for accurate single-cell inference. In scRNA-seq data, non-core genes were too noisy, and adding hundreds of them did not improve accuracy. Remarkably, averaging smFISH counts from only three core-clock genes across ∼70 cells enabled robust phase estimation. In desynchronized fibroblasts, spatially heterogeneous phases indicating local inter-cellular coupling were revealed only upon averaging, not at single-cell resolution. Coarse graining is, therefore, essential for circadian phase inference from RNA measurements alone.