Adipose tissue exhibits a remarkable capacity to expand, contract, and remodel in response to changes in physiological and environmental conditions. Here, we describe recent advances in our understanding of how functionally distinct tissue-resident mesenchymal stromal cell subpopulations orchestrate several aspects of physiological and pathophysiological adipose tissue remodeling, with a particular focus on the adaptations that occur in response to changes in energy surplus and environmental temperature. The study of adipose tissue remodeling provides a vehicle to understand the functional diversity of stromal cells and offers a lens through which several generalizable aspects of tissue reorganization can be readily observed.