The Pleistocene epoch was a period of dramatic climate change that had profound impacts on the population sizes of many animal species. How these species were shaped by past events is often unclear, hindering our understanding of the population dynamics resulting in present day populations. We analyzed complete mitochondrial genomes representing all four recognized chimpanzee subspecies and the bonobo to infer the recent demographic history and used simulations to exclude a confounding effect of population structure. Our genus-wide Bayesian coalescent-based analysis revealed surprisingly dissimilar demographic histories of the chimpanzee subspecies and the bonobo, despite their overlapping habitat requirements. Whereas the central and eastern chimpanzee subspecies were inferred to have expanded tenfold between around 50,000 and 80,000 years ago and today, the population size of the neighboring bonobo remained constant. The changes in population size are likely linked to changes in habitat area due to climate oscillations during the late Pleistocene. Furthermore, the timing of population expansion for the rainforest-adapted chimpanzee is concurrent with the expansion of the savanna-adapted human, which could suggest a common response to changed climate conditions around 50,000-80,000 years ago.