Ya realize that the sun also plays a major part....
Of course - now you are getting it. After the Maunder Minimum and Little Ice Age, the sun's activity picked up. Given that the oceans are so large and the climate system so vast, it takes time for a cold period to warm back up. If the earth were to continue warming, after the pending cool period, the earth might warm up to the same temperature we had in the Medieval warm period.
It is not CO2 that caused most of the change in temperature but natural changes in the sun and climatic system.