Apparently your iPhone and iPad stores a hidden, unencrypted, file on the device and backs it up to your Mac — in this file is location data on where you have been. This is both surprising and not that surprising all at the same time, some clever guys did this:

This open-source application maps the information that your iPhone is recording about your movements. It doesn’t record anything itself, it only displays files that are already hidden on your computer.