OXM: Apparently you can buy horses in Skyrim. I've not had this explained to me in-game, and I haven't been in a stable since I got rumbled robbing the one outside Windhelm. So my travel has been on foot or, more often, using the magical power of fast travel.
After a while, I decided to find out what I was missing out on, and just took one from outside the stable in Riften. Slightly to my surprise, nobody seemed to notice this, and I rode off feeling like I had got one over on The Man.