As per Rick Steve's request, we took a side trip from Madrid to the historic town of Toledo, Spain. While it would have been a great 2-3 hour side trip that I would recommend to all, not so much for 3+ hours. Of course, with our lack of planning this trip (unusual for us), our trip didn't go according to plan.
This was the first trip where we did not look at train times in advance, figuring out when we needed to get there and buy tickets (since we had not already purchased them - mistake). We went to bed the night before and decided to try to leave at 10am. At 10:15, we get on the metro heading toward the train station. When we get to the train station (since we were unorganized and had not already purchased tickets), we search for the machines to sell us tickets (the act of buying tickets on the machine takes all of 2 minutes). After trying multiple ticket machines, we were still left without tickets. Turns out, we could NOT get tickets to Toledo on the ticket-dispensing machines - weird? So I ask a man at the train station, in Spanish for he speaks no English, if we have to buy them at the ticket office (where 57 people are in line in front of us). Unfortunately, yes. We take a number and wait... and wait... and wait.. An hour later, we have tickets, but have to sit and wait another our for the train (again, we should have planned this in advance and saved all the wasted time). The other bad news is, with the train times, our only option is to spend 5 hours there which I know is going to be a bit much (and alas, I was correct). Oh well. After some lunch, beers, and time-wasting, we finally get on the train to Toledo.
Toledo itself is very stereotypical small city Spain, at least in my eyes. All the buildings are sand colored & ancient. There is not a ton to see/do there. I enjoyed walking around aimlessly without a map in the small winding streets lined with interesting architecture, just seeing what you see. Other things to see/do include the Cathedral, the Alcazar, the Synagogue, and most importantly, the San Martin Bridge over the River Tagus (which was gorgeous).
While I would recommend taking a side trip to Toledo if you have a few days in Madrid, I would not recommend staying for more than 3 hours.. OR not planning your trip out in advance (knowing the train times and pre-purchasing your tickets is a must). Toledo is the perfect 2-3 hour small town break from the busy big-city life of Madrid.