I love ruins.
It's probably because of dungeon delving and D&D (AD&D) that got me going on decaying monuments, though I'm pretty sure it's something that I was into way before all of that.
There's just something about old, decrepit stone temples, fortresses and statues that really rock my boat. It could be because of the intense amount of effort or time required to build such things (before industrial power), I don't know. It could just be scale.
I've never been to a huge city like New York, so I'm not sure if I'd get the same thrill from seeing those modern buildings too. However, I'm not quiet as enamored with modern architecture as I am with the ancient.
That said, this concept art knocks my socks off. (Sorry, source unknown.)
