This is somewhat of a broad question. There is more obvious horde presence on Kalimdor (Orgrimmar, the capital, and Thunder Bluff are both located there), but there is an Alliance presence as well (Darnassus and sort of the Exodar). The continents are designed such that there is a clear leveling path (with a few branches) on either continent regardless of faction. That is, you can level an orc on Eastern Kingdoms and have a full questing experience just as easily as you could level a human on Kalimdor. Your map should show a level range above each area indicating the level of content there.
As a dwarf, you could find friendly faces in the night elf areas (Darkshore, Ashenvale, Stonetalon, etc.), although after Darkshore the territories will be "contested" if you are on a PvP server.
Lorewise this is somewhat of a different question, with several races not even being native to Azeroth.
As for which continent players of a given faction prefer to hang around on, that is also difficult to answer. I would assume Horde players would have a bias towards Kalimdor (given Orgrimmar's location) and Alliance players towards Eastern Kingdoms (for Stormwind and Ironforge).