Some buildings, including embassies and customs houses, do not need any workers and do not have any obvious effects on their surroundings. Does it matter at all where I place them?
Embassies can be placed anywhere, supposing they are initially close enough to a road/construction office to be built in the first place. I place them in areas that are not very desirable so I'm not wasting prime real estate.
Custom houses do take 4 illiterate people to run them, so they matter only as much as it matters that you have housing nearby for the employees to live in.