If your bounty is not excessively high (In the neighborhood of 5 thousand gold, I think), guard will not be immediately hostile, rather they will recognize you and, if you talk to them, give the usual options of paying the fine, going to jail, resisting arrest, etc. So if you only wish to have a bounty of 1000 gold, you will likely be able to slink by most guards.
If a confrontation is unavoidable, daggers and bows make no noise (save for a few bugged bows), so you should be able to eliminate particularly eager guards without attracting attention, if you have the perks and the inclination.
Calm spells of the Illusion school of magic are also useful to buy yourself some time, however if that is your choice you would do well to get the main quest over with in short order, as the guards' level scaling with yours will make them harder to calm without spending a lot of time increasing your Illusion skill and getting the high level spells. Muffle and Invisibility will also prove useful for eluding their attention.
Some quests and questlines, however, will become impossible to complete, notably the Imperial route of the Civil War questline, during the Battle for Whiterun battle you will be required to speak to an imperial general, who will try to arrest you if you have a bounty. The Stormcloaks storyline is likely to have a similar encounter.
