The shield icon - "Armour" you've called it - is actually a city's combat rating. As with military units1, the combat rating represents both the offensive and defensive capabilities of city. Therefore, the Walls of Babylon provide +6 to the city's base combat rating.
And yes, this will stack with any other boosts to a city's combat rating. In general, when it comes to stats modifiers in the game, all direct + or - modifiers are applied to the base value first, then all % modifiers are summed and applied as a single multiplier applied (so +20% and +30% is applied as +50%, not a compounded +56%).
1. Except for ranged units that have a ranged rating as well - used for attack - while the combat rating for them is defense only.