I really want to get to the next agency mission and open new black-market items quicker but usually causing chaos (Blowing up gas containers and 'sploding cars) doesn't really make it go up fast. And just shooting government objects with normal weapons takes a really long time to 'splode them and I'm bad at getting to mounted guns and bazookas because I suck at getting past guards. But even when I do get to them and blow stuff up, my chaos bar just goes up only a little more up than it usually does. Are there any ways to make it go up faster that are fairly easy to do?
As mentioned, the side missions are usually a good way to do this; most are quite short and have okay bonuses.
Alternately, find the military bases and destroy the objects you can, then run to the next. I recommend first finding a helicopter and exploring a bunch of locations (you'll need to find several military harbors/bases/airports) in one go, then using your Black Market extraction option to drop you off at the bases one by one (preferably at bases near other military installations).
Bases and harbors usually have a Helicopter you can steal to rapidly destroy all destructable structures and move on to the next location. Airports always have an airplane and almost always a chopper. Prefer the choppers with guns, as they easily destroy anything, even Cranes and large objects your normal guns won't take out. If there's no attack chopper, steal a normal one, take it to the next base and try and find a helicopter with guns.
If you don't get a chopper at a base, do the damage you can and either die & re-extract to a new location, or escape and buy a helicopter from the Black Market. I pretty quickly unlocked the whole story doing this, and found the story part of the game quite short.
Do the various faction side missions. Depending on the level you are, you can also buy explosives that will allow you to blow up large structures (fuel tanks, missile silos, etc) very easily. The best place to gain chaos quickly is the enormous military base on the crescent-shaped island in the bay - a fighter plane, helicopter, or even running around with a turret is extremely effective.