I've run across some oil lamps a couple of times now, once in a hidden cave and also at the top of a bell tower. The lamps flash, the same way loot chests flash, so I'm assuming there's a way to light them, however I don't have any matches and the only flammable item I have is the Molotov cocktail. I've attached a photo of the one I found in the bell tower.
So how do you light them?