You're not crazy. Direct-fire towers use line-of-sight (LoS) to determine if they can attack an enemy or not (and of course range).
The only towers unaffected are the true Area of Effect (AoE) towers (not inferno), and the indirectly firing Meteor tower, here is a brief summary:
Direct fire: Cannon, Gun, Inferno, Laser, Missile*, Telsa
Indirect fire: Meteor, Missile*
*Given that the Missile tower fires seeking missiles (but appears to initially target using LoS), it could be considered to be to be both
Area of Effect Damage: Concussion
Area of Effect Other: Temporal, Command
You'll usually see AoE towers on the second row (being unaffected by LoS), and Indirect fire towers more distanced from the killing zone. There are, however, some good non-AoE second row candidates:
The cannon is great on the second row because it fires high damage (but slowly), taking advantage of momentary visibility. A brief loss of LoS is also often moot, as it will usually be reloading (which it spends most of its time doing).
The laser is also good on the second row, as it continues to cause lingering heat damage for a short while after hitting the target (you'll see them glowing), meaning the LoS interruption is mitigated slightly.