I bought a new Xbox One wireless controller from here. The problem is that when I press either of bumper buttons I feel (little but observable) outward pressure on other bumper, as if they are internally connected. As this is the first time I have bought a Xbox Controller I want to know is this normal for Xbox One controller or not?
Yep, don't worry, this is completely normal for this controller. In between the left and right bumper is a long, thin cable connecting the two and it's the reason you can feel one bumper being depressed while touching the other bumper.
The controller was defective and now it has been replaced and there is no such problem with new one.