Do you place RCS in symmetric pairs/groups relative to COM?
To imagine the issue, let's take a very simple craft that has only one RCS block, somewhere off to the side.
When you want to turn this craft any way using the RCS, there is nothing to offset the thrust. The craft will begin turning as requested, but it will also be accelerated in the direction opposite to the one RCS fires.
Also, KSP tends to fire all RCS at full thrust, so - say, you placed three thrusters in a 3x symmetry.
You want to pitch the craft's nose up. The one top thruster fires retrograde, the two bottom thrusters fire prograde. Net torque makes the craft pitch up. Net thrust - accelerates it forward; two thrusters forward, just one back.
Small asymmetries result in smaller errors, but they do result in errors nevertheless.
First, the closer it is to COM, the lower the torque it can exert; the more thrust it must put into rotating the ship.
And the more thrust it uses, the more all errors accumulate.
To find the optimal placement, you can use the RCS Build Aid mod, or just 'eyeball' it with help of the editor's symmetry.
Also, try to place the thrusters meant for rotation as far from CoM as you can without sacrificing symmetry. Or just disable RCS pitch/yaw/roll actuation controls and use only the reaction wheels for that.
One more thing: completely regardless of RCS, two crafts WILL drift relative to each other in orbit. After all, if they have exactly the same speed but are not in exactly the same spot, they are in two different orbits. If craft A is in a perfectly circular orbit, and craft B is moving at the same speed, but 50 meters farther from the planet, it's currently exceeding the speed proper for circular orbit proper for its altitude; it's at the periapsis of an elliptical orbit which will take it a good bit away from craft A - out towards the apoapsis and backwards as climbing, it loses orbital speed. This is completely regardless of RCS, and while unnoticeable in a short time period, in the 10 minutes it takes to move to the opposite side of Kerbin it can easily accumulate to 10m/s.