It removes 36000, or 10 radiation a second for 3600 seconds, which would last for an hour, but is that in-game hour, or a real-life hour?
How much time does it give 5 strength and 50 damage resistance?
The mysterious serum, as you've said in the question, removes 10 rads per second for 3600 seconds. This is real-time seconds, not in-game seconds.
See this post:
Mysterious Serum, which gives you -10 Rads per second for an hour(irl time, unless you fast travel).
As for the strength and damage resistance buffs, this post says that it lasts as long as the radiation recovery.
So does the +strength/damage resist last the full 3,600 seconds?
Yeah, it does.