As you mentioned, playstyle does have a lot to do with it. I did see a lot of potential in the endurance perks, but I passed since I wanted an int based character to explore all the modifications to weapons and armor. I stayed at one endurance until level 40.
So yeah, you can't fast travel in survival. I eventually realized that I should find a way to minimize my manual travel time and I did it with the following.
Base SPECIAL: Endurance for longer sprint times, Agility for more AP.
Moving Target, (rank 3): Reduces AP spending on sprint.
Action Boy/Girl: Faster AP regen.
Science!, (rank 3): Built the power armor leg mod for higher AP spent on sprinting but you go faster (overdrive servos).
Chemist: Jet fuel. If you get your charisma up (or have good charisma gear) and make grape mentats when you can, buying out vendors of flamer fuel and fertilizer to make Jet Fuel not only helps you get to places quickly, but it sells for quite a bit. Being in need of caps is kind of an abstract concept when you're a drug dealer.
Nuclear physicist; Scrounger (if you're sporting power armor): You'll get more out of your cores and you'll find more of them with scrounger. If you go without power armor, I'd suggest two leg pieces with 10% faster run speed.
Scrapper: Some of the advanced guns and armor give you a slew of raw materials. When in an area with a weapon or armor station, you can loot everything you can and break it down to carry back.
Strong back: Wasn't happy with rank three. The first two are good.
Note on Endurance: At 96 I still only have three endurance (and three life giver). This appears to be plenty for reducing sprint AP considering the other perks I've invested in. More couldn't hurt, though, and the perks are good.