For simplicity's sake I'm going to assume you have B set to special.
It looks like you have the right idea. The trick is going to be pressing Up-B and then immediately holding the stick directly sideways.
Unfortunately, it seems that Smash Bro.'s is still not supported for Video Gameplay Capture. Because of that (and my lack of a capture card), the best I'm only going to be able to show is stills of her recovery (as opposed to fancy gifs).
Here's where I'm standing:
And here's where I am at the end of Up-B when I'm holding left, nothing and right respectively (click for larger image)1:
As you can see by comparing my starting position and the leftmost picture in the above series, going straight up with the recovery is possible! The secret here is to make sure that you're flicking the stick perfectly sideways. If you're at all angling up or down, you won't end up going directly upwards. Here is an example of my stick flicks:
If you're having trouble with the timing and getting your stick to be completely sideways, I'd advise you to go into training mode on 1/4 speed. Press Up-B and when you see Corrin starting the animation, flick the stick in the opposite direction.
A word of caution:
I played Corrin competitively for a few years in Smash 4, so I'm speaking from experience when I tell you that the window to flick your stick is surprisingly small. If you flick too late, you’ll only end up with a slightly angled recovery, but if you flick too early, you'll end up with a B-Reverse, which will turn Corrin around to face the other way.
I can't tell you the number of times I've killed myself from doing an accidental B-reverse instead of a straight upwards up-B. So don't beat yourself up if you can't get this on your first try!
1. Note that these images were taken before Update 3.0.0 where, from the patch notes, Corrin got "Extended launch distance" on her up-special.