In Fallout 4, when there is no one in your sight, a short press on VATS button will do nothing.
A long press, however, will activate an empty VATS. Is there a use for this?
Long press (Hold) VATS button when there is nobody around.
Long Press (Hold) VATS button when there is friends behind me.
Short press VATS button with friends behind me.
This answer is based on hundreds of hours of play time on Fallout 4 and its predecessors.
There are two types of VATS presses: a tap, and a press and hold.
The short tap looks at the area in a cone in front of you. It's most of what's on screen, but not quite all of it, as well as a little bit off to either side. I've had an enemy in the corner of my screen, had the tap not target them, then shifted it a few degrees closer, and it found them. I've also had it target an enemy slightly off the screen to either side.
If there are any enemies in that cone and within range of your weapons, it will target the nearest enemy to your reticle. This takes cover into account, so if an enemy is behind a hill or wall they won't be targeted, even if they would otherwise be in range.
If there are no enemies in the cone, but there are targetable friendlies (not your follower), then it will target the friendly closest to your reticle.
Tapping will not target anything behind you.
One issue with tapping is that tapping is also how you exit VATS. If you're trying desperately to get into VATS as soon as possible, like if you're exiting a kill cam and really need to target that guy with the missile launcher, then desperately tapping VATS will lead to you entering and then immediately exiting VATS, and then getting blown up.
Holding down the button will force the VATS interface to come up even if there are no valid targets in your frontal cone. The range for this is longer than tapping, but still not infinite. It's possible to see an enemy with a long distance scope, but be unable to target them in VATS.
If there's a targetable enemy anywhere around, it will target the nearest one. If not, it will target the nearest friendly. Sometimes, the camera may not actually move to point at them. If you see a name on your screen, though, they're there. Switching limbs will move the camera to look at them.
If there are no available targets at all, pressing the button will still force VATS to come up, but not targeting anyone.
If you need to get into VATS at the soonest possible moment, like the scenario with the missile launcher mentioned above, then you can hold down the VATS button. It will enter VATS as soon as you're able, and won't exit it.
If you know who you want to shoot at, you should usually just aim near them and tap the button. This is a pretty predictable way to target the enemy in front of you. It also avoids re-targeting things behind you that you don't plan on shooting.
If that doesn't work, but you still think there are enemies around, hold the button to check. This can be a little disorienting, as it may swing completely around and target someone you had no idea was even present. If you don't target anyone after holding down the button, you can be fairly confident there no enemies are around. However, keep in mind that there may still be someone who was obscured by cover. I've had it fail to target someone because they happened to move behind a tree at the exact moment I pulled up VATS.
Once the VATS mode is up, you can freely look around. If you hover near a valid target, it will be targeted. If you seem to have someone targeted, but you're not zoomed in on them, then switching limbs will update the camera.
Keep in mind that you can target mines with VATS. If I'm in an area where I think there may be mines in front of me, I will occasionally tap the button to ping the area I'm about to walk into. This is not foolproof, as there may be a mine hiding around a corner that was obscured when you tapped the button. Shooting a mine causes it to explode. You can target friendly mines.
I sometimes target friendlies to check how many shots I have on my AP bar when comparing weapon attachments.
Bringing up an empty VATS still slows time so you can observe something at a slower speed. Maybe an explosion. You'll notice people's voices also slow down during VATS, which can be amusing.
Note that you have 90% damage reduction while in VATS. If a lot of damage is imminent, like when you hear a whistling fat man round from parts unknown, and there's no obvious cover, it could be prudent to hold the button to enter VATS and take less damage.
An empty VATS will also allow you to take advantage of the VANS perk, as pointed out in blgt's answer.
In addition to the other answers, using VATS out of combat activates a glowing trail leading to your closest activated quest objective when you put a point in the VANS perk:
Let Vault-Tec guide you! The path to your closest quest target is displayed in V.A.T.S.
Here's a screenshot of what it looks like in-game:
Using VATS outside of combat and attempting to shuffle through "targets" will reveal things you may not have noticed as the targeting will shift to anything targetable by VATS, such as mines or far-off enemies, which would give you time to think of a plan of attack (or know to avoid said area).
Whenever traversing buildings or enemy strongholds, a quick VATS check every so often will help prevent you from walking straight into a minefield.
