My X-01 Power Armor in Fallout 4 keeps moving to different places at the Redrocket Truck Stop where you find dog meat but I don't have any followers or settlers
Even without settlers it is possible for other NPCs to use your power armor. Hostile raiders, merchant caravans, or even inactive companions are all capable of using power armor.
To protect against this take the fusion core out of the power armor to make it inoperable. NPCs are incapable of entering power armor without a fusion core and it's unlikely they'll find any lying around.
Alternatively, place your power armor somewhere they cannot reach it. NPCs cannot jump and they do not drop off of terrain (if possible). Place your armor somewhere they'd need to jump to reach or into a room they'd need to drop into and they'll avoid it.