You cannot.  Dwellers in the wasteland still requires living spaces in your shelter.

If you need to change the room directly connected to your vault gate, you must kill/remove everyone to be able to delete it.

However, once everyone is dead and removed, dwellers will start appearing outside of your vault right away, so you don't have to worry about not having dwellers.