As stated on the Fallout 4 wikia,
All unemployed settlers will collect one random junk item per day, which is automatically added to the workshopinventory. A settler assigned to a scavenging station will instead collect two random junk items per day. Once the total number of junk items in the workshop reaches 100 + (population × 5), no more junk items will be added to the workshop in either case.
So settlers will collect junk items even if none are present in the settlement and place it directly into inventory. This means that they do not pull items from your settlement.
However, if you have a large quantity of junk then there is nothing that is needed to be worried about.
Loose non-permanent junk has a chance to despawn if the settlement hasn't been visited in a while. If you've ever had a raider attack you Will notice the bodies and items disappear after 30 days out of the settlement. There is also a chance for this to occur with dropped junk after 30 in game days.
The settlers will not pick up loose junk items; however, junk items dropped have a chance to despawn after a certain amount of time.