That seems to be the only difference. Items with a green trim can be picked up, moved, and placed in the settlement, or stored in the workbench for later use (note: it seems you can only store items that you can build yourself from scrap. Some items can be moved/placed and functionally used, but not stored, such as white picket fences). Items with yellow trim are simply for salvaging materials, and their names occasionally imply this, as they are often "ruined" items (ruined house, broken toilet). Yellow items are also not movable in workshop mode, which also supports that they don't have any real use in the settlement.
It seems safe to scrap anything with a yellow outline, with maybe a couple of exceptions. If you scrap a container (such as mailboxes), you might want to make sure it is empty first, or you'll have to retrieve the contents from the workshop inventory later. Or, perhaps, if you wanted to keep any trees that remain upright in the settlement, don't scrap it for wood.