There's quite a bit in your question, so let me unpack it one piece at a time.
a lot of the knickknacks in the ship keeps appearing in my ship's inventory
Whenever you modify your ship (upgrade components, move stuff around, etc.) -- all of the 'stuff' lying around in your ship will be moved to cargo even if you don't save your changes in the ship builder. This includes all of the decorative junk items that come with the different components like engineering bays and so on.
I could sell them at the nearest vendor temporarily but it will come back automatically over time...
This isn't actually true.
As long as you don't modify your ship and add new structural components, the junk items won't "come back over time." Once it's sold, it's sold. Only if you modify the ship to add a new structural component (eg you add a new structural 1x1 store room) will new junk "appear" without any intervention from you.
If you get a new ship -- either by buying it or boarding and taking it over -- cargo from the previous ship transfers to the new one. Any loose items inside of your previous ship will be transferred to your new ship's cargo bay.
So if you had decorative junk lying around in the last ship, it will now show up in the cargo bay of your new ship; plus you'll have any decorative junk from the new ship lying around waiting to be transferred to cargo as soon as you open the ship builder.
Note that both of these mechanics can exceed your cargo capacity. For example, I have a number of ships. Ship A has a cargo capacity of 300. Ship B has a cargo capacity of 200. When I change my home ship to Ship B, all of my cargo transfers, plus all of the decorative items and other stuff I have lying around loose inside the vessel. Now my cargo bay in Ship B is carrying 375 out of 200 max capacity -- 300 cargo + 75 junk.
So, to sell off the junk "for good", you need to:
- modify your ship so that any items lying around are transferred into cargo
- sell everything from the 'misc' category (except the digipicks of course)
- do not modify your ship again
One trick you can use, to make sure all your 'valuables' actually get into cargo regardless of your cargo capacity, is to drop everything on the floor. Pick a room somewhere, and unload your inventory onto the floor ('R' button on PC.) Then, when you get to a planet, talk to the ship services technician and modify your ship -- you don't need to actually make any changes, just trigger the mechanic that moves all of your items into cargo. Now you'll have everything that you dropped on your floor in the cargo, potentially exceeding your storage capacity.
For contraband, you'll obviously want to put that into cargo directly because you'll be caught on a scan if it's just lying around or in your pocket. If you're using the trick to exceed your ship's cargo capacity, this means you'll need to take just enough out of storage to fit your contraband, put the contraband into storage, and then drop the remainder of the stuff on the floor.
(Note that the above mechanics mean that you can't really "decorate" your ship manually by putting stuff in various places, so I wouldn't be surprised if this mechanic changes in a patch of the game at some later time.)