I need to go to jail in solitude but I don't want to lose my quest items. Do they get taken or do I get them back?
They will get taken when you go to jail, but the quest items will be put back into your inventory once you escape (without even having to go grab them from the chest). If you do break out and go to the item chest, however, before escaping the jail completely, you can find them there along with the rest of your items.
However, there's a chance of a bug happening where you don't reobtain your quest items, but this has to do with a very specific jail related quest.
Just to be on the safe side, save your game before going to jail in case you lose your items.