I've set up a district with specialized industry (oil or ore) over an area with that natural resource. Buildings pop up, things work, and eventually the resource is depleted (as seen on the Natural Resources view). The buildings stay up, however, and appear to continue working. Will the buildings continue to work indefinitely, or do I need to eventually set the district back to generic industry?
Yes, they still work, but they need to import resources.
When you place specialized industry on their respective resources, they will create a mix of raw material producing factories and raw material consuming factories. The proximity means that the transport of goods between them is a very small issue. The consumers will supply goods to your generic industry so they don't need to import from off-map.
When the resources are depleted, or when you place specialized industry in an area without natural resources, then all the producers will be replaced with consumers. Now they need to import raw materials from off-map. This will create more cargo traffic at your highway ramps and cargo terminals (train, ship, airport). When you have a good road network, then this might not be an issue. But if the industrial zone is not that well connected, then it might lead to traffic problems.
When you can handle the traffic, then you might want to consider keeping the specialized industry, because it still creates more tax money than level 1 generic industry. But on the other hand, specialized industry can't upgrade. So if the area happens to meet the service coverage requirements for level 3 generic industry, then you might want to consider replacing.
When you can't solve the traffic issue, get rid of the industrial zone. Generic industry isn't a solution, because they too will need to import.