Work boats can only create improvements on tiles with special ocean resources. The water tile must contain clams, fish, crabs, whales, or oil. Furthermore, the tile must be in your territory.
Additionally, in the case of oil, you'll need Plastics to construct an Offshore Platform, and in the case of Whales you'll need to have Optics to construct Whaling Boats.
Unlike workers, work boats are consumed when they build their tile improvement. So if you have 2 fish tiles in your city's radius, you'll need 2 work boats to get the bonuses from both tiles.
In this screenshot, you can see 2 "fish" resources are in the radius of my city. I have a work boat on the lower fish resource tile. When a work boat is on a resource tile, you'll have a special command available to construct the resource improvement, which I've circled.
There are little "fish" icons pointing to these tiles, if you don't have these you can press Ctrl+R to make them show up, I believe. The tiles have blue circles because I think I have some sort of "recommend worker actions" option turned on in the preferences.
The work boat will not be able to do anything on any of the other water tiles in the city's radius - only the two fish tiles are able to be improved by the work boat.