What you have requested is impossible.
Let's look at the Minecraft Wiki page on Adventure Mode:
Adventure mode is similar to Survival in that players receive damage from their environment, dying is possible, and players must manage hunger. However, it is different in many ways as well. In the Java Edition, blocks can be destroyed only with an item/tool that has the
CanDestroy NBT tag, and cannot be destroyed by the player's fist. Likewise, blocks require the
CanPlaceOn NBT tag to be placed. In Bedrock Edition, similarly,
can_place_on NBT tags are required to destroy and place blocks.
Unfortunately, there isn't any way to specify blocks that can always be destroyed even without tools. The best way I would go about this problem is by giving players a tool at the start of the map that can break the block, and tell players not to lose it.