Is there any way to remove or disable the menu entries that let you go to a previous turn? I don't mind being able to save-scum by manually loading an old save but this feature makes it a bit too tempting.
Yes! There are multiple ways of doing so. In the Preferences menu, under General, drag the maximum autosaves slider to 0. This will prevent any autosaves from being formed and persist until the slider is raised.
Alternatively, if you only want to disable it for your current session, you can use the console command nosaves. Just use : to bring up the console, and enter nosaves. The next time Wesnoth is restarted, autosaves will continue to save.