Why did Donkey Kong 64 require more than 4 MB of memory?

Question

As most of you know, the N64 Donkey Kong game had a game breaking bug on the 4 MB memory version. They needed to include the expansion memory just so that bug wouldn't happen.

I can imagine in the age of computers and emulation, we should by now have a pretty decent idea as to why this bug happened. As in, limit ram to 4 MB, see what tries to access outside there and why.

Sadly it probably is hard to figure that out, and I can imagine that at the moment of the development switch to 8 MB, they actually started using more of that ram since it was there anyway.

Does anyone know what exactly, nearly got that game pulled at all?

Answer 1

Does anyone know what exactly, nearly got that game pulled at all?

Unfortunately, the actual issue will likely never be known, at least not any time soon. I will detail several technical reasons for this which I believe are objectively compelling enough to base my answer upon.

The primary issue preventing the diagnosis of that glitch is a fundamental matter of hardware architecture, and how using software to force one architecture to behave another will more often than not be extremely difficult and sometimes completely impossible. This is the reason why emulators in general are difficult to develop, highly difficult to develop well, and darn near impossible to develop so well that they'll run all games for a given platform without issue. This is why there's so many crappy/unfinished/abandoned emulator programs out there and so few fantastic ones.

The second issue is that emulators don't actually emulate the hardware components and chipsets of a console. They just duplicate enough of the high level functions of the components that they're able to play the games. Even the emulators regarded as "perfect" are only cycle-accurate (correct timing), which is plenty for playing games but still not 100% hardware simulation (chip-accurate emulation).

The Donkey Kong glitch had to have been the result of some freak error between the memory management coded into the game and the actual memory control unit of the N64, meaning that to try to debug it via emulation on a PC, we would need chip-accurate emulation to fully simulate each and every logic chip in the N64's ludicrously complex Reality Co-Processor, funky 9-bit memory bus, and every other component. Unfortunately, this is literally impossible with current technology.

Most people will think the idea of a modern PC being unable to simulate the hardware of a 20-year-old Nintendo seems crazy. However, an Intel-architecture PC and an N64 are so vastly different in how they process data that a PC's extra CPU cores, exponentially faster clock, and several-thousand times more RAM isn't even remotely enough computing power to 100% accurately emulate the internal logic of an N64. In fact, a top end PC wouldn't even come close to being able to do chip-accurate emulation for an SNES or even an NES. Intel CPU's just aren't built for it. Even the one cycle-accurate N64 emulator is known to get choppy on anything less than an i7.

Finally, just for funsies let's say those technical limitations magically disappear tomorrow and someone hands us a flawless emulator to use for testing and debugging. I'd still be willing to bet against the true cause of the glitch ever being found. You have to consider the fact that we're talking about Rare in its heyday. This game was programmed by arguably the most talented and competent developers to ever write software for the N64 and who would have been more intimately familiar with the tiniest details of the hardware itself than anyone else in the world aside from the engineers who designed it. They would have known every aspect of the game's memory management logic inside and out because they literally designed and wrote the code for it. For the mystery glitch to elude them so completely as to make Rare say 'screw it' and give away expansion packs, the root cause would have to be something unbelievably obscure and impossible to track down with logical troubleshooting.

If the people who are the world's foremost experts on the DK64 software couldn't fix it when it was literally their job to do so, I doubt anyone else will discover the cause unless they just happen to by total accident.

Answer 2

Actually, there's evidence that the memory bug is just a myth.

The origin of the story about the memory leak comes from an interview with Chris Marlow, a programmer who worked on the game. According to Chris, the expansion pak was used to resolve a last minute bug.

Mark Stevenson, lead artist for the game, says otherwise. He's quoted in this article as follows:

"This one’s a myth. The decision to use the Expansion Pak happened a long time before the game shipped, in fact we were called in by management and told that we were going to use the Expansion Pak and that we needed to do find ways to do stuff in the game that justified its use and made it a selling point. I think the bug story somehow got amalgamated into the Expansion Pak use and became urban myth."

"There was a game-breaking bug right at the end of development that we were struggling with," he clarifies, "but the Expansion Pak wasn’t introduced to deal with this and wasn’t the solution to the problem. My memory is that, like all consoles, the hardware is constantly revised over its lifetime to take advantage of ongoing improvements in technology and manufacture methods to essentially make the manufacture more cost effective and eventually profitable. I think there we’re something like 3 different revisions of the internal hardware by this point and the bug was unique to only one of these versions. We did eventually find it and fix it, but very late in the day."

Something that lends a lot of credence to Stevenson's story is that I can't find a video that shows the memory leak happening. There's a 11.5 hour video where the guy waits for the leak to happen, but it never appears (according to legend it happens around the 10 hour mark). If it happens through standard play, there's a 30 hour video of a single-sitting speedrun of the game.

So what was it used for? One use they found for the Expansion Pak was dynamic lighting:

Artist Mark Stevenson remembers it being beneficial in terms of standard things like level size in Donkey Kong 64, but there were also more creative uses. “One thing I remember that we did use it for was that we had a lot of dynamic lighting in there, which was hard to do and expensive,” he recalls. “One of the engineers wrote a system whereby you’d go into a cave area, and there’d be a swinging light - the first swing of that light, it’d record all of the colour changes on all of the vertices in that area, and then save it as data and just play it back as an animation rather than going on to calculate the lighting constantly. You’d get a little bit of slowdown when you went in, but after that, it was nice and smooth.”

You can see an example of dynamic lighting in the game when you go to Cranky's Lab