It may depend upon which version of Diablo 2 you are playing. Older versions, such as 1.10 and 1.11, still required the CD to be present in the drive during gameplay, at least at the start. So if it is one of those versions, you could still do it if the computers are nearby (same room) but it may be a hassle.
But the current version 1.13 doesn't require the CD to be present each time you launch the game. You would need 4 different CD keys to play all four of you online via battle.net, but for local play via a LAN or via TCP/IP play, you can have the same CD key used during install and there is no need to have it in the drive when playing if you have done the "full install."
So what may make sense, even if you have the old version, is install whatever version on the 4 machines (it takes a while) and then update to the current version by connecting to battle.net or via their downloadable installer. Then you are set with CD-less LAN plan on all four machines.
Also note that there is Diablo 2 ("Classic") and Diablo 2 Expansion: Lord of Destruction ("LoD") and this applies to either version.