In general, I'm looking to create a more userfriendly way of solving a frequent audio issue in various games.
It manifests itself as audible clicks or pops. Those clicks or pops happen whenever audio is played. Higher volume, lower frequency, and more 'full' sound (I.e. synth waves as opposed to drums) produce more pops. Imagine having a Geiger counter added to the background. In other words, the sound data stream corruption is arbitrary and random.
There's however one way to reliably reproduce these clicks and pops:
Normally, when adjusting the volume slider for an application on my desktop, there is no sound. However, when an affected application has its volume changed, there will be a loud rattling noise sent to the sound card. You could compare it to listening to a firecracker roll going off in the distance. With each pixel of change one click or pop is produced.
Games that use directAudio or adobe-AIR seem most affected.
Presumed underlying causes
The problem has something to do with the interaction of Adobe components, Microsoft components, badly written game audio code, sound card driver issues due to undocumented audio hardware quirks, and through this the overly complicated linux audio stack with 9(!!!) places where audio data can be buffered. Finding out who or what is responsible for the various quirks is a nightmare, so my typical modus operandi is to tweak settings (as linux allows the user to modify everything) until it goes away.
Games typically don't know/care about the fact that audio hardware has latency. They run every frame (typically 16ms) and simply expect their audio to be playing immediately after having called the relevant function. Games which do internal audio mixing for e.g. directional audio can thus glitch (and thus crackle) if the hardware isn't fast enough. Let's go through an example with normally distributed lag of
μ = 60ms σ = 15ms
The game starts 2 sound fragments. But, fragment 2 is played before fragment 1, causing an audible audio pop to occur. This is because the smooth audio sample output (music is a bunch of overlapped sine waves) is corrupted by flipping two small sections, which creates a visible jump.
+--------+----------------------+ | T (MS) | ACTION | +--------+----------------------+ | 0 | schedule fragment 1 | | 17 | schedule fragment 2 | | 80 | play fragment 2 | | 88 | play fragment 1 | +--------+----------------------+
What does this do to the actual audio output? Let's look at a 220Hz (Lower A) sine wave:
The jump is audible as a 'pop' sound. In reality digital oscillators can't jump instantly, they produce a bit of ringing as well which is why we can hear it as a short click or pop. Now you can understand why also it's much more notice-able for louder sounds (bigger jumps also) and for lower frequencies (slower, smoother waves make it more notice-able),
Let's dissect the stack: Audio can be copied over (and thus corrupted) in the following places:
- In the audio hardware
- In the sound audio driver
- In the linux kernel
- In ALSA
- In Pavu/Pulseaudio
- In the WINE translation layer
- In the windows sound driver called by the game running on WINE
- In the audio library called by the game
- In the game itself.
Because of this, the standard aggressive audio timing pulseaudio uses is inadequate to solve the situation. It needs to buffer and wait for longer to prevent the components further up in the stack from introducing jitter/stutter and loss into the connection.
Typically I've been sometimes able to solve/mitigate/hack around the problem using the following three things:
- Launch the WINE executable with the following environment setting:
By executing the command below we add 60ms of lag to audio. This allows pulseaudio to put the jittered out-of-order audio fragments back into order.
env PULSE_LATENCY_MSEC=60 env WINEPREFIX=/location/of/prefix WINE game.exe
- Use (or don't use) interrupt-based versus timer-based audio scheduling
Change the pulseaudio config file /etc/pulse/default.pa, changing this line:
- load-module module-udev-detect + load-module module-udev-detect tsched=0 sed s/load-module module-udev-detect/load-module module-udev-detect tsched=0/g /etc/pulse/default.pa
- Configure the specific audio driver scheduling
Further down the stack is the audio driver, which can also be scheduled by a pair of parameters manually.
This can be done in
default-fragments = 2 default-fragment-size-msec = 5
Note that power_use is synonymous with amount of hardware operations required.
latency = default-fragments * default-fragment-size-msec max_jitter = default-fragment-size-msec power_use ∝ 1 / default-fragment-size-msec
Different games require different settings. Normally, I have tsched=0 ON, but, for example, the game 'Noita' will produce sound crackling iff it is not OFF. Other games work the other way around, where crackling is present without tsched, but no crackling is heard with it.
Same goes for tuning the hardware parameters. Games that use a lot of sound files may run into hardware limitations if
default-fragment-size-msec is set too low. Others don't want it too high because of jitter, or plain glitch in other ways with too much latency. The hardware itself typically requires 2 or 3
default-fragments as well in all situations (you create a whole new class of problems by setting it too low). In order to tune these parameters, after editing the file, execute;
to force a restart of the audio server, reloading the configuration.
Yet, even having documented these three cases, for some games this seems not to adequately do the job. I would like to know if there are perhaps better solutions out there, or if there are things missing here.
I'm also interested in perhaps the creation of a program (which would need
setuid) that would handle all the changes; so I could run say
#!/bin/bash # apply 60ms (~4f) of driver latency, 24ms (~1.5f) of hardware latency /usr/bin/wineaudiofix --tsched=1 --latency=60 --fragments=3 --fragment_latency=8 env PULSE_LATENCY_MSEC=60 env WINEPREFIX= (...) WINE game.exe
and store these settings with the executable, instead of spending a long time mucking about with audio every time I want to run one of those games. The hard part about it is managing to do this safely; i.e. without creating a root backdoor.
Next up is the problem of Steam. I like Steam Proton over regular Wine for its userfriendliness. However, I have not figured out how to incorporate either a script or these environment variable changes into Proton. Steam only provides a nondescript textbox for cli parameters, which doesn't really work as we're dealing with three (four?) types of parameters here:
- Stuff passed to the game.
- Stuff passed to WINE
- Stuff passed to Proton
- Environment variables (env) set in the shell environment.
So could it be possible to get this all neatly wrapped up so that clicking 'play' in steam does the required work, and starts the game. Or alternatively, if impossible, I can run a script that configures the necessary stuff when run, then simply click play.
The other hard part is if I manually point the distribution's WINE to the steam-created prefix folder in
steamapps/compatdata/<gameID>/pfx as the WINEPREFIX environment variable it won't actually interface with steam i.e. steam achievements and games using steam DRM won't work via this method.
The games listed below are known to be affected by this issue. I will add more to the list as well as known parameters for an adequate solution when it is found. Caveat though; the solution may depend on the hardware.
- Dungeons 3
tsched=1, 0 latency
- Guild of Dungeoneering
no known solution
tsched=0, 60ms latency