I'd like to add a different perspective here. I don't think this machine is worth upgrading. The components are all well over a decade old. If you upgrade the GPU, the CPU and RAM will bottleneck. If you upgrade CPU and RAM, the GPU will bottleneck and this will be so expensive (as you need to replace everything but the GPU and HDD), you mgiht as well buy a new one. If you go for straight upgrades, i.e. keep your current motherboard you are limited to upgrading to more ancient parts, which won'T serve you well.
I recommend three options:
1) Look for a budget gaming PC, e.g. i3 or Ryzen 3 CPU, 8 GB RAM, 250 GB SSD, GTX 1050ti or RX 560. This should cost you around 500$ and is fine modern PC, which will be certainly enough for your usecase
2) Similar to 1), but the money saver option. Get a Ryzen 5 2400G and no GPU. This should save roughly 100-150$. The integrated GPU in that chip is quite decent and should be enough for your favorite games. You also, of course, always have the option of buying a GPU later, if it's needed.
3) Look for a used machine from ~2013-2014. an i5-2xxx or 3xxx with a GTX 760/770 is a decent machine and you should find those for 300$ or less. You mainly lose out on warranty and an SSD with this option.