Time & effort. Everything that you do means something else doesn’t get done. Whether that be gaming with friends or an item off your project/chore list.
We know that gaming centric distros are great for getting up & running, but it’s still a time sink, and will require effort. Not everybody has a backup drive with their games and will have to re-download everything too. There’s also a risk their favorite game isn’t compatible with Linux
Windows 10 also works just fine. I still have it on 2 of my 4 computers (2/5 if you count my Deck), and haven’t switched those over yet because I’m being lazy on one and the other is a perfect candidate for the SteamOS UX experience since it’s a HTPC. However, I have done some looking around at other HTPC experiences and just haven’t pulled the trigger. Which will be awesome, since Windows did away with their HTPC UX years ago.
I have a friend that regularly games in VR in Linux. Admittedly, he’s always faffing around with it to make it work. But he’s also a bit of a chaotic person that runs Arch, so that could just be him and not a failing of the current level of support.