As others have already pointed out Nvidia drivers aren’t that bad. The only game I’ve had issues with is Star Wars Outlaws, but I think that has more to do with the game itself than Nvidia drivers (It’s not exactly a stable experience on Windows either).
The only big thing holding Linux gaming back is anti-cheat, but that’s mostly because AAA developers don’t want to allow anti-cheat on Linux. It’s worth checking out if your favorite online game can be played on Linux.
I would personally recommend Satisfactory over Factorio. I think it’s a more casual experience while still scratching that factory building itch.