What kind of case do you have and how is your airflow within? The switch from a 1080 to 3080 will introduce more heat into the case and some of that heat is being exhausted directly over the RAM area. If your RAM was already close enough to its limit then even a small amount of additional heat may well have been enough to cause these issues as you suspect. Is your CPU air or water cooled as this can make a difference to how this heat is handled.