No it won't affect your Overclock but it will massively affect the airflow in the case and is a bad idea tbh.
Leaving it off will muck up the airflow in your case and if anything your temp's will increase not to mention it will act like a hoover and collect dust very quickly. Do you have any intake fan(s) ??