Depends, you can deflect too much air around the car and not into the sidepods I guess, and if the aero was part of the side pods.
There is also the fact that aero parts can break off, a fin above the side pods being made a bit too weak so flexed or risked snapping off. Did Kimi lose a piece of body work yesterday, or was it a Williams, can't remember. Airflow not hitting the air ducts around the brakes could also cause breaks getting too hot and being a problem.