Masked layering would work. Find a good tutorial with those terms in google, but in brief here's what I'd do.
File->Scripts->Load images into stack (tick the automatically align box) - this get your images stacked
With the upper layer selected (assuming upper layer is the exposure with the window correct) hit the button at the bottom of the layers toolbar with the white circle in a black square, this creates the mask. Hit control-I to make the mask go black, then paint the window back in with white using the paint brush. You can then alter brightness/opacity etc to get the layers looking right.
Alternatively just combine the images with File->Automate->Merge to HDR, though this will often give a different (less natural) looking result.