It might not cause any problem, preamplifying is often needed in a lot of source equipment to take a low output to a line level one for ultimate amplification.
In PCs sound cards with dedicated headphone outputs have been developed as the output is amplified to drive headphones (some of which have much higher impedance). Historically this was never the case as HiFi headphones were not typically used in desktop computing and were easy to drive simply from an unamplified line out. Or, a speaker system which amplified itself was used.
Preamping a signal too much can cause clipping, for example. The source is also just that, the source. If you have a poor source whatever else happens further down the chain is affected. That being said, you should supply the cleanest output to your amp, - you don't need to preamp avoiding preamping in your sound card may be a good start.
The reason I asked was if you had tried the line out as a point of comparison. Bottom line is yay won't know if you can don't try.