Yes, I know that is just your opinion, but unless there was something wrong with the way you set it up (no deadening, too much power, improper balance, whatever) there is no reason it should sound worse than using the HU amplifier.
I personally don't like any rear fill at all. A good front stage is way more than enough, and to me, rear fill messes with the sound stage. That may be what you were getting at.
But stricly comparing amped to non-amped rear fill, if both are done right, there is no reason non-amped should sound better.