Specular Showdown in the Wild West

Cheers!

I think it definitely makes sense to apply this to the base level. If you don't, the final appearance will look really different (particularly at higher specular powers) between the base and 2nd MIP levels and lead to a nasty, aliasing transition between the two.

Edit:
I had to cut short the reply earlier. What I should have said is that I understand your way of thinking, as, like Brian, I started out only doing this for the lower MIPs. That looked really wrong though and it was obvious when viewing the gloss factor directly and seeing how it faded to white at the base level. You're basically breaking a core assumption of regular MIP mapping by putting different content into the top MIP. It's also essentially saying that the base level of your normal map has a uniform direction, which is probably a complete lie.