How to install 2.6 upgrade from 2.5?

So I just downloaded the 2.6 upgrade (Windows) from Plugin Boutique. How exactly do I install the upgrade? Do I uninstall 2.5 first, or does the 2.6 install right over 2.5?

Should go right over it.