Adding to what Saltypaws said...
You won't lose anything regarding the game itself. It won't be uninstalled or anything. Deleting the icons will make your desktop less cluttered, definitely, but it won't do anything for the speed or running of your computer. All you're removing is that small piece of graphic that is the icon, so it won't make a dent in the space on your computer.