Verified answer

Find the file you don't want, then right click it and press delete.

Excel files by default are stored in the "My Documents" folder, which you can access from your start menu. Just open up the folder, and drag your unwanted files to the recycle bin (on your desktop)