Thanks for the reply! To answer your questions:
-I also got Windows 7, but haven't installed it yet.
-I did try new PSU with my old GFX card, which didn't work either.
-Didn't want to try the opposite, as my old PSU is a measly 350W so didn't wanna blow the whole thing up.
-The CPU and motherboard are original. Since my current CPU is an i7, I would think the motherboard is new enough to support my PSU and GFX card? Is there any way to check that?
Yea I'm sure your CPU/Mobo would be compatible.
From the sounds of it, its your PSU.
If you have another system in your house or a friends you can quickly use to see if for them it works or not would help to make sure 100%. (maybe do this with your GFX card also to make sure)
If not take it back to the store, explain whats happening and ask to clarify there to make sure.
It would be wise to tell them that when you first fitted your new PSU, you didn't use the brand new GFX card yet, (in case they state the GFX card was too over powered and say its your fault) and tell them the specs of your old system just to be on the safe side.
possibly take your system in and show them whats happening, I'm sure they will notice the PSU is faulty.
And fingers crossed they will give u refund or exchange.