New build computer won't boot and PSU makes strange noises

Registration date
Saturday October 1, 2011
Last seen
January 28, 2015

I have just build a new computer with the following elements:
1. GIGABYTE GA-78LMT-S2 Motherboard
2. Be quiet! System Power S6 300W ATX22 PSU
3. 2 sticks of 2 GB DDR3-1333 Memory
4. Athlon II X4 640 Processor
5. Western Digital 500 gb IDE drive
6. Western Digital 160 gb SATA drive

Maybe a bit of to much information, but just to be sure I mentioned all the details. The problem is that when I first turned on the system, with all plugged in except the HDD drives, the PSU started to make a boink-boink noise. Like he was hitting something with its fan. And I could only turn the computer on with my power button, but not turn off.

When I plugged in a keyboard he stopped doing this, turned on normally (with or without monitor plugged in) and could be switched off with the power button. I tried a couple of times and indeed the only important factor was whether or not the keyboard was plugged in.
If plugged in: Normal startup, Bios screen, no weird noises, power button can turn the computer on AND off

If not plugged in: PSU makes weird noises as described above, power button won't turn the computer off, Bios won't load (at all, just black screen forever)

Now I didn't really care for that moment, so I just continued working with the computer, since the keyboard needed to be plugged in anyway. But recently (today, and the computer was build two days ago) I tried a different monitor, an older one. I plugged in a keyboard with which it used to work before and all worked fine. Then I tried to change the keyboard, and he won't boot anymore. Tried that keyboard as usb and through an adapter (to plug it into the keyboard port). Both won't boot and the weird noise is back, and the button won't work anymore. Tried my old keyboard, but it won't work either. Tried a third keyboard, won't boot.

Could someone please help me? Because I really don't have a clue anymore.

Thanks in advance.