RGB in on the TV, every time. You can use that DVI output, and run the TV as a second monitor.
Your digital audio in on the front of the amp is optical, so not compatible with the digital coax out from your PC.
Two options:
1) Look on the back of the amp for a digital coax in (same as PC) - best option.
2) Use the 2 channel (L/R) inputs on the front of your amp and a 3.5mm output from the PC.
Depending on your amp, you might still be able to make all 6 speakers work, but it will only be stereo, not 5.1