My plan was then to reboot the machine with a CD to install an OS on the new drive.
However, no matter how long I press the power button, it won't power off.

All it does is blink with two of the blue lights in the display (one with an "A" in a box, and one with an arrow pointing downwards in a box).
How can I reboot the machine?