Sounds like the connector on your AC in terminal is not reliably. Perhaps use a screw-driver to tighten the screws on the terminal block and try again. Also make sure you’ve inserted the plug all the way in.
I think it’s producing just a little too much A/C power.
Not sure what this means. I did hear many questions about whether the transformer will ‘pump in’ too much current to the circuit — so let me clarify this common confusion: the transformer does not ‘pump in’ current to a circuit, the circuit draws current from the transformer, and it can draw anywhere from nothing to the maximum rated current of the transformer. For example, a 24V AC / 750mA transformer is capable of delivering a maximum of 750mA current and maintain roughly 24V AC output voltage. But how much current it’s actually delivering to the circuit depends on how much the circuit draws.
I can’t measure any DC voltage across the LED or anywhere else for that matter,
There is a +5V pin in the RPi pinout area located at the upper-left corner of the circuit board. If you look at the user manual page 9:
http://rayshobby.net/docs/ospi14_manual.pdf
the third pin on that row is a ground pin. So you can put your multimeter probes across the first (+5V) and third (GND) pin to measure voltage. There are also some ground pins in the ADC pin out area close to the right edge of the circuit board.
You should first make sure the LED lights up reliably between you plug in RPi.