It’s normal to see such stray voltage — when the triac is turned off, its drain (or more precisely MT1) pin floats and can easily pick up stray voltage. It’s typically not sufficient to power anything substantial though. Are you seeing all LED lights turning on? How large is the current limiting resistor for each LED?