The zener diode is optional (although it’s useful for over-voltage protection). You are right that USB power is sufficient to test the microcontroller and the web interface.
It’s very odd that your initial voltage testing passed but then components got damaged when other ICs and LCDs are plugged in. In any case, you can easily test if these components are still working by powering the controller through USB. The worst case is that the LCD and Ds1307 may be damaged (because they are connected to the +5V VIN line), but I assume the zener diode did protect them and hopefully they are fine.