That’s possible. Because DS1307 is powered by 5V, so any spike or overvoltage presented on 5V will damage DS1307 too. If you don’t need RTC you can just remove DS1307. It’s really only useful if there was a power break, and your controller is back online while you don’t have Internet connection. Otherwise RPi can perform NTP sync automatically to make sure it stays synced with the current time.