Ok, after a little more testing I found the following. If both of the first two octets of the ip address are greater than 99 i.e. 192.168.x.x then I immediately get the result above.
I just purchased several of the controllers and am trying to get the windows 8.1 app to work. When I try to add the controller using an IP address the app returns: “Check IP/Port and try again”. The iPhone app works without issue so I know I am entering the right settings. Help?