Mobile app "unable to connect" through SSL reverse proxy (Apache)

[ipilcher]

I’ve submitted a support ticket for this, but thought I would post here as well, in case anyone has seen this or has any idea what’s going on. (Plus the forum seems to offer slightly better formatting capabilities.)

I have set up an SSL-enabled reverse proxy using Apache, and the browser UI is working through the proxy.

The Android app does not work through the proxy; it gives an “unable to connect” error. The Android app does work if I point it directly at the OpenSprinkler controller.

I’ve used tcpdump and wireshark to compare the traffic going to the controller when I try to connect the app to the controller, and the problem appears to be that the app is adding an additional “/” to the GET /js?… request when going through the proxy.

Here is the request/response when the app is communicating directly with the controller:

GET /js?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1
Host: opensprinkler.penurio.us
Connection: keep-alive
Accept: application/json, text/javascript, */*; q=0.01
User-Agent: Mozilla/5.0 (Linux; Android 4.4.4; A0001 Build/KTU84Q) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
Accept-Encoding: gzip,deflate
Accept-Language: en-US
X-Requested-With: com.albahra.sprinklers

HTTP/1.1 200 OK
Content-Type: application/json
Connection: close
Access-Control-Allow-Origin: *
Cache-Control: max-age=0, no-cache, no-store, must-revalidate
 
{"sn":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"nstations":16}

And here it is when going through the proxy:

GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1
Host: 172.31.252.3
Accept: application/json, text/javascript, */*; q=0.01
User-Agent: Mozilla/5.0 (Linux; Android 4.4.4; A0001 Build/KTU84Q) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
Accept-Encoding: gzip,deflate
Accept-Language: en-US
X-Requested-With: com.albahra.sprinklers
X-Forwarded-For: 172.31.253.140
X-Forwarded-Host: www.penurio.us:32697
X-Forwarded-Server: www.penurio.us
Connection: Keep-Alive
 
HTTP/1.1 200 OK
Content-Type: application/json
Connection: close
Access-Control-Allow-Origin: *
Cache-Control: max-age=0, no-cache, no-store, must-revalidate
 
{"result":32}

Note the additional “/” after “js” in the request.

The logs on my proxy show that the extra character is coming from the app; it is not being added by the proxy:

172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 13
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jc?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 349
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jo?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 384
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jp?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 62
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jn?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 236
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jo?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 384
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 13
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jc?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 349

[Samer]

Thank you for the detailed post! I took a quick look at this and find it very odd it’s only adding the slash for /js and not the others. Furthermore, I took a look at my code and the /js is only defined on one line:

https://github.com/OpenSprinkler/OpenSprinkler-App/blob/master/www/js/main.js#L1026 and I can’t imagine how a slash could be added so my theory is the reverse proxy is adding the slash. If the packets you are showing me are down stream of the proxy, then could you provide the proxy configuration or at least double check to ensure it’s not adding the extra slash?

Also when using the web browser UI, the slash is not present correct?

[ipilcher]

I agree that it’s very weird. I also thought that the proxy was adding the slash, but the logs from the proxy show that it isn’t so:

172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 13
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jc?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 349
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jo?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 384
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jp?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 62
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jn?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 236
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jo?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 384
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 13
172.31.253.140 - - [27/Aug/2015:15:14:54 -0500] "GET /jc?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1" 200 349

Here is the proxy configuration:

Listen 32697

<VirtualHost *:32697>

        ServerName www.penurio.us:32697

        ErrorLog /etc/httpd/logs/sprinklers-proxy_error_log
        TransferLog /etc/httpd/logs/sprinklers-proxy_access_log
        LogLevel warn

        NSSEngine on
        NSSCipherSuite +rsa_rc4_128_md5,+rsa_rc4_128_sha,+rsa_3des_sha,-rsa_des_sha,-rsa_rc4_40_md5,-rsa_rc2_40_md5,-rsa_null_md5,-rsa_null_sha,+fips_3des_sha,-fips_des_sha,-fortezza,-fortezza_rc4_128_sha,-fortezza_null,-rsa_des_56_sha,-rsa_rc4_56_sha,+rsa_aes_128_sha,+rsa_aes_256_sha
        NSSProtocol TLSv1.0,TLSv1.1,TLSv1.2
        NSSNickname "www.penurio.us - StartCom Ltd."

        ProxyPass / http://172.31.252.3/
        ProxyPassReverse / http://172.31.252.3/

</VirtualHost>

And yes, the slash is not present when using the browser UI (including when using the browser UI from the Android device).

[Samer]

Apologizes, I missed the last bit of your original post.

Regarding the extra slash though, it doesn’t seem to be an issue when querying the controller. In other words, for me both /js/?pw and /js?pw work and return the expected reply. Therefore, I am not even sure if that is the issue.

If you don’t mind, the best way to identify the error would be to debug using Chrome. You can plug your phone into your computer using a micro USB cable and change the settings to allow USB debugging. You might need to unlock the developer menu to accomplish this which can be done by tapping the build version in the about screen 7 times (if memory serves). Once this is completed, you can open Chrome and navigate to: chrome://inspect.

You should see your phone listed (when the app is open) and it will allow you to “Inspect” it. Here you will have access to the web developer console which allows you to look at the network traffic tab and the console in hopes of identifying the error.

Would you mind trying the above steps to see if you see any errors or warnings?

Thanks!

[ipilcher]

Regarding the extra slash though, it doesn’t seem to be an issue when querying the controller. In other words, for me both /js/?pw and /js?pw work and return the expected reply. Therefore, I am not even sure if that is the issue.

That’s really interesting. Are you saying that you don’t get the failure on the second attempt with curl?

[pilcher@ian ~]$ curl http://172.31.252.3/js?pw=f7a9f061ae223e7ed166a73abe2abd8a
{"sn":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"nstations":16}[pilcher@ian ~]$ 
[pilcher@ian ~]$ curl http://172.31.252.3/js/?pw=f7a9f061ae223e7ed166a73abe2abd8a
{"result":32}[pilcher@ian ~]$

Can you check that before I do all the Android debugging stuff?

[Samer]

This is what I am seeing but please note I am testing on a 2.1.6 beta (not released yet):

Samers-iMac:~ salbahra$ curl http://10.0.3.8/js?pw=a6d82bced638de3def1e9bbb4983225c
{"sn":[0,0,0,0,0,0,0,0],"nstations":8}

Samers-iMac:~ salbahra$ curl http://10.0.3.8/js/?pw=a6d82bced638de3def1e9bbb4983225c
{"sn":[0,0,0,0,0,0,0,0],"nstations":8}

Update: You are right that result:32 would cause a connection failed error on the app. So likely, you are right this is the issue. Wonder why it differs now…let me discuss with Ray and get back to you.

[Samer]

Not sure if it will help but here is my proxy pass configuration:

<IfModule mod_ssl.c>
	<VirtualHost _default_:443>
		ServerAdmin webmaster@localhost

		DocumentRoot /var/www

		SSLEngine on
		SSLProtocol all -SSLv2
		SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM
		SSLCertificateFile /etc/apache2/ssl/ssl.crt
		SSLCertificateKeyFile /etc/apache2/ssl/ssl.key
		SSLCACertificateFile /etc/apache2/ssl/ca.pem

		ProxyRequests Off
		ProxyPreserveHost Off
		AllowEncodedSlashes On
		KeepAlive Off

		<Location /os/>
			AuthType Basic
			AuthName "Restricted"
			AuthUserFile /etc/apache2/.htpasswd
			AuthBasicProvider file
			Require user admin
			ProxyPass http://10.0.2.23/ timeout=1200 KeepAlive=On
			ProxyPassReverse http://10.0.2.23/
			<IfModule mod_headers.c>
				Header set Access-Control-Allow-Origin "*"
			</IfModule>
		</Location>
	</VirtualHost>
</IfModule>

This allows me to access the device using: https://my.domain.com/os and also uses Basic authentication. You can try to fit that into your configuration and see if it helps.

[ipilcher]

I’m pretty much convinced that the proxy isn’t adding the slash. I managed to use wireshark to decrypt the SSL traffic between the app and the proxy, and I can see the app sending the request with the extra slash:

GET /js/?pw=f7a9f061ae223e7ed166a73abe2abd8a HTTP/1.1
Host: www.penurio.us:32697
Connection: keep-alive
Accept: application/json, text/javascript, */*; q=0.01
User-Agent: Mozilla/5.0 (Linux; Android 4.4.4; A0001 Build/KTU84Q) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
Accept-Encoding: gzip,deflate
Accept-Language: en-US
X-Requested-With: com.albahra.sprinklers

HTTP/1.1 200 OK
Date: Thu, 27 Aug 2015 22:11:52 GMT
Content-Type: application/json
Access-Control-Allow-Origin: *
Cache-Control: max-age=0, no-cache, no-store, must-revalidate
Connection: close
Transfer-Encoding: chunked

d
{"result":32}
0

I also connected to the phone with Chrome, as you suggested, and the Console tab shows a bunch of errors like:

Failed to load resource: net::ERR_FAILED https://www.penurio.us:32697/js/?pw=f7a9f061ae223e7ed166a73abe2abd8a

(With the extra slash.)

On the network tab, however, the URLs don’t have the slash.

And ….. son of a B, it works on a different phone! It looks like the phone OS is munging the URLs for some reason.

[Samer]

Do you have an ad blocker on your phone by any chance? The app is technically just using a Chrome webview so it’s likely some sort of hook is there mangling the URL by accident? I did review all the lines of code from the string “/js” to the actual command being sent and see no way for it to gain an extra slash. Anything is possible, but it isn’t apparent to me at this moment.

The weird thing is you said the browser on the same phone works fine and the issue is localized to the app. Have you tried just deleting and re-installing the app? You can even install the latest build of the app from here: https://build.phonegap.com/apps/1373249/install if you want to test. You will need to enable 3rd party app source in order to install from there.

Thanks!

[ipilcher]

You can even install the latest build of the app from here: https://build.phonegap.com/apps/1373249/install if you want to test.

Worked the first time. Unbelievable!

Now I have a question about your setup … since your proxy is at https://my.domain.com/os (rather than just https://my.domain.com), are you able to use the app? If so, how do you do that? (I.e. where does the “/os” go on the “New Device” screen?)

Thanks!

[Samer]

Yes, the app works well with subdirectory type proxies. In fact, it’s what I have been using for over two years now 🙂

Glad it’s working now! That version will hit the app store very soon, just finishing up testing on the new version.

[ipilcher]

Yes, the app works well with subdirectory type proxies. In fact, it’s what I have been using for over two years now 🙂

But how do you tell the app about the subdirectory?

[Samer]

Oh sorry, just put the following for the IP: my.domain.com/os

[ipilcher]

Oh sorry, just put the following for the IP: my.domain.com/os

Ah, that makes sense. I think you need to rename that field.

Thanks again!

[Samer]

Probably so, it originated with just an IP and quickly allowed any URL really as it just concats http or https to form the full URL. Thanks again for all the information which made this easy to debug!

[Author]

Posts