[Brian]

Sign me up!

[Brian]

Hi Ray,

I assume you want us to capture the network traffic I/O from the OS’s Ethernet port, and not a requesting client? This will require either adding a hub (not a switch) between the OS and the TP-Link/powerline adaptor or enabling port mirroring of the OS Ethernet port if you have a managed switch. You would then need to connect to an open port on the hub, or the mirror port on the switch using another machine with wireshark/tcpdump (with it’s NIC in promiscuous mode) to capture the traffic.

[Brian]

I have also had this happen a few times with my HW v2.3, FW 2.1.4 OS. Pings start getting flakey, and most API calls timeout or fail, but some small percentage of the traffic still gets through though. It doesn’t appear to be related to the TP-Link wireless bridge, as rebooting the OS (via API call) immediately resolves the issue. I have attached a graph from my Cacti ping monitor for the OS before, during and after the issue appears.

Attachments:

1. 

   OS-cacti-ping.png
   

[Brian]

Thanks again for the MH module hplato!

You do raise a good point on the weather data. It would be great to get access to this for other HA/IoT purposes. Since we are limited to 500 API calls per day and 10 API calls per minute with the wunderground’s free developer service plan, using the same API key with another consumer service will likely push the request volume/rate above that which is permitted under the service plan. With (cloud-based) ET support on the horizon, is this something better suited for the opensprinkler.com (or our own hosting environment if desired) cloud APIs to support as it could cache the wunderground data for requesting clients w/o increasing the wunderground API request volume/rate?

[Brian]

Just weighing in on this, but I could also see a value of having a program run between specific dates.

As William alluded to in the thread here, the city of Orange, CA has as part of it’s watering restrictions, limits on the number of days per week one can water that changes from one day per week in Nov-March to two days per week in Apr-Oct. This could easily be achieved with two sets of programs for each of the corresponding range of months. There are other cities in California that also have similar day per week changes during the year that could benefit.

Another use case that I personally would benefit from is related to having a different watering schedule that would be used for two weeks immediately after planting the vegetable garden (to allow the plants to take root and establish), and a regular schedule that would be used for the rest of the growing season. As with the previous example, this can be accomplished by manually enabling and disabling two different programs, but it would be great to have this automated by schedule.

[Brian]

I totally agree that the rules the state/city/local providers have published are confusing to say the least. Some rules are in conflict, and it’s hard to say who’s rules take precedence. As we are the ones that are required to comply with these rules, it certainly makes it difficult to do so.

My local provider (San Jose Water Company) recently published a table and surcharge plan (that hasn’t yet been approved by the PUC as of yet) that gives some specifics about how they plan to meter usage and charge for overages. I went back through the last years worth of bills, extracted our water usage and compared with the table (a rough estimate as we are billed every two months and the table is monthly). We were significantly over in all but the past billing period (which could be explained by repairing a leaky toilet flapper and irrigation scheduling changes around the end of the year). I would love to be able to automate this and send warnings if usage was trending towards running over, but our water meters are mechanical 🙁

[Brian]

Thanks Ray,

So there is no need to specify the baud rate with -b?

It was weird. After battling the OSX driver issue in my earlier attempt using the Arduino IDE on OSX, I was eventually able to get the USB passthrough working for the VM (resulting in the debug trace in the first post of this thread). For the attempt this past weekend I kept having a problem when attempting to enable passthrough for the device it gave me an error that the device was already in use (despite multiple host and VM reboots). I have since unloaded the CH340 driver and found an (unsigned) alternative that folks seem to have success with on Yosemite. If that also doesn’t work, I’ll try unloading that driver and just using USB passthrough to the VM.

[Brian]

Hi Samer,

Perhaps I am not including sufficient detail to describe what I am observing.

The test environment consists of two client devices, the first being an Android phone running v1.4.2 of the OpenSprinkler mobile app, the second being a Chrome browser v42.0.2311.135 running on OSX Yosemite.

Test procedure:
1) Open the URL to the OpenSprinkler from the Chrome browser
2) On the Chrome browser, navigate to Edit Programs from the grid on the lower-right of the browser frame (the current set of programs are visible)
3) Launch the OpenSprinkler mobile app on the Android device
4) On the mobile app, navigate to Edit Programs (the current set of programs are visible)
5) On the mobile app, add a new program and save (the current set of programs, including the newly added program are visible)
6) On the Chrome browser, navigate to View Logs from the grid on the lower-right of the browser frame.
7) On the Chrome browser, navigate to Edit Programs from the grid on the lower-right of the browser frame (the previously viewed programs are visible, but the newly added program added from the mobile app is not visible)
8) On the Chrome browser, reload the current page (the current set of programs, including the newly added program are all visible)

[Brian]

Hi Samer,

I was finally able to test the caching changes in the 2.1.4 firmware, and unfortunately the problem persists. To test, I added a new program from the mobile interface, but was unable to see the new program on the browser until I reloaded the page.

Looking in Chrome’s debugger, I only see periodic calls to the jc and jo APIs, but only see a call to jp after reloading the page. I do, however, see the change to HTTP 1.1 and Cache-Control headers when the jp call is made. This suggests something else is going on as browser cached files typically appear in the debugger’s network timeline as such. This is likely something to do with the JavaScript handling this portion of the user interface, but I haven’t dug into it yet.

[Brian]

Hi Ray,

I managed to flash the 2.1.4 firmware but it took some doing.

After multiple attempts upon returning from my trip, I couldn’t get the OS firmware updater to work properly on my Mac, or on the virtualbox VM (hosted on the same machine) due to issues with the USB serial port. The issue appears to be related to a problem with the CH341 driver for OSX Yosemite (downloaded from here: http://raysfiles.com/drivers/ch341ser_mac.zip as referenced in the firmware update instructions PDF: https://github.com/rayshobby/osFWUpdater/raw/master/Instructions.pdf). As with my earlier attempt using the IDE, the avrdude process spawned by osFWUpdater seemed to get stuck when attempting to communicate with the OS, and would only end if killed. The OS firmware updater binary is 64 bit and would not run on the VirtualBox VM. I also failed at getting it to compile on this VM due to what appear to be QT4 library incompatibilities.

I then attempted to use an old netbook running Ubuntu 14.04 LTS which I had previously attempted and failed at compiling the OS firmware on with the 1.0.5 Arduino IDE. Since this machine is also 32-bit, I had to re-compile the OS firmware updater, but also ran into issues compiling (see below). What ultimately ended up working was using avrdude that was included in the 1.0.5 Arduino IDE I had previously installed, and the firmware included in the OS firmware updater package for my hardware version as follows:

sudo avrdude -v -v -v -v -p atmega1284p -c arduino -P /dev/ttyUSB0 -b 115200 -U flash:w:Firmwares/OpenSprinkler_v2.3/firmware2.1.4.hex:i

I haven’t tried uploading the pre-2.1.4 firmware I compiled on my Mac/VM IDEs, but have a fairly high degree of confidence it would work.

The compile error for the OS firmware updater follows:

brudy@brudy-mini210:~/Downloads/Firmware-Updater-master/Source$ qmake
brudy@brudy-mini210:~/Downloads/Firmware-Updater-master/Source$ make
g++ -c -pipe -O2 -Wall -W -D_REENTRANT -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/share/qt4/mkspecs/linux-g++ -I. -I/usr/include/qt4/QtCore -I/usr/include/qt4/QtGui -I/usr/include/qt4 -I. -I. -o mainwindow.o mainwindow.cpp
In file included from /usr/include/qt4/QtGui/QComboBox:1:0,
from ui_mainwindow.h:16,
from mainwindow.cpp:2:
/usr/include/qt4/QtGui/qcombobox.h: In member function ‘void MainWindow::on_btnDetect_clicked()’:
/usr/include/qt4/QtGui/qcombobox.h:233:10: error: ‘void QComboBox::currentIndexChanged(int)’ is protected
void currentIndexChanged(int index);
^
mainwindow.cpp:51:66: error: within this context
ui->cmbDevice->currentIndexChanged(myHandler->curr_device);
^
make: *** [mainwindow.o] Error 1

[Brian]

Hi Ray,

The -D was added by the IDE, not by me intentionally, but I can try running avrdude from the command line and exclude this option once I have access to my OS again in a couple weeks.

 

[Brian]

Thanks Samer,

This looks to be an issue with the CH341 USB serial drivers for Yosemite, as I experienced a few hangs/crashes during my many attempts to get the upload to work last night. I am currently setting up another build environment on a Linux box to see if I can work through this apparent driver issue.

[Brian]

Thanks Samer,

I didn’t see a recurrence of the 403 error in my attempts at testing the new code this evening. Unfortunately I am having an error when attempting to upload the compiled code from the Arduino IDE (1.0.6) on my Yosemite 10.10.3 Mac. I’m an Arduino noob, so please excuse my ignorance if the answer to this is obvious.

I loosely followed the instructions here: https://opensprinkler.freshdesk.com/support/solutions/articles/5000165132-how-to-compile-opensprinkler-firmware however this doesn’t appear to have been updated to reflect the unified firmware yet. After installing the IDE, I grabbed the main sources from here: https://github.com/OpenSprinkler/OpenSprinklerGen2 and the aopensprinkler hardware description for my OS v2.3 from Ray’s repo here: https://github.com/rayshobby/opensprinkler. I then selected the Examples->OpenSprinklerGen2->mainArduino, selected Tools->Board->OpenSprinklerHW2.3, then selected my serial port (/dev/tty.Repleo-CH341-00001014, after first loading the drivers referenced on the firmware update instructions here: https://github.com/rayshobby/osFWUpdater/raw/master/Instructions.pdf). I then Verified/compiled the code without issue. Finally, I attempted to upload the compiled code from the IDE, but received the following (with verbose logging enabled):

/Applications/Arduino.app/Contents/Resources/Java/hardware/tools/avr/bin/avrdude -C/Applications/Arduino.app/Contents/Resources/Java/hardware/tools/avr/etc/avrdude.conf -v -v -v -v -patmega1284p -carduino -P/dev/tty.Repleo-CH341-00001014 -b115200 -D -Uflash:w:/var/folders/d1/h0z2v6cs2wxc1b96112l7qw9m603wb/T/build3006029794844066497.tmp/mainArduino.cpp.hex:i

 

avrdude: Version 5.11, compiled on Sep  2 2011 at 18:52:52

Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/

Copyright (c) 2007-2009 Joerg Wunsch

 

System wide configuration file is “/Applications/Arduino.app/Contents/Resources/Java/hardware/tools/avr/etc/avrdude.conf”

User configuration file is “/Users/brudy/.avrduderc”

User configuration file does not exist or is not a regular file, skipping

 

Using Port                    : /dev/tty.Repleo-CH341-00001014

Using Programmer              : arduino

Overriding Baud Rate          : 115200

avrdude: Send: 0 [30]   [20]

avrdude: Send: 0 [30]   [20]

avrdude: Send: 0 [30]   [20]

avrdude: ser_recv(): programmer is not responding

avrdude: stk500_recv(): programmer is not responding

 

After reaching 95% of the upload the OS restarted, but it doesn’t appear to be running the new code.

Any ideas?

[Brian]

Thanks Samer,

I currently don’t have an OS build environment set up to test the changes, but will see what I can do this evening.

 

 

[Brian]

Thanks Samer,

I had a quick look at the headers that are being returned by OS and noticed something that is likely contributing to the caching issue. It would seem that some of the standard cache control headers are not being set in the JSON responses that likely invokes some default caching behavior on some browsers. Currently OS 2.1.3 is setting “Pragma: no-cache”, in the response but nothing is set in the browser request. The HTTP 1.0 spec specifies either setting “Pragma: no-cache” in the request, or the “Expires” and “Last-Modified” dates in the past in the response so the data is not cached.

Since the OS is responding with HTTP 1.0, the kitchen sink “Cache-Control:max-age=0, no-cache, no-store, must-revalidate” response header for HTTP 1.1 will likely not work (unless going through a HTTP 1.0->1.1 proxy), but may be worth adding for those using a reverse HTTPS->HTTP proxy to protect their OS from snooping or MITM attacks.

[Brian]

I just hooked up my HW 2.3 OS this past weekend (it ran it’s first scheduled program earlier this morning) and haven’t seen any of the problems you are describing thus far. The only oddity I experienced was related to DHCP and the TPLink wireless adapter (OS lost the original sticky IP I set on the DHCP server), but I chalked this up to weirdness with my wireless access point as it hasn’t reoccurred since rebooting my access point (and my OS and TPLink).

My current setup doesn’t let me check if OS is ARPing with multiple macs, so I can’t report on that one.

[Brian]

Thanks again for the assistance Samer!

FYI: I did a quick RBL and SPF check for the client IP that is in the messages that made it through to my Gmail account for my recent hardware order and didn’t spot anything obviously wrong.

[Brian]

Do you have any tips or tricks for logging in to the Android app with an OpenSprinkler.com account that uses 3rd party SSO authentication (i.e. Google/Facebook)?

 

FYI: While this isn’t app related, I originally tried signing up for an account on OpenSprinkler.com with another e-mail account, but never received the verification e-mail to complete the account setup (I did check my SPAM trap). I ended up resorting to using my google account to sign up instead.

[Brian]

The MD5 password hashing feature was added in 2.1.3, and appears to be required now (cleartext passwords will no longer work). Ray has a brief writeup on the change in the 2.1.3 firmware release announcement.

[Brian]

Have you tried with the password MD5 hashed?

[Brian]

In addition to the state-wide water use restrictions in California, my water supplier is recommending (but not yet enforcing) a two day per week irrigation maximum and even/odd day watering schedules (based on postal address and day of month). Further details can be found here: http://www.sjwater.com/news/topic/santa_clara_valley_water_district_sets_new_conservation_target/

[Author]

Posts