[Ray]

Unlike other sensors like rain or soil sensor, flow sensor will generate a large number of clicks (it can be tens or even hundreds per second), it’s not feasible to send notifications each time the flow sensor clicks. I don’t think it’s a good idea to send sensor1 events if sensor 1 is configured as flow. Instead, the current approach is that it will send one flow sensor notification when a zone completes running.

If you want to sample and send flow sensor values, that’s fine. It’s not currently implemented in the firmware but it can be easily added. Again, it should not be very frequently otherwise you will receive a large number of notifications which is not ideal.

[Ray]

You can turn on debug prints. To do so, open defines.h, at the top, uncomment the line that says ‘#define ENABLE_DEBUG’. Save, and recompile the firmware.

To see the prints, you need to run the firmware manually. First, stop the background task by:
sudo /etc/init.d/OpenSprinkler.sh stop

then in the folder where you installed OpenSprinkler firmware, do:
sudo ./OpenSprinkler

[Ray]

I don’t recall seeing this error before. But a couple of things may be related:

1. As I said, you should delete ‘test’ or ‘tests’ folders in all relevant libraries, such as pubsubclient. I find them sometimes lead to compilation errors.
2. This step you described:

git clone https://github.com/esp8266/Arduino.git
#Puts it in ~/Arduino

I am not sure where this is coming from — you clone ESP core again into ~/esp8266_2.5.2 in the next step, so why duplicate the core into ~/Arduino? In the error messages you received, there is this line:
“Arduino/libraries/ESP8266WiFi/src/WiFiClientSecureBearSSL.h”
it implies that the compiler is looking for ESP8266 core files in ~/Arduino/libraries folder. This is bizarre, it should be looking for that in ~/esp8266_2.5.2 since that’s what you set as ESP_ROOT in the Makefile.

[Ray]

Update: I can confirm this issue: it happens when using W5500 firmware and setting the controller in static IP mode. This is due to the problem that the previous way of detecting Ethernet module for ENC28J60 (checking a register value) does not work for W5500, so even if W5500 is unplugged it ‘thinks’ an Ethernet module is plugged in. That in conjunction with the fact that when you set static IP, it bypasses DHCP process, results in this problem. Setting it back to DHCP mode should resolve the issue (i.e. when the controller cannot establish IP through DHCP, it will revert back to WiFi mode).

There are two ways to reset it back to DHCP:
– if you can still access the controller with W5500 module connected, go to homepage -> Edit Options -> Advanced, and check “Use DHCP”
– if you can’t access the controller, you can use buttons to change the option values. Specifically, power off the controller, then power it back on, as soon as you see the OpenSprinkler logo, press and hold the third pushbutton (B3) and continue holding it until you see the LCD displays “Setup Options”. Then click B3 a few times until you see ‘Use DHCP’? Use B1 or B2 to change the answer to Yes. Then press and hold B3 until the controller reboots itself. This will set it back to DHCP mode.

[Ray]

One possibility is that some of the libraries have a ‘test’ folder, which needs to be deleted otherwise they cause compilation errors. I think you didn’t post all the errors so I can’t be sure, but if you see any error pointing to some /test folder, then you should delete those test folders.

Also, the original UIPEthernet libraries does not contain some of the improvements that Stefan and I made. You should use this modified UIPEthernet library (my own fork under /fixes/dhcp branch):
https://github.com/OpenSprinkler/UIPEthernet/tree/fixes/dhcp
I’ve tried to submit a pull request but the author did not seem to be taking pull request yet so I had to keep it in my own fork.

[Ray]

Per my post here:
https://opensprinkler.com/forums/topic/instructions-for-testing-os-3-2-with-w5500-ethernet-module/#post-67408
is your controller running Jul 18 version of the W5500 firmware? If not, you should update. If you are not sure, use the test API script and check the debug (/db) API, it shows the date the firmware was compiled.

There is no 2.1.9(7) for W5500 yet — all the minor revisions from (5) to (7) are meant to address ENC28J60 and none of them has to do with W5500.

[Ray]

Regarding log: there is a variant of the jl API that allows you to retrieve log data from any specified start date and end date. Details can be found in the API document:
https://openthings.freshdesk.com/support/solutions/articles/5000716363-os-api-documents

About ENC28j60: as I said in this post:
https://opensprinkler.com/forums/topic/controller-lockups-crashes/page/4/#post-67707
I’ve done everything I can to debug and the latest version is 2.1.9(7) available for both OS 3.2 and OS 2.3. If this still doesn’t solve the freezing issue, I don’t know what else to try — there is no means for me to debug since I cannot reproduce the problem.

[Ray]

When in double, please always go to:
support.opensprinkler.com
scroll down to ‘OpenSprinkler Pi’ section and check the OpenSprinkler Pi user manual. It contains information about voltage testing, and pin diagrams.

5V on VCC-GND is WRONG: VCC should be 3.3V, not 5V. OSPi does not generate VCC voltage — it’s coming from your RPi. It’s likely you may have misaligned RPi with OSPi when you plugged it in. I’ve seen a previous case where a customer toasted two RPis before realizing he misaligned RPi with OSPi when plugging it in.

[Ray]

The most recent code is in this branch:
https://github.com/OpenSprinkler/OpenSprinkler-Firmware/tree/dev/219-7

Apparently while working on revision (6) I turned off MQTT loop by mistake, so have to turn it back on and re-compile and name it revision (7). You probably didn’t notice this unless if you were using MQTT.

[Ray]

OpenSprinkler Latch uses the same firmware as other OpenSprinklers, and yes you can easily turn on/off a zone manually from the homepage. No programs will run unless if you configure some programs and enable them. The same HTTP API works with OpenSprinkler Latch just as the other versions.

The firmware does not support pressure sensor — if you need pressure sensor reading you will need to modify the code to read the sensor and output that through HTTP so it can be displayed on the homepage. The coding isn’t complicated, largely involving following the examples of existing code.

[Ray]

@Dennis: thanks for posting the work-around.

At the moment I am pretty lost what else to try. I have 3 test OS on my own network, one OS 2.3 DC, two OS 3.2 AC+Ethenret, two are directly connected to my router and one is corrected through a powerline Ethernet adapter. All three are running 2.1.9(5) and have been alive for more than 5 days since I flahsed 2.1.9(5) onto them. I run the Test script (http://raysfiles.com/os/TestOSManual.html) on all three of them, with a browser tab open on the side to show homepage status, and have IFTTT set up to receive notification on station runs. I have encountered a couple of cases where IFTTT notification was missing (debugging information shows it wasn’t able to connect to IFTTT server at that moment). But otherwise the 3 controllers have been running fine, no hanging, no issue accessing them.

So the issue some of the other users are experiencing is pretty much beyond my knowledge — the only way to find out would be to go to their home in person to debug the issue… The feature implemented in 2.1.9(6) debug version — performing a Ethernet reset when the microcontroller detects ENC28J60 registers are in one of the erroneous states, is akin to performing a reboot, though it’s a softer reboot than a power-on reboot. But it seems even with this, it doesn’t solve the problem for Water_my_lawn, and some of the register values reported, like ECON1=80 or 84, I have’ never seen these values on my controllers. In any case, I am pretty lost and I have to move on for now to other priorities and come back to it when anyone has more insight to what’s happening.

Regarding work-around, in addition to what Dennis mentioned (using a timer to trigger a power-on reboot once a day), I still think using a secondary router is an effective approach — there are inexpensive routers less than $20, it won’t affect access to the controller from the primary network as long as you set up port forwarding on the secondary router.

Another work-around, for OS 3.2 users, is to try W5500 Ethernet module (https://opensprinkler.com/forums/topic/instructions-for-testing-os-3-2-with-w5500-ethernet-module/). Though, this only works for OS 3.2 and doesn’t work for OS 2.3.

[Ray]

@slick: if I understand it correctly, you want to use the controller in AP mode (without another router). This is possible. You have to either use the OpenSprinkler mobile app, or as KSprinkler75 said, download the OpenSprinkler app from github, and run its index.html. This is NOT the same as typing 192.168.4.1/index.html — the correct approach is to open the index.html in OpenSprinkler app folder.

You can of course also write your own front end interface, using the OpenSprinkler HTTP API. In fact, as I explained in this post:
https://opensprinkler.com/forums/topic/useful-script-for-testing-opensprinkler-api/
the testing script is a simple front end (not much features other than just testing a subset of the HTTP API) and you can obviously improve it significantly to make it more usable.

[Ray]

@Water_my_lawn: no, the buffer is not the issue. The actual ether_buffer is 2x the size of ETHER_BUFFER_SIZE:
https://github.com/OpenSprinkler/OpenSprinkler-Firmware/blob/dev/219-5/main.cpp#L67
and all client.read is constrained to read maximum of ETHER_BUFFER_SIZE number of characters. Even in previous versions of the code, the buffer’s actual size is ETHER_BUFFER_SIZE+TMP_BUFFER_SIZE (which is 256). So I don’t think there is buffer overflow.

[Ray]

@Water_my_lawn: 29|41 matches the problem I was describing earlier, that ‘receiver error’ flag and ‘receiver buffer error’ flag are both been set. Once these are set, it seems eventually the ethernet controller will hang, though not immediately.

So based on the debugging information I collected, I’ve compiled a new firmware 2.1.9(6) with the following work-around: the firmware will periodically check enc28j60 register values, and if it detects any indicator of problems (so far the indicators are: EIR.RXERIF and ESTAT.BUFER are both set; or ESTAT.LATCOL and ESTAT.TXARBT are both set), it will issue a soft reset to enc28j60 to re-initialize its state. This is completely seamless to the user: you will NOT observe any reboot, programs will continue to run; all it does is to trigger a soft reset of the ethernet chip to recover the registers to initial states.

The firmware is at the same place before:
http://raysfiles.com/os_compiled_firmware/v3.0/experimental/
the name is “os_219_6_enc28j60_debug.bin”. I’ve also changed the debugging information displayed on the LCD a bit, to remove numbers which are now irrelevant. Once flashed, you should see on the top four numbers, for example it might show:
8|1|4|0
the first three are EIR, ESTAT, and ECON1 register values (all in HEX format), the last one is a count of how many re-initializations it has done so far.

It’s important to use the debug version, as only the debug version has the re-initialization logic I described. This firmware also fixes another issue we discovered just today, where an invalid NTP server/IP can cause the controller to get stuck in NTP syncing state (a rather rare situation that only happens if you’ve put in a wrong NTP server/IP).

At this point I am pretty much doing blind debugging — as I cannot reproduce the situations that Water_my_lawn and bena encountered, I am coming up with theories to address issues without actually being able to observe the issues. So if 2.1.9(6) still doesn’t solve the issue, I’m gonna admit it’s beyond my knowledge then and I don’t know what else to try 🙂

[Ray]

@beta: I am not sure what else to try honestly. So far your is the only case where even with W5500 it loses response. I honestly don’t know where to start to diagnose the problem.

[Ray]

@bena, the sequence of numbers are: the first four are ENC28J60 registers EIR, ESTAT, ECCON1 and ERXFCON (all in hex format); followed by a count that detects main loop timeout, and the last one is the available RAM (in KB).

Most of the numbers you have are the same with mine, but one is different after you said it becomes unresponsive: 13 (ESTAT). According to ENC28J60 datasheet, 0x13 means the following bits are set/flaged: Late collision, and Transmit abort. This probably explained why it’s not transmitting data out. Late collision is defined as collision that occurred after 64 bytes have been transmitted. I have never seen this flagged ever on my test controllers. I did some googling and found a few pieces of information here and there:
http://www.firewall.cx/networking-topics/ethernet/195-late-ethernet-collisions.html

I don’t know how to reproduce this situation since I’ve never seen it on my network.

[Ray]

It’s the same firmware, just turning on debugging, so I didn’t create a new minor revision because nothing else has changed. As I said, if the LCD shows the sequence of numbers at the top then that means your firmware is loaded correctly. That will allow you to tell if it’s the debug version or not.

If you use the API script:
http://raysfiles.com/os/TestOSAPI.html
and pull Json debug (/db) it will show the firmware’s build-time. That will provide the difference (the previous version has a different built time).

Also, I don’t know what’s the frequency of pinging you are doing, but I would suggest a slower ping rate (maybe once per minute or something) to begin with.

[Ray]

@Water_my_lawn: I said, the information is displayed on the LCD — meaning the LCD on the controller, not in the ‘About’ page (if network error happens, you can’t load the webpage so it won’t be useful to show debugging information there). Therefore the debugging information needs to be displayed on the LCD so that even if the webpage doesn’t load it still shows.

[Ray]

@Water_my_lawn: I’ve generated a version of 2.1.9(5) which prints debugging information on the LCD. it’s at:
http://raysfiles.com/os_compiled_firmware/v3.0/experimental/
and the firmware name is “os_219_5_enc28j60_debug.bin”. It prints a sequence of numbers separated by | at the top line of the LCD, basically containing various register values for ENC28J60. You can let me know that entire string 1) upon fresh reboot of the controller; and 2) after network error appeared, then by comparing how the values changed I may get some information about what’s going on.

[Ray]

“Is it difficult to use wired for over the air firmware update” –> yes, no one has ever implemented this. The WiFi OTA update is a built-in functionality of ESP8266 library so that works out of the box. The wired Ethernet was a fairly recent add-on that we only started selling last year. The firmware update feature for it does not exist and it’s going to take quite a bit of efforts to figure out how to do so. Honestly back when we had OS 2.x which only has wired Ethernet, everyone wants WiFi; now we have built-in WiFi, it seems people want wired Ethernet again. Almost all smart sprinkler controllers on the market use WiFi, and OpenSprinkler is probably among the very few that provides a wired Ethernet option. If you really want reliable wired Ethernet, I would say OpenSprinkler Pi is probably the best since it’s based on RPi which runs a full Linux system and it’s also lower cost than the microcontroller-based OpenSprinkler. You can perform over the air firmware update on RPi whether it’s connected through WiFi or wired Ethernet.

“Could it be that this library has not been rigorously tested on larger networks where there is a lot going on and many other programs issuing broadcasts” — sure, it’s possible. As you know, all our products are open-source and we rely on open-source libraries. But the downside is that some of these libraries are not used on commercial products so are only tested for hobby projects and not tested rigorously over long term.

“To me, turning on DHCP only when needed is not a real fix.” –> I don’t see why this isn’t a fix. The DHCP request usually finishes well within one second, and it only does so every few hours. The chance of the controller getting tons of broadcast over less than 1 second is extremely small. Also, if you use static IP, or set DHCP reservation on your router, then DHCP is completely off (or the lease time is infinite so after booting it won’t ever request DHCP again). Of course without knowing your specific network I can’t say this for sure, but again, if that’s an issue then why not choose OSPi.

“But the enc28j80 puts in the RX ring buffer almost every packet seen on network and the library must analyse the packet.” — this is NOT true — the ENC28J60 chip has hardware filters. When I say ‘disable broadcast’ it is setting a register bit on the chip so that broadcasts messages are completely dropped and are not put int he buffer. This is hardware level filtering, not software level.

If you haven’t tried firmware 2.1.9(5), I suggest you give it a try. If you are still on firmware 2.1.9(4) or earlier version, then the broadcast issue is likely what’s causing the hanging on your controller, which is what (5) is meant to address.

Finally, UIPEthernet is really the only library we can use for ENC28J60 now. The previous EtherCard library we’ve used for OS 2.3 is not available for ESP8266 (there is a forked branch that attempts to make it available for ESP8266 but last time I tried it failed). So there is no other library we can use for now. You can always try the W5500 Ethernet module as I described above.

[Ray]

OK, well at this point I probably will have to send you a custom firmware that prints debugging information on the LCD screen, to help understand the issue. I am mostly puzzled that you said Stefan’s firmware worked for you at least for several days. This firmware (2.1.9(5)) is very similar to Stefan’s firmware, except it turns on broadcast when processing DHCP. But if you are already using static IP, then it will never turn on broadcast. So I am puzzled why this works much less effectively than Stefan’s firmware. In any case, I can generate a custom firmware with additional debugging information on the screen to help understand the issue.

It’s certainly also possible that your ENC28J60 module has a defect, in which case you can send a support ticket so we can send you a new one (or you can buy one from amazon: https://www.amazon.com/ENC28J60-Ethernet-Network-Module-Arduino/dp/B01FDD3YYW).

Some other options to consider are: 1) use a secondary router (or VLAN if your router supports VLAN), as described in a post above); 2) try the W5500 Ethernet module as described in this post: https://opensprinkler.com/forums/topic/instructions-for-testing-os-3-2-with-w5500-ethernet-module/)

[Ray]

“This would seem to be based on the assumption that this section of code will be reached _AT LEAST_ once each second. If for some reason it took longer, that could explain the results I saw. ” ==> yes this is correct, and is likely the cause of the issue you are seeing. When the remote station feature was implemented, I was assuming only a very small number of zones are set up as remote zones and didn’t anticipate a situation where a lot of zones are set up as remote. So you are right that if one HTTP call takes longer than a second, then it would miss that cycle and would have to wait for the next cycle. The reason each remote zone was given 2*MAX_NUM_ZONES (in your case, 2*200=400) seconds of run time each time is so that it will get two chances before the remote zone shuts off. In any case, I think there is already a pull request in Github to address this issue so I will take a look as soon as possible. This can be easily addressed without much difficulty.

Next, the question regarding “why not give the remote zone the correct run time” — this is harder to do because of the way remote zone is implemented: it is at very low level, in OpenSprinkler::apple_all_station_bits function. It’s done this way so that the remote zone is guaranteed to trigger independent of how the scheduler is implemented. At that low level, it does not know what is the scheduled run time, all it knows is whether it should turn on the zone or turn it off.

And about the API document, yes will correct that part about /cm command.

[Ray]

First, to clarify: ‘Ignore Rain’ does NOT mean rain from online weather data, it means the manual rain delay that you manually trigger through the UI. This is explained in the firmware 2.1.9 user manual:
https://openthings.freshdesk.com/support/solutions/articles/5000716364
just search ‘Rain Delay’ in the PDF.

Ignore rain sensor is now called ‘Ignore sensor 1’ or ‘ignore sensor 2’ depending on whether you connected rain sensor to sensor 1 or 2.

Next, I am confused whether you do want your bamboo and tomatoes to be affected by watering percentage, or you do not want them to be affected. Generally I would imagine if you put them on your balcony, maybe you do not want them to be affected by watering percentage. If so, you should go to the programs and un-check ‘Use Weather’. If you want them to be affected by temperature and humidity, but not by rain, you can choose Zimmerman algorithm and give 0% to precipitation.

[Ray]

Can you provide some more details, like:
– does the controller respond to button presses?
– is the time displayed on the LCD correct?
– does it respond to ping?
Just seeing ‘network error’ does not necessarily mean it has lost response — the UI has a default timeout (3 seconds or something like that), if it doesn’t get response within 3 seconds, it will display network error. But it could be that it took 5 seconds for the response to come back. For that reason, it may be better for you to use the follow API test script:
http://raysfiles.com/os/TestOSAPI.html
which does not rely on the UI, and it just pulls data directly from the controller. If you get a response, that means the controller is working fine.

Also, just to double check the firmware is flashed correctly, go to About page and see if it shows 2.1.9(5). Or, if you use use the TestOSAPi script, check the /ja result and look for a variable named “fwm” see if it’s value is 5.

One last thing you can try is: if you haven’t done so, try set a static IP on OpenSprinkler: go to Edit Options -> Advanced -> turn off Use DHCP, it will auto populate the fields with your currently assigned IP and DNS, gateway etc, so you can just submit changes (or make edits if anything looks incorrect). Then do a reboot of the controller. This way it will disable DHCP and won’t send DHCP requests (which also keeps UDP broadcast disabled).

[Ray]

About flow meter pulse rate: yes we will change the resolution in the next firmware. For now, honestly this is just a scaling factor. You can leave it to 1 for now. The result is simply the pulse count multiplied by this number. If you can’t set it to 0.0001 just set it to 1, and remember to scale the number you read by 0.0001.

[Author]

Posts