Announcing OpenSprinkler Unified Firmware 2.1.9(4)

[Ray]

What’s New

Firmware 2.1.9(4) is now available. The primary change in this firmware is the initial support for MQTT (thanks to the generous contributions by jbaudoux, PeteBa, and vsoltan). To use MQTT, this firmware requires app/UI version 2.2.0. In particular, MQTT configurations are now available in Edit Options -> Integration, where you can set MQTT broker/host, port, and (optionally) user name and password if your broker requires authentication. Details about how to subscribe to MQTT can be found in OpenSprinkler’s support website: https://openthings.freshdesk.com/support/solutions/folders/5000099525

Limitations: The firmware currently only support publishing to MQTT broker — it does not yet support subscribing. Also, the root topic is currently always opensprinkler. Work is already underway to allow users to customize the root topic (to provide better support for users with multiple OpenSprinkler devices), and also to support subscribing. These will be made available in the upcoming firmware releases.

A minor change in this firmware is support for the second sensor (SN2) for OpenSprinkler Pi (OSPi) 1.5.

Eligible Hardware

As before, this firmware is only avialable for OS 2.3, 3.x, and OSPi. It is NOT available for legacy OS (1.x, 2.0, 2.1, 2.2).

Before you upgrade

Because the main change in this firmware is support for MQTT, if you don’t need to use MQTT, there is no need to upgrade.

Assuming you are upgrading from a previous minor revision of firmware 2.1.9 (such as minor revision (1) (2) (3)), this firmware will NOT trigger a factory reset.

If you are upgrading from a previous major revision like firmware 2.1.8 or prior, we suggest that you save a copy of configurations before upgrading, because upgrading across major revisions will trigger a factory reset. Having a copy of configurations will help quickly recover settings and programs after the upgrade.

If you use OpenSprinkler mobile app, make sure it’s at least version 2.2.0 as prior versions do not support MQTT configurations.

How to upgrade

Please follow the instructions here to update:
https://openthings.freshdesk.com/support/solutions/articles/5000381694

Support questions

While we’ve done our due diligence at testing the firmware internally, there may be bugs that we didn’t discover, particularly those that require long-term testing. If there are technical issues, please don’t freak out. You can post your comments and questions here, or submit a support ticket. In the worst case you can always downgrade to the previous firmware. All prior firmware files are still available in the archived folder.

[Alan Tschirner]

Good morning Ray,

I have been very happy with version 2.1.3 since installing opensprinkler a number of years ago. I haven’t had a reason to upgrade until a recent inability to connect issue so I didn’t.

Yesterday after studying all of your available material I took a deep breath and updated to 2.1.9.4 using a Windows 10 laptop. The upgrade process was painless and worked first try except when I attempted to login there was no webpage. I was able to ping the controller and my router saw it. The LCD display showed the DHCP IP address. I tried a factory reset but still no webpage.

I dropped back and uploaded 2.1.7 and the webpage was accessible. After importing my configuration file normal operation and schedules are restored. I will continue to monitor for connection issues but I am wondering where I went wrong with the attempt to upgrade to 2.1.9.4. Do you have any insight of possible issues? Looking forward to your reply.

Thank you, Alan

[tmittelstaedt]

Alan, I’m running OSPi 2.1.9 it only works for me on Chrome and Microsoft’s new Chromium. You may need to delete all your cookies also in the browser to get it to come up.

[Ray]

In our support website, we’ve added a Troubleshooting and Technical Help page:
https://openthings.freshdesk.com/support/solutions/articles/5000859137
your question is under the paragraph of “I got a blank page when accessing the controller’s homepage”. Please check the suggestions there.

[Bigboat]

so just quick question i sub’d to root i wanted to see flow and i get back the following after running 2 stations:

offline
online
{"state":0,"duration":120,"flow":0.00}
{"count":5,"volume":14.15}
{"state":1}
{"state":0,"duration":120,"flow":0.00}
{"count":7,"volume":19.81}

You see the flow topic listed a count and volume. why does the station say flow 0.00?

[Michael]

i’ve had this error for a while since 2.1.7 … when I execute the debugger I get a
{fwm:219}

response. Since it is truncated it seems like some sort of memory issue. I’ve reset multiple times. Would love to run a more viable debugger to help fix this as it is very frustrating.

[Ray]

@Bigboat: the per-zone flow rate and the global flow count are implemented slightly differently. Specifically: because flow rate at the beginning of a zone run is often not stable, the firmware has a logic that only starts counting flow past the first 90 seconds:
https://github.com/OpenSprinkler/OpenSprinkler-Firmware/blob/master/main.cpp#L98
the total count and volume numbers reported do not have this logic. Since the zone runs in your case seem pretty short (120 seconds), it’s possible that in the last 30 seconds there just weren’t enough clicks to calculate a meaningful flow rate, resulting in 0.00 value reported.

[Ray]

@Michael: {“fwm”:219} is not a truncated result — it is a complete json result. This is what the firmware returns if the HTTP API command you send does not contain device password or has the wrong device password.

[Michael]

so is the password being corrupted, because on reboot the command works just fine and provides a complete debug output. I’m going to disable the password and see if this continues.

[Bigboat]

@Ray thanks for the explanation. It makes sense. My concern comes from even after a full station run, which in the attached was 26m, the bottom of the screen still says flow 0.00. So I feel like something is amiss with the per station flow calculation.
When there is truly no flow I actually don’t get a “Total Water Used:..” line in the log. Which is how i previously knew there was no flow.
Unfortunately there is a neighborhood pump here that is frequently down.
Luckily with the recent MQTT additions I’m now just checking the rough expected volume from subscribing to opensprinkler/sensor/flow and I’ll use that to send me an email if the main pump is down.
But in the future I wouldn’t mind checking individual stations to see if there is a broken head but they always report 0.00 at the end.

Attachments:

1. 

   Screenshot_20200630-124824.png
   

[Bigboat]

and just to clarify what i was hoping to do….
I was thinking the flow per station was some sort of average. Starting 90secs into it is great. I was going to get that avg flow and the time the station ran and save the avg flow per minute. Doing this for each of my zones. Then in ea of my zones I was going to pop off a head and get new avg. Anytime I saw that difference per station run I was going to shoot off an email to me to check the heads. And the reason I was going for a 1 min avg (maybe will do 2min or 5min) is because station times will vary based on weather adj.

[pirracas77]

Hello.

I’ve just update to the new firmware.

I was trying to set up MQTT configuration.

I put broker IP, choose ENABLE, user/password… When I click on submit nothing happends. No messages are sent to the broker. Additionally when I open again the MQTT set up window, this is empty again.
What am I doing wrong?

Thanks in advance.

[EDITED]: ok, I forgot to save the new configuration LOL

[dkkim]

Hello All,

Has anyone seen sensor events actually logged individually to MQTT?
ie. opensprinkler/sensor1
and opensprinkler/sensor2

I’ve got 2.1.9(4) running on an OSPi, commit SHA f0b800, and I’m seeing other events but not these.

Looking at the source, it’s a bit hard for me to follow with all the enums and such, but it looks plausible and I don’t see anything commented out.

Thanks,
David

[dkkim]

Hello All,

Thanks to Ray (see: https://opensprinkler.com/forums/topic/easy-debug-setup/#post-67770) I was able to get some more debugging info. Also looked at code further.

I figured out that raw events for sensor1 (used for flow in the OSPi 1.4) are not actually logged via MQTT since the logic does not check for a SENSOR_TYPE_FLOW, only for RAIN or SOIL types. If I make the following patch, then it works:

 /** Read rain sensor status */
 void OpenSprinkler::detect_binarysensor_status(ulong curr_time) {
        // sensor_type: 0 if normally closed, 1 if normally open
-       if(iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_RAIN || iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_SOIL) {
+       if(iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_RAIN ||
+               iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_SOIL ||
+               iopts[IOPT_SENSOR1_TYPE]==SENSOR_TYPE_FLOW) {
                pinModeExt(PIN_SENSOR1, INPUT_PULLUP); // this seems necessary for OS 3.2
                byte val = digitalReadExt(PIN_SENSOR1);

Note that the reason I want to do this is to read raw flow sensor events for fine-grained water use tracking and leak detection in my hybrid surface/buried drip system. I have a 1 gallon/cycle reed switch flow meter, so events aren’t too frequent. MQTT seems a great way to glue all this together.

Question for Ray and the other devs:
Was the intention to omit flow sensor raw events from MQTT? Or is that an oversight? I.e. Is there any reason not to enable these as I did?
If you are amenable, I will prepare a pull request…

Thanks,
David

[Ray]

Unlike other sensors like rain or soil sensor, flow sensor will generate a large number of clicks (it can be tens or even hundreds per second), it’s not feasible to send notifications each time the flow sensor clicks. I don’t think it’s a good idea to send sensor1 events if sensor 1 is configured as flow. Instead, the current approach is that it will send one flow sensor notification when a zone completes running.

If you want to sample and send flow sensor values, that’s fine. It’s not currently implemented in the firmware but it can be easily added. Again, it should not be very frequently otherwise you will receive a large number of notifications which is not ideal.

[dkkim]

Thanks Ray, I was wondering if the thinking were something like that.
Since I get 1 cycle/gallon, it’s not too much IoT network overhead, esp. if I’m on a local network. And it also helps give me real-time alerts if a mainline hose pops loose, etc.
However, I can imagine some much higher-rate sensor being problematic.

Does a good method suggest itself to you to elegantly support both cases? I can maintain a side-branch for my own purposes, but I would imaginee it could be useful to others, and it’s more convenient to not keep re-basing my own branch on releases.

Would it make sense to add an additional MQTT channel something like /opensprinkler/sensor/rawflow or flow-ticks?
And/or maybe have the message send conditional on a configuration setting that defaults to off?
That way we could add this reporting option to the regular firmware without overwhelming folks with high-rate sensors.

Thanks for your quick and expert response, and the cool OpenSprinkler effort!

David

[dkkim]

A follow-up.
The simple change I made in OpenSprinkler::detect_binarysensor_status above (https://opensprinkler.com/forums/topic/announcing-opensprinkler-unified-firmware-2-1-94/#post-67771)
solved the lack of MQTT event sending but unfortunately results in multiple spurious-appearing entries in the visual Log (see tick marks under “Rain Sensor” below).
So it looks like something in the UI layer (which appears abstracted into .js and .css on ui.opensprinkler.com rather than in Github) treats sensor1 and sensor2 events as representing Rain.
It’s hard to tell why this happens, but it doesn’t seem correct as this routine is trigger by flow, soil, and rain sensors.

Any thoughts?

Thanks,
David

Attachments:

1. 

   Screen-Shot-2020-08-07-at-7.16.19-PM.png
   

[Samer]

Just wanted to mention the UI is available on Github here: https://github.com/OpenSprinkler/OpenSprinkler-App.

[Ray]

I don’t think you should use “detect_binarysensor_status” to handle flow sensor. This function is meant for slow changing events like rain or soil sensor activations / deactivations. In addition, they are controlled by the ‘on delay’ and ‘off delay’ parameters to filter out noise / false triggerings.

As I said, flow sensor events are much more rapid and the sensor can fire multiple clicks per second. This will generate a large number of events. If you treat flow events the same way as rain / soil (which is what binary sensor is referred to), the UI will not display it correctly.

What you should do is to take the average number of clicks over a period of time and log/sample that number. The ‘flcrt’ parameter can be used for that — it counts the number of clicks within each 30-seconds window. You can certainly modify the firmware and the UI code to log this paramerter.

[dkkim]

@Ray thanks for the reply. If the intent of that routine is for slowly-varying parameters, and the UI implicitly reflects this, that makes sense.

I am a bit puzzled in that I chose a dry-contact reed switch for the flow sensor, as recommended, and I think these generally have lower click rates…? But I realize some people are also using Hall effect sensors and other sensors that might have different intrinsic scales, some much faster.
In your opinion, does it make more sense to log the flcrt (os.flowcount_rt, I think) every interval, or perhaps log the total counts, or perhaps even a rate with a wider window? I would image a different MQTT topic at some reasonable granularity… or maybe even have it be a switchable parameter or conditional compilation setting (if you think this is a low-frequency use case).

I’m essentially trying to create a raw-ish data stream to do analytics on usage and also detect small mainline and pre-valve fitting leaks. Logging flow only for scheduled runs hides these flows amidst the much larger flows. Also a leak can be subtle as the absolute amounts vary with rain adjustments. Having access to a background “stream” (albeit at a reasonable rate) can facilitate these analyses.

@Samer thank you! I will look further into the UI, to understand, though might be moot given what Ray says about the implicit UI design.

Traveling for a week, so will read/experiment thereafter. Thanks again for the quick help!

David

[Ray]

Well, by ‘slow-varying’ sensors I mean those whose statuses don’t change more than once per few hours. Flow sensor certainly does not fall into this classification.

If you are looking to publish to MQTT every time flow sensor clicks, and you are sure your flow sensor clicks slowly, it may be feasible. You probably want to insert code into the flow_poll function instead of binarysensor_status function, because as I said binarysensor_status is meant for rain and soil sensors, and they are affected by the sensor on/off delays which you don’t want to apply to flow sensor. Another consideration is that while publishing to MQTT is ok, don’t log flow events to the controller’s log file because that will generate too much log data. If you have OSPi this is probably ok, but microcontroller-based OpenSprinkler does not have much flash memory space, logging raw flow events will quickly fill up the space.

‘flcrt’ is always available, regardless of whether a zone is running or not. So checking flcrt is a good way to detect, for example, water leaks or water usage at any time. If you want to log the total click count, that’s also fine — flcrt is just the rate of change of total click count. But keep in mind that the total click count is volatile — once the controller reboots it will reset to 0.

[George]

Hi. I will update firmware this weekend…. can’t wait for MQTT!
at some point it would be nice to have the mA value when the station is activated. This is a “simple” way of detecting if a Zone is not working.

[George]

Well, I got MQTT working and now I get an email when the program starts and finished and if for some reason a station’s mA is <25 I get an email. With this I can now know if a station has a problem.
Thanks Ray!

[slick]

Hello, and Happy New Year.

I’m trying to compile this version using the instructions provided.
I have been successful using earlier versions.
One of the differences I see is the addition of the Arduino/libraries/pubsubclient code that I’ve cloned from git like I did for the esp8266_2.5.2 code.

When I compile using make -f make.lin32, I get the following (among several others, but this is the first error):
/home/brian/Arduino/libraries/pubsubclient/tests/src/receive_spec.cpp: In function ‘int test_receive_stream()’:
/home/brian/Arduino/libraries/pubsubclient/tests/src/receive_spec.cpp:65:12: error: cannot declare variable ‘stream’ to be of abstract type ‘Stream’
Stream stream;
^
In file included from /home/brian/esp8266_2.5.2//cores/esp8266/HardwareSerial.h:32:0,
from /home/brian/esp8266_2.5.2//cores/esp8266/Arduino.h:263,
from /home/brian/Arduino/libraries/pubsubclient/src/PubSubClient.h:10,
from /home/brian/Arduino/libraries/pubsubclient/tests/src/receive_spec.cpp:1:
/home/brian/esp8266_2.5.2//cores/esp8266/Stream.h:38:7: note: because the following virtual functions are pure within ‘Stream’:

Any suggestions would be greatly appreciated.
Sorry for the newbie questions, but I’m a hardware guy 🙂
Thank you for all you’ve done on this topic.

[Smartin]

What an amazing piece of software this is!

I’ve had no real C++ experience but learned so much by modifying it. But I’ve managed to add a NOTIFY_CUSTOM_SENSOR to your “push_message” MQTT-method and call it in the main loop() debug window to send I2C-sensor data. After resolving the server.h Server.h bug on Windows it was basically a breeze!

I wanted to thank you all wholeheartedly and especially Ray and Samer for all your hard work on this great project!

With love from Austria,
Martin

Attachments:

1. 

   setup.png
   

[Author]

Posts