Forum Replies Created
-
AuthorPosts
-
robohackParticipantThank’s for the UPS suggestion — though there’s no link, though I can find something that seems to match on Amazon.ca. I don’t know if it will fit, though it might. Having to hack into the OpenSprinkler case and find the 5vdc pads is annoying though. How do I know that a slight voltage difference won’t cause problems between the UPS and the built-in power supply? Is there already a protection circuit? There are too many unknowns to make me comfortable with this.
As for change freezes during production, well, yes, that’s what I will do, basically. The “pause” feature will work for some maintenance needs, provided the total runtime isn’t approaching 24 hours due to % watering budget compensation for the hot season. The ability to cancel zones without changing the schedule of following zones will also be used for maintenance and working in areas without getting soaked.
However having to wait 24 hours or more for a new zone to work, for example, is also rather annoying to say the least.
I.e. the same feature needed to recover after a reboot could also be used to rebuild the schedule after a program change, i.e. without having to wait until the next day or whenever the program restarts. I should be able to add a new zone to a program, or change the ordering (by changing the zone names), and have the schedule immediately reflect those changes.
Fail safe recovery into operating mode is still required for crashes too — unless you can somehow give me a mythical guarantee that the firmware will never ever crash!
robohackParticipantYou still need to do the simulation!
The last known good timestamp in NVRAM would just allow you to more easily calculate the exact start time for the simulation run — it would be the start time of the longest running program before the last timestamp.
But, yes, as you say if the NVRAM can’t handle too many writes, then it can’t be used for such a timestamp, even if it’s only once per minute.
So, the easy and sure good time to start the simulation is just before two runs of the longest running program. The simulation will have to run through more steps, but it will be impossible for it to miss scheduling any programs that would have been running at the reboot time or started during the downtime.
You probably know very well how futile it would be to try to convince any commercial controller manufacturer to change their feature set in any way no matter how interesting or valuable such a proposal might be — they’ve designed their firmware as much as a decade ago or more and have planned it to run unchanged in their product line for as many years to come as they can possibly manage while still making enough sales to justify manufacturing. Even most of the WiFi connected ones are not yet offering OTA updates so far as I know. There may be a kickstarter project or similar startup that’ll try do better, but if they try to go head to head with the big guys without also being open source, well they’ll die. An open source project like OpenSprinkler is the only niche where one can hope for a more proactive approach to firmware upgrades with new features and fixes.
If you’re doing well enough now in ongoing sales to keep supporting the project, imagine how much better you might do by being able to offer additional key features that the commercial controllers already offer. I’ve found in software-only open source projects that word-of-mouth is the primary driver for increasing user counts. You already support OTA updates, and if you can reach out to existing customers, help them update their firmware, then I’m sure you can get most of them to convince at least one of their neighbours to buy a unit as well. As is though the people I’ve talked to would be unwilling to give up their current controllers for one that can’t maintain operation for up to 24 hours after an unattended reboot, no matter how clunky and old their current controller is. I’m obviously not the best sales person when I point out a flaw in the product I’m trying to sell, but what else can you do when you know about such a problem?
robohackParticipantBTW, if you have room in non-volatile storage then you can keep a regularly updated timestamp there, and maybe also the current program(s) state too, then you’ll know exactly when to restart the simulation after a reboot.
robohackParticipantThat’s pretty big UPS for an outdoor enclosure — sure large enclosures are available, but the best UV resistant one I can find that would fit everything with a UPS is US$350, plus shipping, so that’s about CA$400.
So, that’s still an absurd solution to the real problem of just handling reboots properly.
robohackParticipantI don’t have a DC version — I’m replacing commercial Irritol controllers in a 24VAC system.
As I mentioned above there are clearly no existing separate power supply connections for the controller CPU alone (on the AC version), and as far as I’m aware there’s no battery charge controller already in the (AC) OpenSprinkler.
robohackParticipantSorry, but it seems like you’re not paying attention to the full set of concerns I have and their implications.
I did already consider creating one program per zone (indeed I researched doing it that way back before you added the ability to sort the run order by zone name because you did already allow easy re-ordering of programs — I posted about doing this in these forums), but that doesn’t allow for easily managing total run time when the system is nearly maxed out at 21 hours. It would also be extremely tedious to configure and then make changes with 32 zones — the UI is already heavy on requiring many clicks per operation. And so until you added the sorting feature I was still looking for alternative systems. The sorting feature was the key one that brought me back to looking at OpenSprinkler.
I’m already maxed out in time restrictions — as-is I can only run each zone once per day and just long enough for that zone. I’m not currently running any zones more than once per day, yet my program requires a full 21 hours, and during the hottest times of the year it will require a full 24 hours. I’d go for even more time if I could increase the % watering budget beyond that limit, but, well, there are only so many hours in the day.
I cannot run zones for just a few minutes multiple times per day — I need to run zones for enough time to overcome evaporation and to deliver water that will soak in; and I need to run zones in high-traffic areas _only_ at times when nobody can be expected to be there. There are not enough features in the current OpenSprinkler zone programming to build in such scheduling with those kinds of restrictions. It’s not unimaginable, but now you are talking serious amounts of additional code, not to mention storage for the additional values representing such restrictions.
I’m also concerned about reboots when the system is unattended — it is critical that it not sit idle for any significant portion of the day no matter whether there’s a power failure or a crash.
As for your imaginary miniature UPS, well there are obviously no existing separate power supply connections for the controller CPU alone, and as far as I’m aware there’s no battery charge controller already in the OpenSprinkler, and while I’m not opposed to a little bit more hardware hacking, that’s not something that was advertised as necessary. But even a UPS doesn’t solve the problem of reboots, either from crashes, or from necessary maintenance downtime.
If you don’t understand how a life support system needs to work in order to fail safely, then I guess that explains why you didn’t take this essential feature from all other sprinkler controllers into account in the original design of OpenSprinkler. I thought I explained it in a clear and obvious way, but perhaps you don’t have the background engineering experience with working on critical systems that have such requirements. One more try: Pretend you’re on the moon and you’re using OpenSprinkler to control your CO₂ scrubbers and they have a similar cycling requirement as my watering system — the program is maxed out to 24×7 operation and there’s no room for any unscheduled downtime. If the controller reboots just after midnight and then remains idle without restarting the program that should be active for the next 24 hours, well, then you’ll die. Such a controller _must_ restart the running program at the point it should be at for the current time of day after any reboot for any reason. Obviously my trees and garden won’t die from one day’s less watering, but it will start to impact their health, and if the failure went on for a week during the hot season, well some plants would die and trees would be highly stressed.
You ask why I think you haven’t added this feature already. Well, beyond it now being clear that you don’t understand how critical control systems must behave in order to fail safely, I think you’re also over-thinking the problem. You kept referring to how dynamic events impact the schedule generated from a program, but you don’t seem to realize that such things are entirely unimportant in the bigger picture of just getting the machine back on track and working again automatically.
Why not choose another product? Well, I do like to support open source projects — if you look me up on GitHub you’ll find I already maintain a number of them. I have already looked at most other products that have the capacity we need, but they all lack the one feature you were actually easily able to add to OpenSprinkler — the ability to control the order of zone operation without having to re-wire things. OpenSprinkler has the capacity to handle all the zones I have and more with just the one controller (I’m replacing two independent controllers), and the price is quite reasonable. All these factors, plus the fact I couldn’t imagine anyone designing a system that wouldn’t restart a program after a reboot, made OpenSprinkler a very attractive looking solution. I didn’t realize this problem until I was testing my primary zone control program and I couldn’t even find a way to manually start it after a reboot.
So as a supporter of open source projects I’d rather help you understand how this issue impacts users of your product and help you figure out how to change it so that it can support this obvious feature. Being able to say that OpenSprinkler can fully support all of the features of traditional controllers and then some would help you, me, and everyone else better promote it as an easy no-brainer upgrade for existing systems. I alone could easily sell five or more people on buying it, and without any need for convincing if it had this feature. However without it being able to do this most simple but necessary thing as all existing controllers can do I wouldn’t expect very many people to want to use it when they have an existing controller that already runs reliably and unattended even with flaky power.
Let me ask you: Why are you so reluctant to add a feature that you’ve clearly already figured out the algorithms for? What exactly in the controller code makes you think this is going to be so difficult to do? Are you still hung up on trying to support previous dynamic events that might have arrived between program start and the reboot, and if so, why? Where would you start in looking at the code to add such a feature in the simplest way possible without taking into account dynamic events?
robohackParticipantSo, Ray, I don’t think you’ve understood the magnitude of the problem for me (and perhaps others) when the controller doesn’t fail-safely.
I live in a semiarid desert and most of my ground is essentially gravel with a thin layer of topsoil, but it’s orchard and vineyard country and we have pretty good water sources (Lake Okanagan, for one, plus reservoirs up in the mountains) so we try to grow things on it anyway.
As I said I have 32 zones currently and during average weather conditions I have a total run time of 21 hours. That means during dry months I have room for about 112% watering budget expansion — which would take me to a teeny bit over 24 hours. Last time it was over 40°C every day for a few days I did run 24×7.
I can’t run any zones simultaneously because most of them max out the capacity of my system for water delivery.
I can’t run zones for just a few minutes — no water would get into the ground to the roots, it would all evaporate first. During the day I need to run some zones for upwards of 10 minutes before the area cools off enough for any water to be soaking in. I already have a few zones that only run for about 20 minutes. It’s obviously “wasteful” in absolute terms, but then again if we in this region were sensible there wouldn’t be an agriculture here except in the very bottom flood-plain of the valley.
I might be able to split all the times in half and run the program repeating every 12 hours, but that would be the limit. However 12 hours lost watering is still not acceptable — it doesn’t leave room for maintenance and field/orchard/lawn work. It would also make it nearly impossible to avoid running some zones in high-traffic areas when there is activity in those areas (at least not without a lot more, and far more complex, programming features). Getting that kind of timing right is something you want the computer to do and not have to work it out manually — but OpenSprinkler doesn’t have enough programming features and controls to do it.
Asking someone with a complex system to run every zone multiple times a day just to work around the lack of fail-safe reboots is nearly as absurd as making a UPS a requirement for reliable fail-safe operation! Both are non-starters — entirely impractical.
When I say “fail safe”, I mean it in the sense of something like a life support system, which in fact it is, for plants. You don’t allow a life support system to sit inactive for up to 24 hours just because it rebooted at the most inopportune time. The system has to come back fully operational and running on schedule, no matter how long or short the downtime or when it occurs. The downtime is unavoidable, but what the system does on reboot is entirely within its ability to manage.
When I compared OpenSprinkler to the old controllers I was trying to convey the facts that: (a) this form of fail-safe operation is a very common long-standing feature that I would expect every modern controller to have; and (b) OpenSprinkler has a great deal more computing power and resources than those old junkers (I think – there is a microcontroller CPU in the Irritrol) so there’s no reason why it can’t _easily_ support this feature.
Furthermore, as I said, past dynamic events are irrelevant if they can’t trivially be accommodated. “fail safely” means simply do what the programming says to do as if the controller had been rebooted while nothing was running, say at midnight, and no dynamic events were received since then. If you’ve got room to store weather events or pause requests, etc. (with their arrival timestamps) such that they won’t be erased by a reboot, then, sure, take them into account when running the program forward to the current time of day, but otherwise they don’t matter as much as simply getting the program on track again. If I tell the system to pause while I do some maintenance and there’s a glitch causing a reboot before I finish that gets me all wet, well it’s not the end of the world. If I really don’t want to get wet at the wrong time, well I know where the power switch is, not to mention the master valve! And if it would come back alive when I rebooted it I wouldn’t worry about turning it off for maintenance work!
As for OpenSprinkler being open-source, well yes I know it is, but as you may recall what I said in the past: while I have considerable programming experience, including in low-level systems code, the OS firmware is written in a language I have little ability in, and absolutely zero interest in expending any effort in learning. So unless the feature is so trivial to implement that even an old C programmer could work it out in short order, I’m unlikely to be able to help with it. Also, if it’s that simple, well…. If I had been the designer of the system I wouldn’t think it would be a very difficult feature to implement, but there are aspects to the current design that I don’t fathom the reason for even from a user perspective, so my guess isn’t likely worth very much.
In any case, as I said, I’m still stunned and dismayed that OpenSprinkler didn’t have this long known feature from day one. The lack of this feature was certainly not obvious to me before I made the purchase and committed to changing my system over and I really didn’t expect it to be missing.
robohackParticipantSorry, but using a UPS is an extremely unreasonable suggestion!
The controller is mounted in a weatherproof box on the outside of a building using a pre-installed outdoor socket — there’s certainly no room for a UPS in that box and no cheap or easy way to connect one at the breaker box either.
Even if the power only went out once a week it would be critical here — I have a minimum of 21 hours of total program run time every day over 32 zones and losing any significant part of a day’s watering in some of those zones can have a significant adverse affect on the plants being cared for. A UPS doesn’t even solve the whole problem — I would also use a manual reboot to restart the system after doing any maintenance, or, e.g. mowing.
Both of the old controllers I’m replacing, one an Irritol Systems 24-zone unit for example, will restart their programs at the current time of day, even if they’ve been off and then turned on. Normally they only restart at the next zone’s start time, of course, not at the zone that should be running at the current time, but they’re much simpler microcontrollers with their firmware frozen in ROM and unmodifiable.
However I still need a reliable controller that “fails safely” in a basic engineering sense — which in this case means it only stops watering for the time the power is out (and maybe up until the time the next programmed zone should start). Even a manual reboot shouldn’t leave me with having to manually configure watering for the rest of the day — this is a computer that’s already supposed to be able to figure out and manage such things!
To me this is a critical feature that I’m stunned wasn’t included from the get-go, especially since existing commercial timers have been doing it for decades. Anyone replacing almost any existing controller will most certainly expect this!
Note that it’s not obvious to a user that the “Preview Programs” the web page isn’t produced on the controller — but either way that’s not important — I just thought it would be an obvious solution since it already seemed to exist already in software and it shows exactly where the execution should continue from. In any case I didn’t mean to keep using the preview — I just meant to use it to find where the current state should be, then to continue on as normal from there.
How about instead just look through the list of program start times, find the ones that should have started “today” (i.e. since the previous midnight), run through all the zones in those program(s) until their elapsed time matches the current time, turn on the zone that should currently be active and run any remaining time for it (or like the old controllers, wait for the next zone that should start, at the time it should start), then continue on with the active program(s) as if the controller had been actually been rebooted just before midnight?
Just do this on reboot, or perhaps if there’s a manual request to do it through the web API, though the primary concern is after any reboot.
None of the other issues are important — only future events need be dealt with, and they should be handled in exactly the same way as if the controller had always been running and a program had started at the actual start time but was otherwise unaffected by any dynamic or user-driven events. Any dynamic events that might have actually occurred between midnight and the current time can be ignored as if they never happened — all it needs to do is “fail safely” on reboot so that it continues on running any stored program(s) from the current time of day.
robohackParticipantHas there been any change in the status of the first question — i.e. can OpenSprinkler with v2.2.1 firmware now restart a program midway through after a power failure or reboot?
I’m guessing not — I’ve rebooted my controller and even though the current time has passed through points where a next zone should have started, nothing is happening, no zones are active.
Watering is too critical here and I can’t have it fail midway through a program, or even midway through a zone, just because the power glitched! My program is far too complex with far too many zones, and the need to regularly edit zone times, to be broken up into small intervals with fixed start times.
I _really_ need this feature too!
It seems absurd that there’s no “resume the running program” feature in a controller this sophisticated. It knows what time it is and it knows if a program should be running at the current time, and it knows what point in the current program the current time represents, so should be able to immediately restart the zone that should be running now and continue the program on from there.
The program simulation feature shows all of this information quite clearly, including the red bar for the current time.
On reboot the controller should just run the simulation of the day’s program(s) through to the current time, and if anything should be active it should simply jump in there and continue on as if nothing had happened.
robohackParticipantThanks for your reply Ray!
While I think it would make the most sense to have arbitrary zone ordering per program, for my own needs I will be perfectly happy with ordering by zone name.
“001 front lawn”
“002 back lawn”
“003 side lawn”
etc.That’ll work just fine.
In my own use I don’t see much need to re-order in different programs — mostly I just omit certain zones in different programs, e.g. for mowing days.
While I appreciate that control via the network API would also allow arbitrary programs and ordering, my current thoughts are that I don’t want anything in the chain of control that doesn’t absolutely have to be there for regular operation. Maybe if there were native HomeKit integration I might risk it, but even there I’m happier having stand-alone embedded systems driving critical infrastructure than having to depend on my AppleTV to run the show.
The reason traditional sprinkler controllers don’t allow arbitrary ordering of zone is because their designers are hardware-first engineers concerned about getting costs as low as possible. I once turned down work on an embedded system that cost several thousand dollars per unit because they wouldn’t spend more than $0.25 on the microcontroller for it. Hardware designers also don’t think like software people at all — software is an annoying necessity to them for more modern low-cost hardware design, and besides they’ve traditionally relied on physical switches and knobs for input controls so programming different ordering would likely require more switches and knobs (or be even more convoluted than their existing horrendously ugly control interfaces), and so would require even more costly hardware. Also since traditional sprinkler controllers are not typically remotely controlled you have to go to the controller anyway to change a program, and so re-wiring is only a screwdriver away. I had some hopes for the high-end Hunter HCC systems with WiFi control, but I can’t get a firm answer from them, and if I were going to spend several thousands of dollars upgrading my system I need to know it’ll do exactly what I want and more well ahead of time, without needing internet access to their cloud! All the other so-called “smart” irrigation controllers are abominations that seem like they were designed 20 years ago.
As for the code, sorry I just don’t touch C++, no matter how few C++ features it purports to contain. I have over 40 years of C programming experience and I’m firmly in the anti-C++ camp.
For the record I’m wanting to replace an old Irritrol 24-zone and a Hydro-Rain 9-zone, with only one zone each unused, integrating them all into one controller. A little SoC board, like an RPi or BeagleBoard, with an add-on relay controller board, while highly flexible is far too ad-hoc for me, and I don’t want to do the software from scratch either, so something like OpenSprinkler is highly appealing, provided I can bend it to do things the way I want.
robohackParticipantSo, first off, I’m surprised this feature is still not present given this thread started so many years ago!
I’ve been looking for exactly this feature in a sprinkler controller, and it would seem to me to be an absolutely essential feature for anyone with more than two or three zones. I sure as heck don’t want to have to go to the machine and physically re-wire it just to change the order the zones are run. This is a problem for software, not screwdrivers!
All that said I did fairly quickly find a very simple, but highly limiting, and somewhat tedious, way to make this sort of work with the current firmware.
One simply designates one program for each zone. I.e. create as many programs as you have zones, and in each program give only one of the stations a run time (each with the same start time, e.g. 00:00, and each set to run at an interval of 1 day).
Now since it is relatively easy to arrange the programs in any order, voila, zones can now be run in non-sequential arbitrary order. (Why can’t zones be re-ordered the same way programs are?)
It’s still highly limiting to do it this way — one really needs support for ordering zones within any given program (and of course it must be all done in the firmware so that it’s reliable). If the code were plain C I might even be able to make it work myself, but I’m very reluctant to try to dive into C++ code.
Attached is an image of a program preview in the demo system of an example setup where zones are run in an arbitrary order.
Attachments:
-
AuthorPosts