OpenSprinkler Forums OpenSprinkler Unified Firmware Penmen-Monteith ETo method python script for possible use as weather script

  • This topic has 66 replies, 2 voices, and was last updated 8 months ago by Ray.
Viewing 25 posts - 1 through 25 (of 67 total)
  • Author
    Posts
  • #37137

    ShawnHarte
    Participant

    I have included a quickly put together file using either the P-M ETo method or if enough data is not available the Hargreaves ETo method.

    The theory is that a user would create a program using the number of seconds required to distribute 1mm of water in the coverage area.  This would be saved as a program called 1mm.  After doing this the user then inputs their WU api key just as they would for the Zimmerman method.  The script included would then read the current need or excess water from the logs stored on the uSD card and create a dynamic program with runtimes utilizing the best averages for decorative grasses/drip systems, or a standard lawn grass used in the majority of the world.

    The script accounts for wind, freezing conditions, and current/recent rainfall when considering start and run times.  It will avoid watering during midday, unless early morning winds prevent earlier start times.  The starts are serialized so no odd overlaps should occur.  Mornings are preferred to evenings to allow for the best use of water and absorption without causing mold and fungus growth by leaving grass wet overnight.  The script is commented quite heavily, so that anyone my edit or use it to their liking.  Please be mindful that other authors work was used or modified when the code seemed generalized enough that I shouldn’t be stepping on toes.  Please do not pester the original author if something doesn’t work for you, as they will probably have enough on their own plate with their own original works.

    If someone smarter than myself can find a way to hack this into the current Firmware for OS I would be extremely grateful as I’m sure quite a few others would be.  This script should comply with most watering restrictions in the US, however, I must say use at your own risk, I simply don’t have the time to puruse the near 600 pages of legal craziness for California alone.

    Everything is done based off your latitude and longitude, however, the script can find the info when provided with a city/state or country, a US Zip Code, or a PWS ID.

     

    This was tested with python 2.7.8

    Attachments:
    #37157

    Samer
    Keymaster

    This is very well written and if you don’t mind I would like to help incorporate it into the main code. The documentation alone is worth it (and apologize for the minimal documentation in our code).

    In regard to the logging of ET data, I am sure we can incorporate that into the firmware and offload the ET calculation to the weather service. The runtime calculation (using log data) would be calculated on the firmware side.

    Also, feel free to submit pull requests to the repo as we are happy to review and merge changes like these.

    Thanks!

    #37177

    ShawnHarte
    Participant

    I’m decent with python, however, a lot of the methods for writing logs and incorporating changes into the core files for opensprinkler is a bit beyond my scope and time available.  You are more than free to use any or all of the code with or without credit.  I have been using this in a very rudimentary manner by adjusting my runtimes based on the outputs.  I run the script about every hour and have an autoit script log into my OS and edit the program for me.  Total hackjob but it does a fantastic job of watering only when needed, and avoids watering in less than desirable conditions.  It would be awesome to see it incorporated in a truly functional manner.

    I assume I would submit the pull request to the weather script section of the repo?  My biggest hangup came when the run times were reduced to save space, until then it worked well, now it balances between slight over and under watering.  I was using the 1mm times multiplied by the ET up until that change was implemented, as the ET output is mm of water required, and that made it a perfect candidate for that method.  I suppose the 1mm time could be multiplied by the desired single run max and then converted to a percentage of the max, and use the current format of the Zimmerman method.

    I sincerely thank everyone that has put effort into making this one of the most end user friendly sprinkler systems I have come across.  I also have a basic winterization system i have been working on using a few electrically activated valves, to shut off and drain the water lines whenever a freeze is detected, which could easily incorporate dates for areas like where I live in Colorado which restrict water before and after certain dates.  Right now it uses a single zone to turn off the main water valve, and open the drain lines to stop pipes from bursting, but I’m sure I could look into the wireless device add-on capabilities that appear to be incorporated into the OS system.

    #38105

    hplato
    Participant

    Thanks Shawn,

    I tried this tonight, and two things;

    – Without data for yesterday. the mmFromLogs subroutine ‘oops’ when trying to open the data. I copied your data to yesterday’s date, and the script ran. However, what should an actual starting point be for ET/<date> and logs/<date>?
    – From what I gather, I need to figure out the number of seconds for each station to distribute 1mm of water, and then edit the 1mm file. OK got it.
    Using your script, I’m a little confused by the first set of values in the weatherprogram/run file. When I run the script, I get:

    Resolved your lat:xx.xxxx, lon:-xxx.xxx, they will be stored
    0.633333333333 mm adjustment for weighted 3 day rain forcast
    1.0 mm adjusment for rain occuring today
    P-M ETo
    6.4366 mm on grass
    7.4292 mm on shrubs
    sunrise & sunset in minutes from midnight local time
    327 1301
    Current filename we are using: 16587
    Looking for log file: 16586
    Weather Station ID: xxxxxxxx
    [75, 150, 75, 60, 75, 75, 60, 0]
    [[-1, -1, -1, -1], [0, 0, 0, 349, 0, 0, 349, 0]]

    From my understanding, this it to turn on stations 4 and 8 for 349 seconds, however the -1 means that the station is actually disabled?

    #38125

    ShawnHarte
    Participant

    As far as logs are concerned these would be the run logs stored on the OS, the data that requires editing is just dummy data for a starting point, and so that the script works. I really need to have the script default to 0 run time for a program if no log exists, as I do believe the OS only creates a log for a program if it runs.

    ET should be started at zero. It will be pretty well balanced after about a week of average weather for your area. If it rains a ton in the first few days it can look pretty scary in the negative, but usually remains pretty accurate for watering needs. When I started using it we got 14inches of rain the second day running, after about a week everything was leveled out, and the grass was a touch on the dry side but not dead.

    the 1mm file should contain the number of seconds for each station to distribute 1mm of water over the coverage area. You may have to calculate the values if you don’t have a water meter that can display how much water is going to the zones.

    There is a minimum and maximum water amount per start time. I used 5mm and 15mm respectively. If there is less than 5mm required the program will not run, in the next available start time. For example if your ET came out to 25mm you would see the first 2 start times enabled, the first would water for 15mm then the second would start and finish off the remaining 10mm.

    If any precipitation is detected in the most recent call to the script all start times will be disabled. There is a flaw in that part of the script as it may disable watering for the remainder of the day, however, I figured if it was raining we could probably wait till the next day to catch up. Most places disallow watering during or immediately after measurable rainfall anyway.

    #38132

    hplato
    Participant

    Hi Shawn,

    Thank you so much for the response. I just got the OS a month ago, and new to overall irrigation.

    That makes sense, I’ll zero out the ET parameters once I start this. I think I understand your comments, but just to confirm;

    – so the 1mm file, I don’t have a water meter so I thought I’d get a few buckets around the area, run the sprinkler for 10 minutes, and then measure the average water and do the math?
    – my stations right now just start once in the schedule. why would there be multiple start times?
    – that makes perfect sense about disabling watering during the rain. That’s what I’m trying to figure out on my home automation interface, since we are getting a lot of rain, but want one station to run all non-rainy evenings as we have some new plants.

    Do you run this script once a day? That would definitely save on WU calls. I see right now that the OS calls ~40 times a day, so there is headroom. I was thinking about either running this every 10 minutes to also populate weather details in my home automation system, or run it once a day at sunset, since all my watering is at night.

    #38146

    ShawnHarte
    Participant

    I used a pickle jar lid and a 5 minute run time to get my measurements. After doing the calculation you’ll find you need a small adjustment one way or the other but it comes out pretty close.
    A run/soak cycle is a bit more on the water conservation side of things. I don’t ever allow for more than about 1/2inch of water per cycle. For my yard much more than that I get runoff and I’m just paying to see if gutters grow when you water them. That’s where that minmax file becomes important. The max should be single cycle saturation point for your yard. The script will also lean toward watering in the morning to prevent fungus growth from grass being wet at night.

    I run the script every 15 min, nothing changes aside from the current weather conditions. It could reasonably be run just prior to sunrise and just prior to sunset, with no discernible difference in daily watering. If you use your WU data for something else and make a few calls I would suggest running the script 4 times a day just to make sure you get at least 1 reliable reading per day.

    I’ve made a few adjustments to the script based on your response, I’ll post it in another post after I check to make sure I haven’t created any problems.

    #38184

    hplato
    Participant

    Awesome. Thanks Shawn! Really appreciate the guidance and advice.

    I have a .2 acre property and 11 stations, with 47 sprinkler heads, so it will take a while to get all the measurements. However, I really like the idea of the science-based watering.

    What I’m thinking now is using that interface I built to run the calculates and push the data to the OS. I’ll run it every 15 minutes, and also use the data from the calls to populate the weather details in my system.

    I’m not strong in python, so I’m wondering if I can ask for two enhancements. If not no problem, I can probably hack it together, just be nice that I don’t have too much of a custom piece:

    – take a command line argument as the path to the logs and ET data.

    – print out current conditions, and save them to a file. Maybe tab delimited to make parsing easier:
    – wuData[‘current_observation’][‘observation_epoch’]
    – wuData[‘current_observation’][‘weather’]
    – wuData[‘current_observation’][‘temp_f’]
    – wuData[‘current_observation’][‘temp_c’]
    – wuData[‘current_observation’][‘relative_humidity’]
    – wuData[‘current_observation’][‘wind_string’]
    – wuData[‘current_observation’][‘wind_dir”‘]
    – wuData[‘current_observation’][‘wind_degrees’]
    – wuData[‘current_observation’][‘wind_mph’]
    – wuData[‘current_observation’][‘wind_kph’]
    – wuData[‘current_observation’][‘solar radiation’]
    – wuData[‘current_observation’][‘precip_1hr_in’]
    – wuData[‘current_observation’][‘precip_1hr_metric’]
    – wuData[‘current_observation’][‘precip_today_string’]
    – wuData[‘current_observation’][‘precip_today_in’]
    – wuData[‘current_observation’][‘precip_today_metric’]
    – wuData[‘current_observation’][‘icon’]
    – wuData[‘current_observation’][‘icon_url’]
    – Is_Raining. I think this is calculated somewhere else, just a 1 or 0 if actually raining…

    #38208

    ShawnHarte
    Participant

    I have no problem doing that if you don’t mind waiting another day, I have a friends wedding to attend, then I’ll have plenty of time.

    #38217

    hplato
    Participant

    Don’t mind at all, really appreciate the help and suggestions!

    #38494

    ShawnHarte
    Participant

    So my daughter decided to assist me with watering my main computer, needless to say the computer hasn’t grown any…I have a new computer now, and the old hard drive is still working so I’ll get back to working on the script in the morning. My apologies for the delay.

    #38576

    ShawnHarte
    Participant

    Finally got to this, sorry for the incredibly long delay, I have attached the customized version, you will need to add a folder called wuData.
    The script can be run from the command line as:: python weatherCustom.py logsdir ETdir wuDataDir
    Hope this gets most of what you wanted done. The icon URL in wuData does some funny things in the output so I omitted it from the file output, you can add it back in if you’d like just be aware it may break something.

    Attachments:
    #38591

    hplato
    Participant

    No problem on the delay — I haven’t measured my rain rate yet since it’s been thunder storming for the past two weeks. I’ll give this a try and let you know if there are any issues. Again, I really appreciate the help!

    #39348

    scottsh
    Participant

    Passing this along in case it helps others – a good video on how to figure out the precipitation rate of your sprinklers.

    http://www.youtube.com/watch?v=W_wn-hwLNtg

    #39349

    scottsh
    Participant

    Samer – just curious, but how goes the inclusion of this method into the main code? Is the PM-ETO method actually in the unified FW now?

    Thanks!
    Scott

    #39374

    scottsh
    Participant

    To do this right, I think we need to have zone-specific measurements for precip rate as well as amount of water required and run-off rates. I have 18 zones and many heads too. I thought about this a little in my plug-in to Dan’s program (see image attached.) I didn’t implement the ETO algorithm at the time because I was worried about getting all the required weather data. I’ve implemented something else but it isn’t perfect (it over-watered in the winter, for example.)

    Also, it is hard to get good localized data for is solar radiation; most weather stations don’t collect that data. That doesn’t mean this isn’t worth tackling, however. It might be that getting solar radiation data from a nearby source is good enough.

    Attachments:
    #39379

    Samer
    Keymaster

    Scott no the ET calculations are not yet incorporated into the firmware/weather service as we are still working this out. I like the work you have done with OSPi and will definitely look over that to incorporate some of the things you have done. I admit time has been an issue and not enough work has gone into ET based adjustment at this point to give a meaningful ETA on when it will be out. This is key to getting WaterSense certification so we do have a motivation to get this done and are trying to as soon as possible.

    Thank you!

    #39380

    scottsh
    Participant

    A few other comments if you decide to incorporate something like this. I don’t mean to belabor the obvious, just passing along some knowledge I gained.

    – Perhaps obvious, but rotary or spray heads outside the US are measured in mm/hr and not inches.
    – Drip systems are measured in either gallons per hour (gal/hr) or liters per hour (l/hr) and thus you might want to support those units for entering per-zone watering rates and amount of water required.

    There are a few python libraries out there for calculating PM ETO and alternatives. A google search turned up several, many of which are based on FAO’s underlying library like the one here. Here is another: http://python.hydrology-amsterdam.nl/modules/evaplib.py

    The key weather data inputs appear to be:

    Minimum and Maximum temperature (C)
    Mean temperature (C)
    Dewpoint temperature (C) or relative humidity (%)
    Mean windspeed (m/s)
    Atmospheric pressure (hPa)
    Total solar radiation (in Jules)

    #39381

    hplato
    Participant

    Hi,

    So I’ve gathered all the data from my stations to calculation seconds/mm. I’m at a loss to figure out how to calculate runoff and how to determine the actual water needed. Is there any reference for those data points?

    #39382

    ShawnHarte
    Participant

    The actual water needed is delivered by the script provided in the first post, the ET value. As far as runoff (saturation point) values there is quite a few different calculations and after researching for quite a while I found that 15mm per hour is a good starting point for most soils. If you have sandy soil this will go up, and hard clay soils or steep grades it will go down.

    Historical data, watering logs, and a running ET balance are crucial to maintaining a watering plan.

    scottsh-Drip systems have been giving me headaches for months, all my personal watering equipment can be accessed above ground and I can run my dripline into a cup and get the rate at which it “precipitates” and doing a bit of rough math came up with a “mm/sec” that works for me. My ultimate desire is to have this calculated scientifically and properly.
    As far as solar radiation is concerned most water sense approved devices use a calculation to adjust readings taken from a semi-local weather station. I simply used the external radiation and cloud cover percentages (calculated based on charge rate from the solar panel, not a true solar radiation measurement) from my PWS. After purchasing a solar radiation sensor and running the numbers side by side for just under 2 months, I was within 10% of measured. Seeing as the solar radiation plays the smallest part in the calculation for ET I accepted the error rate as fair.

    I am currently working out how to write all of the nvm data to the SD card on the avr based OS, so that I could have programs that do not require encoding of the times as this skews watering times quite heavily. It appears as though StdioStream in the SDFat library gives me the few basic functions required to mimic the OSPi and BBB versions using the “nvm.dat” file. I have to write/copy the nvm read and write functions for the avrOS make a few changes to the weather.cpp file and I should be able to get everything working.

    Takes a while because I need my controller to work while I experiment on it…I should probably shell out the extra cash for a bench/test unit but it’ll probably be winter by the time I don’t have something else for my money to go to, cars and computers are expensive hobbies.

    scottsh-where in the world do they use Jules for weather related solar radiation calculations? I have always used Watts/M^2, going between the US and Germany quite a bit I thought I had the metric and imperial systems down, but it looks as though I’ve missed a measurement.

    #39386

    scottsh
    Participant

    hplato – I personally calculated runoff time by watering and watching and it is on a per-zone basis. My mulch-filled raised flower beds have a different runoff limit (.1 in) than my grass (.2) and my drip-irrigated lemon trees in pots are much lower (.05in.) AND, my county extension service (local government agricultural department) has average data for this they make available.

    I was able to do some web searching for the plant/grass types I have to get the average water required in my part of the world.

    Shawn – When purchasing drip systems, I believe the drip heads have a l/hr or gal/hr rating. The ones I have all have ratings from .5gal/hr to 2gal/hr (GPH) drip rates. I thought the same was true in the rest of the world for l/hr (LPH) but I suppose if you were using something hand-made or drip hoses that might be less visible.

    I was thinking of how to build a solution for everybody (all users of OpenSprinkler) who don’t have PWS or solar panels. That’s the good side of using Wunderground because you can likely find a PWS near you. Is there a way to estimate solar radiation based on data available from most PWS?

    I think you are right about Joules not being the right unit for solar radiation. You can convert from Joules to watt/m2 but watt/m2 is probably the right unit to start with. I got J from one of the python libraries I was looking at but it was petty academic and maybe not practical.

    And I know exactly what you mean about building a solution on a unit you need to work – I have the same. It can be challenging. If it messes up I either end up with dead plants or a lake.

    Luckily I use an OSPi which has far more computing capability than the Arduino version so I wasn’t worried about that but I appreciate why the Unified FW uses the cloud-based approach to offloading hard work from the Arduino.

    #39391

    ShawnHarte
    Participant

    Ya I’m sure my drip emmiters had the L/hr listed on them at one point, but at 15 years old, it has escaped me, and measuring the up and coming subsurface irrigation systems will prove difficult. But someone has already figured out the code for water flow meters, so I’ll give that a look for inclusion.
    I wasn’t meaning to sound rude with the joules thing, it’s just a small measurement and would make for some really large number calculations. But being base 10, kilo-joules could be used like kilocalories on a cereal box.

    If you look at the script in the first post it has a library in the zip for PM-ETo, solar radiation has 3 calculation methods available each with varying degrees of accuracy. The main script uses them in order from measured to least accurate, depending on data available.

    #39422

    hplato
    Participant

    Hi Shawn,

    So I’ve updated the 1mm file to specify the number of seconds that each station needs to run to distribute 1mm of water. I’m not sure what ‘crop’ does:
    {“mmTime”:[54,20,33,17,37,18,27,20,30,37,16],”crop”:[0,0,0,0,0,0,0,0,0,0,0]}

    Running python weatherCustom.py I get the following output:
    Content-Type: text/html

    Resolved your lat:XX.XXXX, lon:XX.XXXX, they will be stored
    0.0556 mm precipitation forecast for next 3 days
    2.0 mm precipitation fallen and forecast for today
    P-M ETo
    5.8051 mm lost by grass
    6.6252 mm lost by shrubs
    sunrise & sunset in minutes from midnight local time
    348 1297
    Weather Station ID: XXXXXXXX
    Current logged ET [5.8676, 6.6252]
    [54, 20, 33, 17, 37, 18, 27, 20, 30, 37, 16]
    [[-1, -1, -1, -1], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]

    So since it’s rained today, all stations should be 0, so I get that. If the numbers change, then I should push the new values to the current program. However, I’m not sure what the [-1,-1,-1,-1] is telling me?

    #39423

    ShawnHarte
    Participant

    The -1 is the start times in minutes from midnight if a start time is needed. -1 means disabled. This matches the JSON output when viewing the programs using http://os-ip/jp?pw=xxx

    Crop is a 0 or 1 for grass or shrubs respecticvely. If a zone is primarily flowers 1 should be used for that zone. Primarily grass or low growing plants (less than about 4 inches high) should use 0. This will denote which ETo value to use.

    #39481

    hplato
    Participant

    Got it. So I’ve updated my interface to Misterhouse to allow for running an external script and then uploading the data to a specific program. So far it’s working. Thanks again for your help with this.

    Two quick things to confirm my understanding.
    – So for this to work, the script needs to run at least once a day to upload the new start and runtimes to a program. I think you mentioned running it every 15 minutes, however I don’t know how that helps, since really it just needs the data before midnight to calculate the morning schedule?
    – Since this dynamically adjusts a program, the program needs to be set to run every day, correct?

    Also, since I call this as an external program I needed to make a few slight changes so that the site libs are found, as well as a path to the output. Here’s my updated file as well as a bash script I use to call the program.

    Attachments:
Viewing 25 posts - 1 through 25 (of 67 total)
  • You must be logged in to reply to this topic.

OpenSprinkler Forums OpenSprinkler Unified Firmware Penmen-Monteith ETo method python script for possible use as weather script