Rain delay from Yahoo Weather

[andrew]

I’ve created a very online weather-based rain delay script to add to the mix. It’s a very simple bash script that accesses Yahoo’s Weather API service (which is fed by Weather.com). It interfaces with Ray’s interval program to impose a rain delay based on the current conditions.

Code and instructions are available here:
https://github.com/andrewshilliday/OpenSprinkler-WebRainDelay

Questions and comments are welcome.

[Samer]

This is fantastic, thank you!

Question, is there anything that prevents the script from setting a fresh 24/48/etc hour delay each run (5 minutes)? If not, do you think this is a problem?

Going to play around with this and see if I can incorporate this into my web application, if you don’t mind. I am hoping there is easy way to programmatically resolve the location on the OpenSprinkler into a WOEID (saving a configuration step).

[andrew]

salbahra: There isn’t anything preventing that and in fact, the overwriting of existing delays was intentional. The idea was that as long as it’s still raining, we might as well push the rain delay out to whatever it’s set for that condition. So if I have it to set a rain delay of 48hrs on rain, then because the script is constantly overwriting the delay setting, the timer won’t start until it stops raining.

Now that I write it down, however, I am realizing that there could be a problem with that approach: If you have condition A set to delay for 48hrs and condition B set to delay for 24, and A hits first but then the weather changes to B, the delay will be changed to 24 hours even though you might still want it at 48 because that original condition did occur.

Perhaps it should be set to only overwrite the rain delay if the new delay is higher than the existing one. What do you think?

As for integrating into your application. By all means. I will also look into writing a script to resolve the WOEID, if it’s not too complicated.

[andrew]

salbahra: I uploaded a bash script to obtain the woeid from Yahoo. it can be found here:

https://github.com/andrewshilliday/OpenSprinkler-WebRainDelay/blob/master/get_woeid.sh

Pass in an argument with the string that you want to search for. For example:

> ./get_woeid.sh "Washington, DC"

2514815	Washington, DC



> ./get_woeid.sh "12345"

12762240	Schenectady, NY 12345



> ./get_woeid.sh "Disneyworld"

23686031	Walt Disney World Resort



> ./get_woeid.sh "Beijing"

2151330	Beijing

It will return the woeid and the name of the matched location. They’re separated by a tab, so you can pipe it to cut -f 1 to get just the woeid.

I wouldn’t recommend calling this every time you want to query the weather. Rather, I would run it once to obtain the weoid and somehow save that along with the user-specified location. Yahoo limits the number of times this part of their API can be accessed.

[Samer]

Nice! I will start migrating the code into PHP and let you know when I have a working copy with my web application. The goal in the end is to have all the codes modifiable from within the application, use the OS location information, and offer settings to turn on/off the weather based rain delay. I think this will make a wonderful addition to the app, thanks!

I am thinking it might be okay to forgo setting a rain delay if one is already set? Or maybe only checking the weather once a day, maybe early in the morning or late at night? What do you think?

Edit: Keep in mind I know nothing about home irrigation and just love programming. So, any advice/feedback on best practices for something like this are always appreciated!

[andrew]

Sounds great. Let me know if you need any help with the integration.

As for the frequency, I have it running more frequently because it only checks *current* conditions so if we’re only getting the weather once per day, we could easily miss the thunderstom that rolls in and soaks the ground. We could look into looking at future forecasts, but I’m not sure what kind of rules we’d want to apply over them. I had been looking for a while for an online data source that could be queried to obtain rainfall accumulation data (e.g., inches of precipitation in the last x days) but couldn’t find one.

[Samer]

@andrew I got the weather code incorporated into my web application. The only thing left is adding a configuration option to enable/disable the feature, add a way to edit the CSV file which contains the code to delays, and decide how often to run the function. My station watcher script (for logging) runs every minute and I intend to piggy back off of it. However, I think I will have it run the specific weather function less frequently for multiple reasons. The main being 1 minute is unnecessary I doubt the weather changes that often and I also don’t want to get IP’s banned from Yahoo’s API.

I added a regex to parse the data and have pulled all the current weather conditions not just the weather code. I want to incorporate the settings into a “current weather” page on my mobile app.

Here are my functions I am using so far:

#Weather functions



#Resolve location to WOEID

function get_woeid() {

$options = get_options();

$data = file_get_contents("http://query.yahooapis.com/v1/public/yql?q=select%20woeid%20from%20geo.placefinder%20where%20text=%22".$options["loc"]."%22");

preg_match("/(d+)/", $data, $woeid);

return $woeid[1];

}



#Get the current weather code and temp

function get_weather_data() {

$woeid = get_woeid();

$data = file_get_contents("http://weather.yahooapis.com/forecastrss?w=".$woeid);

preg_match("/
$weather = array("text"=>$newdata[1],"code"=>$newdata[2],"temp"=>$newdata[3],"date"=>$newdata[4]);

return $weather;

}



#Lookup code and get the set delay

function code_to_delay($code) {

$codes = explode("n",file_get_contents("yahoo_weather.csv"));

foreach ($codes as $line) {

$data = explode(",", $line);

if (!isset($data[2])) continue;

if ($code == $data[0]) return $data[2];

}

return false;

}

The get_options function is something in my code to pull the options off the OpenSprinkler. I do this to prevent a WOEID configuration from the user. I am using options and not settings because the OSPi does not properly report the location in the settings. Here is an example of how I use these functions:

#Check the current weather for the devices location, and set the appropriate delay, if needed

function check_weather() {

$weather = get_weather_data();

$delay = code_to_delay($weather["code"]);

if ($delay === false) return;

send_to_os("/cv?pw=&rd=".$delay);

}

Just wanted to share what I have so far, thanks again!

Edit: I just saw your comment about WOEID update frequency! I will update my code to take that into consideration, thanks!

Edit 2: I just checked and the API limits the WOEID lookup to 2k per day. I am checking once every hour in the current implementation so I will not worry too much however it is still an HTTP request which increases latency. Since it is not something the user has to deal with I am keeping it a low priority fix, for now. In the future I might upgrade my web application to use a SQL database rather then files. At that point, I think I will switch the code to set the WOEID and poll the database instead.

Edit 3: I actually pushed this code into my web application. It is currently working but has no GUI options, yet.

[andrew]

Looks good. Can’t wait to see it in the app. Does the file_get_contents do proper url encoding? I could see that line running into problems if locations contain commas and spaces.

Also, I’m not a huge fan of hitting the woeid database every time you wish to obtain weather data. Is there a way to cache the woeid and continue to use it so long as the location option hasn’t changed since the last time? For one thing, Yahoo is a little bit more restrictive on their placefinder API than they are with their weather API. Also, the woeid queries can actually take some time (I ran the query using “Disneyworld” and it took several seconds to return).

Edit: looks like we’re cross-talking and you’re already considering the same issue.

Edit 2: Also, I completely agree that 1 minute is too frequent for weather checks. An hour is probably fine. I really only had it running every five minutes so I could test it out more easily.

[Samer]

I agree with you, I hate that I am calling it each time. But, there is no good way for me to cache it other then writing out to file because watcher.php ends its execution every minute. The best way is to switch to MySQL which will be a large headache for everyone already using my web application. I would need time to smooth out the migration path and a more compelling reason to switch than just automatic weather delays.

On the positive side, the few seconds do not effect the user at all since watcher.php is a shell run script which won’t block the UI.

[andrew]

I was tracking the code that you checked in in preparation for changing the ui to allow for delay settings and I wonder whether it may be overkill (at least initially) to allow custom delay settings per individual condition code. Most users, I suspect, will want to say: “when it’s raining, set a delay for X hours”. They won’t care to specify hours for each code. Perhaps we ought to just identify which codes signify rain and then apply a single user-set delay when they come up.

[Samer]

I agree! I was struggling to figure out the best way to do that. Do you think 24 hours is a good default value?

[andrew]

Yes. that sounds right

[Samer]

Okay, got the initial version posted 🙂 The options are in the settings panel (under primary settings).

It works by checking the weather code against known adverse codes/reset codes. The the current code matches the adverse code, it sets a delay equal to the configured auto delay duration. Default is 24 hours. If a reset code matches it removes the rain delay if one is set (used for the hot code). I tried to keep it simple but later we can build on it. Let me know what you think!

Thanks!

[andrew]

Awesome. Just pulled the code and it appears to be working (it’s not raining so it’s a bit difficult to test).

Two things:
1) The max rain delay should be a bit longer, perhaps 3 or 4 days.
2) it would be really cool if the rain delay events could be logged. Not sure how difficult that would be though.

[Samer]

@andrew Awesome! I can absolutely increase the maximum interval and will do that now.

The logging is actually a great idea, I could tack it on to the current logging system fairly easily but not sure if I should just wait until I make a switch to SQL database or not. I feel bad just adding paramaters to the log especially since I fragment user’s each time I do (those with the old log format and those with the newer ones). The last log change went fairly smoothly I believe so I will debate just adding it.

The other thing, in regard to logging, is distinguishing between automatic and manual rain delay. I don’t think I can. So the log would only show when rain delays were active not what set them. If that makes sense.

Edit: Pushed the maximum up to 4 days (96 hours).

[Samer]

I like our implementation for being very simple but I was looking at how other’s have accomplished similar goals and the one that stands out is HydraWise. They set certain paramaters and check against them. Here is an example of what I mean:

I think we can easily add something like that (very easy). My question is: do you think this is useful or just a lot of extra options?

[andrew]

nice job finding that. I had been looking around for a good example and gave up. I think this would be a great UI for it, but first we’d have to find a service that provides that kind of data. I’ll have to look through the Yahoo API to see if we can get rain accumulation, etc. Forecast is no problem.

So let’s say we got the data. I guess we could just set it up so that the script ran once per day and either did or did not set a 24 hour delay depending on the conditions. Is that what you’re thinking?

[Samer]

Exactly! Check if the predefined conditions are met and if so rain delay. Maybe check at midnight?

I know personally I start my programs early before its too hot (2AM). Not sure if that’s common practice but I like the idea of one check and 24 hour delay prevents a lot of possible problems (program continually adding a fresh delay every hour when run).

Let me know if you find the data. I know another user on the forum was parsing a lot of data from weather services but I personally couldn’t decipher all of it.

[andrew]

Can you point me to the other post?

[Samer]

Sure, I should have off the bat but was on my iPhone and being lazy.

Here is the forum post: http://rayshobby.net/phpBB3/viewtopic.php?f=28&t=148
He also posted his code on Github: https://github.com/vladbabii/multiweather

Edit: I never noticed his second commit. Looking through it I think it would be easy to formulate the correct parses to grab the data. But, I am not seeing rain fall and other data needed to implement some of the rules we want.

Edit 2: Forecast IO seems to report, precipIntensity. Not sure if that is the value we’re looking for?

[andrew]

I suppose it depends on whether we want to aggregate the precipitation ourselves or find an API that provides a rainfall report at both the 24 hour and 7 day level. Might be more realistic to do it ourselves, but we’d need make use of local storage to store the data.

[andrew]

I suppose it depends on whether we want to aggregate the precipitation ourselves or find an API that provides a rainfall report at both the 24 hour and 7 day level. Might be more realistic to do it ourselves, but we’d need make use of local storage to store the data.

Edit: yeah, the parsing will be easy. Just need to find the data.

[andrew]

Weather underground provides daily summaries which include total rainfall for the day. The problem is that users would have to obtain their own API keys and enter them (because the free keys are too limited that we all couldn’t share one).

[Samer]

I want to avoid that, seems too confusing to have them getting keys. At least from my experience.

[andrew]

Well, I can’t find any historical data sources (providing, e.g., yesterday’s weather) that do not require an API key. Furthermore, the ones that do have API keys are too restricted that we couldn’t use one API key for all of the users to share. I suppose we may just have to stick with what we’ve got so far and perhaps we can revisit if we can identify an adequate data source.

[Author]

Posts