Forum Replies Created
-
Posts
-
——- Wow, the forum just deleted my post as well ! I’ll try not to take it personally and hopefully this wont be duplicated ————
Hmm … OK, so a bit more on the inner workings of Firmware. The three log api calls all operate in a vary similar manner. In each case the OS is reading every entry from the log within the date range. In the case of the “no type” call, it will discard any “wl” and “fl” readings. In the case of “fl” and “wl” types, they discard anything but the specified type. So in this way the OS is reading the log from SPIFFs the same in in all three cases. If there was a SPIFFs problem then I would have thought all three responses would be impacted. Given that only “wl” is effected then it appears that’s not the problem.
But to rule out any possible data corruption of the “wl” log entries causing problems, have you tried doing a full “walk” of the log history using overlapping bite-sized “wl” requests. If you can cover the full range and the records look well formed then that should rule out data corruption in the “wl” records much the same as we have done for the others.
As you point out, a big difference between the three commands is the size of the response. The “wl” entry happens every hour and so there should be, give or take, 24 entries per day. In many setups this could be perhaps an order of magnitude more entries than station events. Thus the response from the “wl” query could be a larger packet. Having said that the buffer on the ESP8266 is 4k and will segment the response if needed. Hence a buffer overrun seems unlikely but not impossible. In one of your earlier responses you mention network connection problems coinciding with a large “wl” request. How consistent is this behaviour (i.e. always at a certain number of records) and is there any indication that the OS is resetting at this point?
So barring the above, I am kind of left with suspecting a network problem. An alternative way to get the OS view on wifi signal is to issue “http://os_ip/jo” command and look for the RSSI field in the returned json string. I know you have a couple of units and one works well on the wifi extender but I am bit old school and always keen to remove variables when testing. You wouldn’t need to have the expansion module/valves connected to do a bit of api testing just a strong connection to your main wifi router. But that’s easy for me to say as I don’t know how hard it is to “just 8-)” disconnect everything so your call.
UPDATE: I think the command to get RSSI is actually “http://os_ip/jc” but just going from the code on that one.
What you should be seeing on the network via wireshark is three API log commands coming from the App. One is for all water level log records by specifying the “type=wl”. Another is for all flow records with “type=fl”. And another for all other records by leaving the type filter off. The App then puts the three sets of data together to display the timeline/table. This is not particularly clear in the API specification but is consistent in the firmware and app code. Although I recall seeing a reason for splitting the request into three, I cant locate it at the moment. It may just be historical as “wl” and “fl” logging may have been added at a latter point in time.
For completeness, I should have suggested to do the same three commands from the browser (i.e. to also try “http:/os_ip/jl?&hist=21&type=wl” and “http:/os_ip/jl?hist=21&type=fl”). If you don’t have a flowmeter configured then the “fl” call should return “[]” and if you dont use a weather adjustment method then the “wl” call should also return “[]”. If you do have a weather adjustment method setup (i.e. via weather underground) then you should be seeing entries of the form “[0,”wl”,75,1493943439]” which represents 75% watering times.
Be good to check that these calls return valid data as well. From your comments above it sounds like you tried “http:/os_ip/jl?&hist=21&type=wl” from the browser but got no response rather than a “[]” response? If that is the case then there may be a firmware/hardware-side issue. Also, can you confirm your settings for flow meter and weather adjustment so we know what we should be seeing.
If the problem is network related then a few things worth trying. Firstly, it would be good to know the signal strength of the wifi connection between the unit and the extender. From memory the OS 3.0 creates an Access Point at boot time that you can connect to and scan the network. You might be able to see the signal strength for your extender’s SSID from there. Secondly, can you “ping” the unit from your mac and see what the delay time is. I connect to my OSPi via wifi->ethernet->powerline->wifi which is quite a trip and has a long ping of around 250ms but with no packet loss. Thirdly, it would be worthwhile relocating the unit near your primary wifi controller and again via the OS 3.0 Access Point connect the unit to the main wifi, i.e. remove the extender from the equation, and retry all of the tests. Finally, it might be worth swapping two of your units with each other (use export/import config from App to help) to see if the problem follows the unit or stays with the location (but this would require take a week to build up a sufficient log record.
Let me know how you get on.
Pete, So, I configured my OSPi for 6 programs and 22 stations to match your log file. I stored the records in the OSPi log directory and used the App interface to display the log from 22/01/18-11/02/18. The timeline view and the table view both showed 193 records which corresponds to the number in the log file (note the rain delay event correctly shows up in the timeline/table but is not included in the count as per the code). There were no errors flagged in the debug log from the Firmware nor any issues highlighted within developer tools from the App. I have attached a screen shot of the log view. I am unfortunately unable to replicate the situation you describe. This may be down to the different hardware but the code (apart from reading the SPIFFs vs SD card) is pretty much the same.
One question, did you pull the records via the View Logs Export button in the App or did you capture the records by typing “http/os_ip:8080/jl?hist=300” into a web browser? The file you attached suggests the former but I was wanting the latter. The feed provided by the http command is a “raw” feed from the log files whereas the View Log Export is a more processed list.
As one suggestion, given you have Chrome browser, have you tried having the Developer Tools open (…/More Tools/Developer Tools) when displaying the log. You may get lucky and see an error message that helps point to the problem.
Attachments:
Pete, not sure if I can help but if you could attach a copy of your log (accessed via the API right back to 23 Jan) then happy to try and replicate the situation on my development system. I’m on OSPi so not quite the same setup i.e. SD card vs SPIFFs but we might get lucky. Peter
The current version of the software supports this capability. Installation/upgrade instructions are on the opensprinkler support webpage here
For others, the server field expects a server identifier as either an IP address or a hostname. There is no support for url encoded authentication via the server field. As the above post points out, supplying “<username>:password@<IP Address>” into this field causes OpenSprinkler to look for a device named exactly that on the network and fails.
The “man-in-the-middle” approach suggested above is a good workaround where authentication is needed but requires an available web server. For those using OSPi, you can install the apache server onto your existing OSPi raspberry pi to leverage this option without needing additional hardware. Not the ideal but a way forward…
Yes, the GPIO Station functionality lets a raspberry pi only solution drive many relay boards. You would need to ensure that the gpio pins do not conflict with what the firmware reserves for driving an OSPi expansion board.
Mike, I did the HTTP Station function as a fairly simple extension of existing code to provide basic support for off-the-shelf http devices. As you say there is no way to specify dynamic on/off commands. What you would need is some kind of meta_tag that could be specified in the on/off command fields but substituted live at time of sending (i.e. $time_remaining or $off_time). This would get tricky real quick as you would need to support multiple variables and formats based on whatever the remote device wanted (e.g. relative time in seconds vs minutes or perhaps an absolute time in a particular ISO format). A great feature but probably not high up on anyone’s list I suspect. Cheers, Pete
The periodic resend is the default operation. This is to ensure the http station is kept in sync if a message goes missing. You can change the default in Options/Advanced so that only a single message is sent on the state transition.
A .png format version of the OSPi v1.43+ schematic is on Ray’s github here.
OK, so I too had this problem. I took a look at the logs on my OSPi and found a few strange entries across the last 6 months of files:
[254,4,120,1485693673, inf]
[99,4,120,1485696549, inf]
[99,4,120,1485697194, inf]
[99,4,120,1485698351, inf]
[4,4,108,1486404109, inf]
[4,4,117,1487008918, inf]If you have a flow meter enabled then the final field should be the flow rate associated with the station run. I think the “inf” could be the result of a divide by zero error and the log entries are rejected by the App and trigger the UI error message. Removing these lines resulted in successful logs being displayed for me in the UI. Looking at the code, this could happen if you have a Flow Meter enabled and short duration/low flow runs (my station 4 is just a single dripper for < 2 minutes).
I would be interested to know if this is the same issue as mentioned above and in #46873. To check, on OSPi, you can login to the Pi, change into the log directory and type “grep inf *” to see if any of your logs have this. On OS (and OSPi), you can use the web api to download all of the logs by typing “http://os_ip:8080/jl?hist=300” into a web-browser (may need MD5 password hash if enabled) and take a look. You can then delete just the individual daily files rather than the entire log history. Let me know.
Crosmax, Code for above post as unable to attach .ino file. Rename as tester.ino.
#include <ESP8266WiFi.h> #define VALVE_A_CONTROL_PIN 0 // GPIO0 (D3 on ESP-12E with ESP Motor Chield) #define VALVE_A_ENABLE_PIN 5 // GPIO5 (D1 on ESP-12E with ESP Motor Chield) void setup(void) { Serial.begin(115200); Serial.printf("Setting up pins\n"); pinMode(LED_BUILTIN, OUTPUT); pinMode(VALVE_A_CONTROL_PIN, OUTPUT); pinMode(VALVE_A_ENABLE_PIN, OUTPUT); digitalWrite(VALVE_A_ENABLE_PIN, HIGH); Serial.printf("Starting loop\n"); } void loop(void) { Serial.printf("."); digitalWrite(LED_BUILTIN, 1); digitalWrite(VALVE_A_CONTROL_PIN, 1); delay(2000); digitalWrite(LED_BUILTIN, 0); digitalWrite(VALVE_A_CONTROL_PIN, 0); delay(2000); }[Apologies if duplicated post – @Ray, forum doesn’t seem to allow attaching .ino files ?]
Hey Crossmax,
Glad the code was useful but sorry to hear that the new hardware is not up and running. Can I check that the motor drive board you have is the same as I have pictured above?
I have attached the wiring diagram that I use but, from your notes above, I’m not sure why your approach hasn’t worked. In my case, I need a higher voltage than the onboard regulator can handle to guarantee latching of the valve so I use a 12V DC supply to drive the solenoid and a DC-DC converter to peel off 5V for the nodemcu. In this configuration, I have the jumper between VM and NC ensuring the power rails are kept separated.
I have attached below a short test routine that I used to check that I could control the motor board. The routine just toggles the Motor A terminals between +VM and -VM on a 2 second cycle in sync with the pulsing blue led on the board. You compile through the Arduino IDE with NodeMCU board selected. This should let you put a voltmeter across the terminals (suggest without valve attached) to check you have the hardware working.
Only other thought off the top of my head is that they may have updated the board and swapped the control pins. Do you have any documentation to confirm pin outs (mine attached) and/or can you follow the traces to see if that might be the issue.
Let me know how you get on and happy to help,
PeteAttachments:
Till,
Thanks for the feedback on the HTTP Station code. I have to put my hand up here as the guy responsible. I submitted a PR to Ray and Sammer last year for this as a simple extension of the Remote Station code. I recognised the space limitation of circa 250 characters across all parameters. This is a design template constraint where parameters for Special Stations get processed within the OpenSprinkler working buffer (~250 bytes). Using the existing design pattern meant for a simple implementation but with a knock-on constrain. This is ok in many cases but falls down when you need to authenticate as username/passwords push out the character count!
Nowadays, many browsers/services are moving away from encoding username/password in the URI and are using html headers to carry the authentication. So I saw this as more of a need to add authentication headers in the future rather than needing to extend the command length (good bit of post-rationalisation 8).
The defects on the other hand aren’t great. Thanks to your investigations, I have replicated the issues above and have seen that the length checking I do is too late in the process i.e. I check lengths in the Firmware after decoding the html message from the App. This is too late to catch the buffer overrun and inform the user. I have a fix that I’ll package up and submit.
In terms of immediate options, I have seen that Domoticz allows you to remove authentication if that is acceptable. No good if you are allowing remote access but might be OK if everything is behind your router firewall? Alternatively, I see Domoticz also allows you to whitelist a network (i.e. waive the authentication if sourced from a known secure network). On their Setup/Settings page you have “Local Networks (no username/password)” field into which you could add 192.168.0.*, or equivalent, to waiver authentication for the local network. Not sure if that helps but I’ve just tried it out and it seems to work.
As I say happy to take feedback on importance of this feature and whether authentication/command length is something in demand. To be honest, I didn’t think many others where using this mode and glad to see that there is some interest in it. Let me know your thoughts and I’ll try to improve the implementation.
Thanks, Pete
@crossmax, No problem. I have put up the code I’m using on my GitHub here. It is specific to the Motor Shield that I have but should be pretty easy to modify depending on the particular hardware configuration you are using. Happy to help out so feel free to follow-up in the other thread or via github. Cheers.
@crossmax, I think what you are looking to do is use a single raspberry pi to run the OSPi firmware and to also control a latching solenoid? This is different to my situation. I do not have OS firmware controlling GPIO pins to drive latching valves. The firmware is not geared up to do this out-of-the-box as you mention above.
In my case. I already had an OSPi controller running all of my stations, programs and regular valves out in the garden. My goal was to find a way to add an indoor latching value into the system without having to run wires into the house. So I wrote a bespoke program to run on an ESP8266 that simply “listens” for on/off commands from the garden OSPi and in response opens/closes the latching valve. So my solution has OS firmware running on the garden RaspPi and a simple gpio routine running on the ESP not a combined unit.
You can check out the this post where I set out the solution. I’m happy to post the code if useful and talk through the hardware but it was Arduino based for an ESP8266. So it would not be a simple copy and paste onto a Raspberry Pi.
If you really want everything in the one unit then I think you would need to either extend the OS firmware to support latching valves or alternatively create a “man-in-the-middle” programme that monitors the OS firmware status and then appropriately configure the H-Bridge hardware. You should also check out the new OSBee 2.0 unit as an option
Happy to talk through.
PeteCan I check that the four GPIO relays operate correctly it is just that the relay pump GPIO has the LED illuminated? Also, does this only happen after the RPi board is reset and before OSPi has had a chance to run a watering cycle that uses the relay pump? If that is the case then the following may help you out.
When the Raspberry Pi is reset, the GPIO pins are set by default to be inputs rather than outputs. This is a safety measure so that the GPIOs do not start driving unexpectedly behaviour in attached hardware before the software has a chance to get in control. However, some of the GPIOs in input mode are “pulled” down so that they default to read 0V whereas others are “pulled” up so they default to read 3.3V. This is all very well defined behaviour set out in the RPi GPIO specification. Unfortunately not all relay boards deal with the pull up/down difference very well.
As it happens, GPIO11 is one that is pulled down to 0V and as a result a very small current can trickle through the LED on some Relay Boards. On my SainSmart relay board it is just enough current to dimly illuminate the LED but nowhere near enough to trigger the relay. So it operates the relay correctly but just a little disconcerting that the LED can sometimes imply an incorrect state. Since you are only using 4 GPIOs, you could switch to using GPIOs 5, 6, 7 and 8 as they are all pulled high and should not flow enough current to illuminate the LEDs following a reset.
Let me know if the above helps or not as there may be other things we can try (the following youtube gives a pretty detailed explanation of how the circuitry works but not for the faint-hearted relay video 8)
Gerrit,
I don’t think there is a way to setup authentication parameters via the UI. OS expects the contents of the server field in the UI to be either a simple server name or ip address, i.e. 192.168.1.210. It does not have code to parse the user:pass elements. I had a hand in writing this bit of the code so apologies for not making it more extensible to authentication.
If you are comfortable changing the firmware code and rebuilding then there may be a way to hardcode the user:pass details into the firmware by modifying a line in function switch_httpstation() in file OpenSprinkler.cpp ((line 1172 in current version):
From: ether.browseUrlRamHost(PSTR(“/”), cmd, server, httpget_callback);
To: ether.browseUrlRamHost(PSTR(“/”), cmd, server, PSTR(“Accept: text/html\r\nAuthorization: Basic dXNlcjpwYXNz”), httpget_callback);Where dXNlcjpwYXNz is the base64 encoding of user:pass string which you can get via one of the online converters (google “base64 encode” – https://www.base64encode.org/)
Unfortunately, I can’t test this (I dont have an http station requiring authentication) but it “may” work. Obviously, it is somewhat limited if you have multiple http stations with different usernames/passwords. Ray may have a better answer but I am curious if this works 8)
Jeff,
I added the zimmerman baseline parameters to the OS firmaware a while back and happy to try and answer your questions as below:
1) My temp and rain amounts are in Celcius, the directions for settings show in english units, will it still work for me?
The web app uses the user’s location to determine whether to display weather parameters in imperial or metric in the user interface dialog. So provided I have the location logic correct then you can enter in your baseline settings in locale specific units which the UI will convert to imperial before sending on to the OS Firmware to do its magic.
2) In the Weather Adjustments Options show you can set baseline conditions. What should these be set at?
The basic zimmerman logic is based on shortening/lengthening watering duration depending on whether the weather conditions are above or below an “average” or baseline day. In the default set-up the baseline day is set as 21C, 30% humidity and 0 rainfall. Unfortunately, in London this is not a great match to our average day. Here the weather is colder and more humid so it makes more sense for zimmerman to adjust watering times based on variation from an average London day which is 15C, 70% humidity and 0 rainfall. By configuring a more locally relevant baseline, zimmerman will operate in a more intuitive manner. So the recommendation would be to set the baseline to be your average weather conditions and then set the watering duration for your programs to be the right amount for an average day. Zimmerman will then increase/decrease watering duration based around the local baseline.
3) There are sensitivity adjustments, how should these be set? The documentation doesn’t mention these.
The sensitivity options allow you to further customise the water duration calculation. If you find that the zimmerman logic is not increasing water duration enough in hot weather then you might increase the temperature sensitivity. Alternatively, if you wanted to ignore any change in humidity then you could set the humidity sensitivity to zero. This is where trial and error comes in as you calibrate the system to your specific watering needs/conditions.
4) My Weather Underground account is in F but in opensprinkler the values are metric. How to change? Does it matter?
The weather underground api provides access to both imperial and metric values. As it happens, OpenSprinkler always draws from the imperial values behind the scenes and only converts units back and forward for display purposes in the user interface. So whether you set weather underground to imperial or metric should not matter.
5) What does the response from this mean: http://weather.opensprinkler.com/weather1.py?loc=XX.XXXXX,XX.XXXX&key=XXXX (filled in my GPS and WU key). Just wondering?
The response from the weather server (weather.opensprinkler.com/weather1.py) contains 6 parameters the first and most relevant for here is the “scale” parameter which is the output of the zimmerman calculation as a percentage multiplier that gets applied to the zone water duration (a number between 0-200%). This is what the firmware uses to adjust the watering times when running a watering programme. Another interesting parameter is “rd” or rain delay that tells OpenSprinkler to defer watering if raining. Other parameters help OpenSprinkler understand sunrise/sunset times as well as timezone.
Let me know if any other questions. I am quite interested to hear if people are using the baseline parameters and have any feedback on the way it is setup.
Cheers, Pete
adrian78666,
I have attached the instructions that I used with a friend to set-up Apache for local web hosting on his OSPi. The steps are quite detailed and assume a clean install from a new SD card. As Samer said above this requires a bit of knowledge of Debian and Apache so not for the faint hearted!
Note that hosting the web site “only” gets you a system that is less dependent on the OpenSprinkler cloud services which may be your preference. If you are looking to customise the web app then that is a whole other order of magnitude when it comes to configuring the system for web development. Curious what you are looking to achieve ?
Let me know if the attached helps. Pete
Kimbhoot,
If I understand this and your related post, you are trying to use a Raspberry Pi to control a relay board using OS firmware? If this is correct then it would be useful to know the version number of the firmware (you can find this from the About menu in the web app). The GPIO Station functionality requires the latest Firmware version 2.1.6 (2).
If you are not on the latest Firmware then you will need to upgrade using the instruction on the opensprinkler.com support website (link).
In the latest firmware, you should see a new advanced configuration option when you select the settings cog of a station in the browser. You should see a GPIO Station option that lets you associate a GPIO pin with a station.
Let us know if that resolves the problem.
PeteKeith,
OK. so the new version of the App exposes the new Zimmerman baseline feature hence you can see the fields for baseline temp/rain/humidity. However, to make use of this feature you also need the latest version of firmware 2.1.6 (2). Since you have firmware 2.1.6 (1), the App has “grey’ed out” the fields as your firmware version does not support the new feature parameters.
So if you want to make use of the zimmerman baseline – this is essential for me as I’m in a very high humidity location – then you will need to update the firmware using the instructions on the support page (Update Guide)
If I were coding this feature again, I might have kept the fields hidden until both the new App and the new Firmware were both running rather than “teasing” a new feature that causes some end-user confusion. Sorry about that and I’ll take the knowledge on-board for next time.
Pete
