Incorporating other information into station log

[Vaughano]

Hi Samer and all.

Really impressed with the way your mobile webapp has progressed. I’ve been a bit distracted with other projects, but the weather here in South Australia is warmign up, forcing me to come back to irrigation!

One thing that I am looking at is incorporating other logs into the mobile webapp. In particular, for each station I want to be able to monitor the pressure in the line after that valve. Here’s a system diagram of what I’m attempting:

What I’m thinking is having the logs appear in a timeline form, like the program preview. I think this would allow more possibilities for overlaying other logs in the same timeline.

I have been looking at the code briefly, but I am struggling a little with working out the data structures and presentation in the PHP code. Admittedly, I am using nano editor on my server!

I could use some tips on the following:

• Any pointers on where to focus my attention in the code?
• Strategies for branching and pushing changes back to main codebase.
• Ideas for overlaying other logs (will just be time series numbers, probably sampled at 60 second intervals or something)
• And also a good IDE/editor for PHP?

Cheers
Vaughano

[Samer]

Very nice diagram, looks like a fun project!

I believe, you want to understand how the program preview is rendered so you can adapt it to logging. If that is the case, understanding the PHP will be difficult due to the data generation used. To simplify it here is what the data structure should look like:

Line 402 in main.php:

echo "{'start': ".$start.",'end': ".$end.",'className':'".$class."','content':'P".$pid."','group':'".$data["stations"][$sid]."'},";

For each object you append a list to an array. The list contains start and end time, content, a group name (station) and optionally a classname.

Line 762 in js/main.js shows how the data is retrieved from the server and rendered.

Github allows making changes and pushing them upstream. It is very easy, all you do is fork my code, push changes to your repository. Once you have a working feature make a pull request and I will approve the merge.

Right now, I show many data points on a graph. Using the preview method you can group them by type (instead of station), maybe?

Personally, I use Sublime on OSX to edit any scripts. For Windows, I believe I used to use Scite.

[Vaughano]

Thanks Samer.

I’m not too familiar with PHP, but I do have a software background in Java, so this info should be enough to get me going.

Cheers
Vaughano

[Samer]

By the way I should note two things.

First, the documentation and original javascript: http://almende.github.io/chap-links-library/js/timeline/doc/

Second, the code I am using for my mobile app is slightly modified. I commented the portion of the code that alphabetically sorted items which allowed me to manually sort them. I could not find an API hook for disabling it, but if you do let me know.

[Vaughano]

Thanks Samer, I’ll have a look at the API for the timeline component.

Just by way of an update (and a written plan for my own benefit), I have dug through the code and here is my plan:

• add a radio button to the logs page for ‘Timeline’ option, in main.php
• add code in main.js.php; a function called get_logs() to retrieve and parse logs into timeline component JSON format (partially done, plugs some HTML in saying ‘not implemented’).
• add another function in main.js.php to perform the parsing of logs to this format, based on the existing get_preview() function. Perhaps call it get_logs_timeline()
• a server side handler for this function, that does the lookup to the interval webapp, and possibly another webapp that serves pressure sensor data, in a similar manner to the existing functions get_preview() and process_programs() in main.php.
• Slight changes in log JSON data for the Timeline widget to show average pressure on the coloured bars (like the way programme name is shown in preview).

Any thoughts on this? I will take a light-touch approach with existing code to make it easier to integrate back into yours. I have forked your code into my own repository at https://github.com/vaughano/OpenSprinkler-Controller, and will send you a pull request when I am happy with it.

Cheers
Vaughano

[Vaughano]

I have a prototype log timeline pushed to https://github.com/vaughano/OpenSprinkler-Controller/tree/timeline-logs. This just renders some static (hard coded) data for 2nd September 2013, with associated average pressures on the lines after the relevant station.

Next step is to gather that data, from the interval program logs, and another web api to get pressure info for a given time range.

Cheers
Vaughan

Edit: Note that the colour of the bars indicates the strength of water pressure on that line. Strong colour – strong pressure; weak colour weak pressure.

[Samer]

That’s awesome! The only suggestion I have is renaming it from timeline to it’s actual name? something like water pressure? The only reason I mention it is because the logs first two options represent water timing per station whereas the timeline in your modification seems to do water pressure?

Subsequently, we can make the 3rd tab only appear if a setting is present to show logged pressures. This is done in config.php and basically enables/disables reporting of the data if its available. You could also add options to point to the data if needed.

Let me know what you think though

Update: I just looked through your code, you nailed it! Looks awesome can’t wait until its smoothed out.

[Vaughano]

Thanks Samer.

Yeah, I was also thinking that the naming could be better. Not sure that this need be specific to showing water pressures though; people may prefer this visualization to the plotted graph even without the pressure readings. But timeline is a bit generic and could easily describe the plot under the ‘graph’ option.

Definitely the handling of water pressures needs to be a configurable option. I will get to this and using real log data soon. Thanks for the encouragement!

Cheers
Vaughano

[Vaughano]

I’ve just pushed another bunch of changes to https://github.com/vaughano/OpenSprinkler-Controller/tree/timeline-logs. These changes fix some introduced bugs in the preview timeline (due to component name clashes), introduce a new action in main.php for looking up detailed log data (including water pressure, flow, air temps, whatever else we can think of), and changes in main.js.php to render the new detailed JSON on the timline component.

It is still serving static JSON for this data, but the infrastructure is there now to plug in an external web service that can produce it.

TODO:

• Allow configuration to enable/disable extra log details and/or timeline view in logs
• Allow configuration of water pressure thresholds for timeline bar colouring
• Allow configuration of external web service that will provide detailed logs (might branch the interval program to do this; still thinking this through)
• Plenty of other things I haven’t yet thought of!

Cheers
Vaughano

[Samer]

By the way, the get logs function has become clunky which is 100% my fault. I should make those hide/show into proper functions so we don’t have to keep declaring things all over the place. I can do one of two things, I can push some changes to your Github branch timeline-logs or I can wait until you are done and it is merged with my line and make some changes.

More of a Git/Github topic now but if I made changes to my copy now and later merged your changes, what would happen to my changes? Would they get overwritten?

Update: Even the navigation elements can be condensed. By the way, if it helps for testing purposes, you can setup an array in PHP and send the JSON using one line:

echo json_encode($array);

This way you can generate random code based on some loops and see the results?

Also, I have some code early in main.php (Line 32) do demonstrate how to set new options such as an upgrade or new install situation. This makes the new option in config.php but uses pretty limiting filter to prevent injection which is integer conversion. Initially, this seemed great for 1/0 situations. I also adapted it for auto rain delay duration setting. But I have a feeling we might be reaching the limit if you need to configure a web source for the data.

I am thinking we can do a cross-domain to the other service instead. This way you are not relaying data around through web servers. The client should get a quicker result. The best part is JSONP is allowed to be cross-domain but regardless we could add the cross-domain header if the IP was configured in the settings.

[Vaughano]

@salbahra wrote:

By the way, the get logs function has become clunky which is 100% my fault. I should make those hide/show into proper functions. So we don’t have to keep declaring things all over the place. I can do one of two things, I can push some changes to your Github branch timeline-logs or I can wait until you are done and it is merged with my line and make some changes.

I’m not sure there is a right or wrong approach here. You could make changes to your main branch, and I can pull these in to mine (I’m new to Git, but I’ve used CVS and SVN for years, and that’s what we’d do with those tools). That way I would be performing the merge on my branch, theoretically making the subsequent merge of my branch back to the main trunk easier.

Otherwise you or I could make the change on the timeline-logs branch and merge all the changes eventually back to the main trunk. My gut feeling is this would make for a more difficult merge later.

More of a Git/Github topic now but if I made changes to my copy now and later merged your changes, what would happen to my changes? Would they get overwritten?

As I said above, if you did that, I would take it on myself to merge forward those changes to my branch, so they would still be there when my branch is merged back to trunk (master branch in Git talk).
I’m still learning Git myself, but we’ll work it out.

Cheers
Vaughano

[Samer]

In this situation, being a nit-picky thing, I will wait until your working copy is merged in and then do some post-merge tidying. But I get what you are saying about pulling my changes before pushing, makes sense.

Thank you!

[Vaughano]

No worries, I will keep you up to date with the progress here.

[Vaughano]

Would really like to get back to this, but my part time job has become full time coding, and I simply haven’t had the time (or inclination) to get back to it.

Just an FYI to anyone who was following.

Cheers
Vaughano

[Author]

Posts