Questions on running an OS local weather service for a Netatmo Weather Station

[franzstein]

Some thoughts and questions on running an OS local weather service for a Netatmo Weather Station.

Netatmo encrypts the communication between the Netatmo station and the Netatmo servers, without providing end-users a way to access their data locally. This means that there is no man-in-the-middle solution possible for Netatmo weather stations. the weather information has to retrieved from the Netatmo servers.

It is possible to use WeeWX for this purpose. There exists a WeeWX driver for Netatmo weather stations, which has to be run in cloud mode. Once installed and capturing data, the WeeWX solution can send the weather observation to Weather Underground or onto the local Weather Service.

Looking at the hardware needed the easiest solution will be to have two Raspberry PI’s. A first one is running the WeeWX program retrieving the weather data from the Netatmo cloud sending it to the local Weather Service. A second one acts as the Local Weather Server running the OpenSprinkler-Weather program.

My questions are: Is there a way to use only one Raspberry PI for both processes? What kind of Raspberry PI will be needed (e.g. Raspberry PI 3 Model B+)? I have the OpenSprinkler DC 3.0 installed.

At the time being I have programmed a low cost ESP32 NodeMCU to retrieve weather data from WeatherUnderground’s new API, perform the Zimmerman calculation and update the OpenSprinkler’s water level via API commands. This works perfect for me, but has the disadvantage to depend on the Meteoware Plus and Weather Underground weather services. The above solution cuts these dependencies, gives more local control but needs more HW to be installed and maintained.

Has someone already installed an OS local weather service for a Netatmo Weather Station? Comments or recommendations are highly appreciated.

Thanks in advance.

[Peter]

@franzstein, so it sounds like WeeWX may be able to extend our support to netatmo which would be great news. I think you only need a very modest raspberry pi to do this, say, Pi Zero connected to your network wifi or Pi 2 if you want Ethernet connection. The Zero should have enough horsepower to run both WeeWX and a local Weather Service. You will need to specify the local Weather Service IP as either localhost or 127.0.0.1 in the weewx configuration file. The instructions are on the opensprinkler-weather github page. Let me know if you need any help.

[franzstein]

@Peter, thanks for offering your help. It’s great, that both services can be run on the same Raspberry PI. I will give it a try, but it will take some time to get everything in place.

[franzstein]

@peter, some more questions that you can maybe answer?

According to Ray’s update on the current state of the weather script the OpenSprinkler-Weather Service implements a ‘rolling window’ approach. It also offers the selection of OWM, DarkSky or a local PWS as weather service providers.

My current understanding is that OWM is based on forecast data for the next 24 hours. DarkSky provides weather data from today and yesterday. Does this result in the use of historical data, if DarkSky is selected? How about the PWS selection? Does it also result in the use of historical data?

Or the more general question does the selection of a local PWS lead to an OpenSprinkler weather calculation similar to the old WeatherUnderground layout? I have looked at the github OpenSprinkler-Weather Service repository for details. However, I still have no complete understanding of the calculations. A more descriptive explanation is highly appreciated.

[Peter]

@franzstein, that is a good summary. Currently OS supports OWM using forecast data for the next 24 hours as the basis for a Zimmerman calculation as the standard service. But for users that are prepared to host a local Weather Service within their own network then other options become available. Firstly, a DarkSky option is in pilot that uses the last 24 hours as the basis for the Zimmerman calc. For users with a WU compatible PWS then the local Weather Service option can use that to calculate Zimmerman based on yesterday’s temp/humidity/precip plus today’s precip. The later option is the most closely aligned to the old WU method. So you can see that there are a few options based on slightly different approaches. This is all new work as a result of the recent WU changes and is going through a bit of revision and refinement so things may evolve

To break it down a bit more. There are basically two variables here; 1) the choice of weather provider, and 2) the “window” of data used. Unfortunately nothing is simple in this world and the best option is very dependent on the user’s situation.

In terms of the weather provider, then the choice goes along the lines of: 1) the PWS in your backyard is generally the best option, 2) the next best choice would be a nearby PWS but unfortunately the change in WU api has ruled this out, 3) then we need to look at an online weather provider that best matches your local conditions i.e. either OWM and DarkSky. For some OWM will provide better results but for others DarkSky may be more representative so a bit of research is needed. The last option, 4) is to go old-school and revert to manual watering levels!

In terms of the “window” of data used i.e. historic or forecast, then that is harder to answer. The theory goes that in a perfect world using forecast weather is better as it avoids stressing the plants. To put this a different way, the forecast approach is about “giving the plants enough water to see them through an upcoming dry patch” whereas the historic approach is about “giving the plants a load of water that they have been desperate for all day”. Unfortunately, the logic breaks down a bit as data is not perfect. Unfortunately, historic data is pretty accurate whereas forecast data is prone to error. So whilst the forecast approach may be better in principle it may not be better in practice. It comes down to the sophistication of the weather provider’s algorithms and their source information.

So at the moment there are a few options available to users but the best one is probably down to individual circumstances and requires a bit of trial and error.

[franzstein]

@peter, thanks for your detailed answer. It makes the decision for me easy to opt for the local Weather Service with my WU compatible PWS (Netatmo).

In the meantime I have WeeWX and the OpenSprinkler-Weather service running on the same Raspberry PI. It is definitely not needed, but I have chosen the Rasbperry Pi 3 Model B+ to be up to date with dual-band 2.4 GHz and 5 GHz wireless LAN.

Testing the OpenSprinkler Weather Service shows that the service is running correctly:

http://192.168.178.39:80/weather1.py?loc=49.47369,10.94380&wto=”h”:100,”t”:100,”r”:100,”bh”:70,”bt”:65,””br”:0
results in:
&scale=66&rd=-1&tz=56&sunrise=310&sunset=1283&eip=3232281125&rawData={“h”:81.33,”p”:0,”t”:59.5,”raining”:0}

One last thing: To configure the OpenSprinkler DC 3.0 to use the Local Weather Service I have to set the Weather Service IP and PORT (Default: weather.opensprinkler.com) to 192.168.178.39:80? TheJavaScript for the UI (default: https://ui.opensprinkler.com/js) remains unchanged? Is the OpenSprinkler System Diagnostics thereafter showing the data received from the Local Weather Service?

Realized that I have to use PORT 80 to answer requests coming from OpenSprinkler. Unfortunately the OpenSprinkler System Diagnostics still shows no Last Succesful Weather Call.

[franzstein]

It’s a little bit tricky. After entering only the IP address 192.168.178.39, without the PORT number everything is working now!

Am I right that the OpenSprinkler System Diagnostics shows the current temperature, humidity and precipitation values as retrieved from OWM? The precipitation value shown is multiplied by 10, which seems to be an error. However, these values were not used for %Watering calculations and have nothing to do with the values retrieved from the Local Weather Service?

Attachments:

1. 

   OS-Weather-Service-configuration.jpg
   

[Peter]

@franzstein, great that you have this working! The Netatmo PWS is pretty popular and the cloud plug-in for WeewWX is definitely the simplest approach. If you have a spare moment and would be willing to post a brief set of steps that you took then I would be happy to “write it up” and add to the support page to help others.

The OS version (as opposed to OSPi) only accepts an ip/hostname in the “su” configuration popup for the weather service. It is hardcoded to port 80. You are right that the temp/humidity/precip that is shown in the System Dialog page is obtained directly from OWM rather than from the local Weather Service. We are looking to update this to show the actual values used during the water level calculation so that users can better understand the results.

[franzstein]

@peter I’m more than happy to describe the necessary steps taken for including Netatmo Weather Stations.

Netatmo Weather Stations are not automatically supported by the drivers that come with WeeWX. However, there exists a Netatmo/WeeWX driver written by Matthew Wall that can be added to WeeWX. The procedure to install the driver is straightforward and best described at his github repository.

Before following these installation instructions it is necessary to obtain a Client id and a Client secret via the netatmo.com web site. For this purpose a Netatmo Connect project has to be started and an APP has to be created in addition to the username and password already available for the existing Netatmo account.

Whilst creating the APP there is a form showing up and the fields to fill in are at least <Name>, <Description>, <Data Protection Officer name> and <Data Protection Officer email>. I have filled in WeeWX Interface, WeeWX Interface to Netatmo cloud data, my Name and my Email Address. The Email Address should be the same as used for logging in to the Netatmo account, all the other text may vary. After saving this form the so called “Technical Parameters” Client id and Client secret can be obtained.

These credentials together with the username (Email Address) and password are needed in Step 3 of the Installation Instructions outlined by Matthew Wall, wich can now be performed.

I have had the problem that the “sudo wee_config –reconfigure” command didn’t change the /etc/weewx/weewx.conf file as expected. So it is best to have a closer look at this WeeWX configuration file, which should contain:

…

# Set to type of station hardware. There must be a corresponding stanza
# in this file with a ‘driver’ parameter indicating the driver to be used.
station_type = netatmo
…

##############################################################################
[netatmo]
username = <Email Address as used for Netatmo login>
client_secret = <Client secret as obtained from the Netatmo Connect website>
password = <Password as used for Netatmo login>
driver = user.netatmo
client_id = <Client id as obtained from the Netatmo Connect website>
mode = cloud
##############################################################################

…

If not the corresponding changes can be made with: sudo nano /etc/weewx/weewx.conf.

It is also necessary to stop and start WeeWX whenever there are changes made to the /etc/weewx/weewx.conf file.
Commands to use are: sudo /etc/init.d/weewx stop and sudo /etc/init.d/weewx start.

Remark: As you already described, if WeeWX and the OS Weather Service should run on the same Raspberry PI it is also necessary to specify the local Weather Service IP in theWeeWX configuration file as follows:

..

[[Wunderground]]
enable = true
station = anyText
password = anyText
server_url = http://127.0.0.1:80/weatherstation/updateweatherstation.php
rapidfire = False

…

Hope to be some help.

[Peter]

@franzstein, that is fantastic and much appreciated. I will get this up along with the other instructions around setting up a local Weather Service.

[skyynet]

I have a Synology at home which already does some home automation jobs. I saw some hints about successful installations of WeewWX on Synology. Unfortunately I didn’t find a complete installation and configuration manual. When I bought my Netatmo with Rain Sensor I wanted to connect it to my OpenSprinkler to have reliable data from my own station. I understood that Netatmo stations can send data somehow to WeewMX and OpenSprinkler could access these.

Anyone with more infos on this?

[Peter]

@skynet, it is possible to use WeeWX and a local Weather Service to connect your Netatmo PWS to OpenSprinkler.

However, most of the instructions to do this assume you are using a Raspberry Pi (see HERE for end-to-end Netatmo guide). I dont know of any equivalent end-to-end guide for a Synology specific setup. My suggestion would be to use a RPi in this situation, if you have OSPi then you can use that.

If you really want to go down the Synology route then below are some pointers but you will need to adapt them to your situation and the Synology operating system:

A) Install NodeJS on the Synology: A good guide is this LINK. Follow the instructions for Package Centre and also to test that node js works using the little “test.js” script. Dont bother setting up node to restart when the Synology boots up just yet as we can do that later

B) Install Weather Service on Synology: The instructions to install the Weather Service on a Synology are very similar to installing on a Raspberry Pi and can be found HERE. Follow steps 2-5.

C) Install WeeWX on the Synology: I haven’t seen a specific installation guide for WeeWX on a Synology NAS but you might be able to follow the generic WeeWX instructions HERE.

D) Configure WeeWX for Netatmo: The steps to register your Netatmo PWS with their cloud service and then install a Netatmo plug-in for WeeWX is provided HERE with Step 3 onward.

E) Ensure Weather Service Will Auto-Start: To ensure that the Weather Service will automatically start whenever the Synology boots then go back to the Synology video LINK and right at the end it shows an example script that needs to be created in /etc/init directory. Do this but delete the line with “setuid” and change the the reference to the “test.js” file to the “server.js” file in the Weather Service directory.

F) Test the Set-Up: Go back to the standard instructions for installing a local Weather Service HERE and follow steps 7 and 8.

[skyynet]

Wow @Peter, that’s a great help to start and as a thank you I’ll bother you with stupid newbee questions 😉

I managed to finish the installation of node.js successfully which is v8 at this time.

I can’t install the weather service described in B) as my Synology doesn’t know the “git” command. Does this mean I have to install a git server on my Synology?

[Peter]

@skynet, if you go into the Synology Package Manager, you should be able to install the Git Server package. This is a bit overkill but will provide the git command line tools.

[skyynet]

Okay, I’ve installet git and managed to finish B)
I have Python 2.7.1 installed and downloaded WeeWx to weather and unpacked it. Then I cd’ed into weather and run

./setup.py build

and get the error:

Traceback (most recent call last):
File "./setup.py", line 19, in <module> import configobj
ImportError: No module named configobj

Any idea?

[Peter]

@skynet, unfortunately, I don’t have experience of installing WeeWX on Synology. As a wild guess I would suggest trying pip install configob. But I fear you might get “pip: unrecognized command”!

You might want to try the WeeWX google forums as I have seen some threads about Synology installations before. If you do manage to get past this obstacle then happy to help with the other steps.

[wifi75]

Traduci da: Inglese
737/5000
I already have the configuration of wundergroud in weewx because I transmit the data directly to Wundergroud as follows:

[[Wunderground]]
        enable = true
        station = IPONSO2
        password = my_password
        server_url = http://192.168.1.50:3000/weatherstation/updateweatherstation.php
        rapidfire = False
        #api_key = replace_me

I added the server_url line, do you think it transmits data both to the local weather and to Wundergroud?

[franzstein]

Interesting question! From what I know it transmits data to the local weather only, if you add the server_url line to the Wunderground section. In general it should be possible to transmit to Wunderground, too. However, there are probably some changes to WeeWX needed?

[skyynet]

I found a really easy solution for this: http://plus.meteoware.com/netatmo/.
This web service acts as a translator between Netatmo and other weather services.

[franzstein]

I use Meteoware in addition to WeeWX to transfer the Netatmo data to Wunderground. In cases of heavy rainfall the precipitation summaries of WeeWX gives less rain amounts than Netatmo or Wunderground. Everything else works perfect!

[wifi75]

what version of weewx do you have?

[franzstein]

I use WeeWX 3.9.2 right now. There are problems with WeeWX 4.0.0 in conjunction with Netatmo PWS. Please have a look at: https://github.com/matthewwall/weewx-netatmo/issues/15.

[wifi75]

My PWS is ecowitt with gateway GW1000

[Author]

Posts