Weather Monitoring

The MIT sailing pavilion uses a Davis Vantage Pro weather station with weewx.

In the autumn of 2011 we installed an Ambient Weather WS2080 weather station at Whitehead Light Station. For data collection hardware we tried a DreamPlug from globalscale and Nimbus from Ionics. For the data collection and display software, we started with wfrog/pywws, then wview, then weewx.

System Summary
Station:WS2080 with two consoles
Computers:Nimbus plug computers
OS:Debian 6
Software:weewx
nagios and nagiosgraph
check_weewx


Software

exfoliation for weewx
0.31 27 aug 2014
screenshots

This is a skin for Tom Keffer's weewx weather collection/display software. It can be installed along with the standard weewx skin, or as a replacement for the standard skin.

simple for weewx
0.3 7 feb 2014
screenshots

This is a skin for Tom Keffer's weewx weather collection/display software. It can be installed along with the standard weewx skin, or as a replacement for the standard skin.

amphibian for weewx
0.9 7 feb 2014
screenshots

This is a skin for Tom Keffer's weewx weather collection/display software. It is designed to look and feel like wfrog, but without the googlechart dependencies. It can be installed along with the standard weewx skin, or as a replacement for the standard skin.

forecast
2.4.1 27 aug 2014

The forecast extension includes a service, skin, and search list extension for weewx. The service will download forecasts from sources such as the Weather Underground and US National Weather Service, generate forecasts using the Zambretti method, or generate tide predictions. The skin and search list illustrate how to use the forecasts inline or in tabular form.

cmon
0.5 21 aug 2014

This is a weewx service that monitors the cpu load, memory usage, disk usage, and network load.

owfs
0.7 26 aug 2014
owfss
0.9 21 aug 2014

One-wire support for WeeWX. The owfs is a driver for weewx, the owfss is a service for weewx. Use the driver for standalone one-wire applications. Use the service to integrate data from one-wire sensors with data from a weather station.

wbug
owm
wetter
emoncms
seg
xively
Extensions to weewx that upload data to:
  • OpenWeatherMap
  • WeatherBug
  • wetter.com
  • EmonCMS
  • Smart Energy Groups
  • Xively (Pachube/COSM)
icons-aw-0.1.tgz
icons-wu-0.1.tgz

Icons for use with the weewx forecasting module.

exfoliation for wview
0.7 12 feb 2012
screenshots

This is a skin for Mark Teel's wview weather collection/display software. It provides a display of weather data in a format close to that of the classic and chrome skins, but with less visual clutter. It still uses tables for the layout, but it uses CSS for styling and there is much less duplication of code in the implementation. It also adds a phone.htm page targeted toward mobile devices. This skin is included in the wview distribution as of 5.20.

wview-util
0.1 26 nov 2011

This is a collection of utility configurations for wview, including logrotate, logwatch, rsyslog, and apache.

check_wview
0.3 13 nov 2012

This nagios plugin checks either the database or html index file to see whether wview is doing what it is supposed to do. Warning or critical is indicated if the last update is greater than the warning or critical threshold, respectively. If the perl DBI::Sqlite3 module is installed, this plugin will report weather observations as performance data.

check_wfrog
0.3 16 nov 2011

This nagios plugin checks the last entry of the wfrog csv data file and compares that with the current time. If the delta is greater than the warning or critical threshold, then warning or critical is reported. The delta is reported as performance data. If not warning or critical, the weather data are also reported as performance data.

Guides

weewx on a plug

How to install weewx on a plug computer. Plug computers often have only flash for storage, so there are a few tips and tricks to reduce flash wear when installing weewx on such a computer.

weather station comparison

This table compares features of many weather stations.

Notes about the process

Fine Offset Bearings

The wind direction bearings failed after 2.5 years of exposure to saltly coastal air and occasional salt spray. Attempts to free them failed. Replacements are easy to find - search for 5x10x4 bearings. These are 5mm by 10mm by 4mm bearings that come with either rubber shielded or metal shielded ball bearings. They cost less than $5 for a package of 10 from companies such as Apex RC Products.

 

The Voltage/Power Really Matters

If you are experiencing USB issues with a Fine Offset station, check the power supply of the USB hub (or computer) into which the station is connected.

If you plug the console into a USB port with low voltage or insufficient power, the console might repeatedly reboot itself. It would appear that the reboot happens as the console attempts to connect with the sensors.

With slightly more power (but apparently still less than the station requires), the station will display data and communicate with the sensors. However, attempts to communicate with it via USB result in the same symptoms as a USB lockup.

The supplied power also seems to affect the station's ability to communicate with the sensors, and this seems to vary from console to console. We tested two consoles in a location 350 feet from a sensor. One console reliably found the sensors and reported data, whether powered by USB or by partially discharged batteries. The other console frequently lost connection with the sensors when powered by batteries, but less frequently when powered by USB.

 

Automatic Power-Cycling for USB Lockups

The automatic power-cycling feature is now fully functional in the weewx fine offset driver. It is included in the 2.6.4 release of weewx. If weewx loses communication with the station, it will attempt to power cycle the station until communication is re-established. This feature works only with hubs that support per-port power switching, such as the linksys USB2HUB4.

 

WS28xx Console Battery

Pay attention to the battery indicator on the La Crosse WS28xx weather station. On 24 March the console battery indicator showed FAIL. Almost two weeks later, around 7 April, the console sensor readings started going bad. Replacing the console batteries on 9 April brought everything back to correct values. No restart of weewx was required - everything just worked.





The StdQC MinMax filter kep egregious temperatures out of the database, but obviously it could not catch all of the bogus sensor readings. The humidity sensor seems to be most sensitive to battery voltage - it was the first sensor to report bogus values.

The console batteries seem to last about 9 months.

 

Rain Sensors Compared

Here is a preliminary comparison of sensors. In this case the data are from the rain sensors on a FineOffset WS2080, LaCrosse WS2317, and Meade TE923W. The data are from three different winter storms, one the night of 6 January 2014, another on 11 January 2014, and the other the night of 14 January 2014. Units are inches.




The WS2080 regularly under-reports rainfall. In each of these storms there was relatively little wind.

 

icons for the forecasting module

The exfoliation skin now includes only the aw icon set. The wu icon set is not yet fully populated. There is a sampler that shows all of the icons.

 

OWFS and WeeWX

Here are two different approaches to integrating data from one-wire devices with weewx. The first approach is to use a driver owfs.py, the second approach is to use a service owfss.py.

The driver approach works well if the one-wire bus is the core of the system. The polling interval determines how often to sample the devices.

The service approach works well if there are one or two devices that augment a weather station, and whose output fits into the default weewx database schema. The archive interval determines how often the devices will be sampled, although it is possible to use the loop event to collect data more frequently.

Adventurous souls might want to try multiple weewx instances writing to a single database.

It's nice to have options...

 

Uploader for wetter.com

wetter.py is an extension to weewx that will upload data to wetter.com.

 

WeeWX driver for OWFS

owfs.py is a weewx driver for sensors connected via one-wire file system (owfs). Since there are so many different types of one-wire devices and many, many ways of configuring them, the driver is pretty basic. Instructions are in the comments of the driver itself.

 

Comparing Forecasts

For quite some time now I have wanted to compare forecasts. Sometimes I would like to know how good a single forecasting algorithm is. For this we need a sliding window of the forecast over time, compared against actual values. Other times I would like to compare one forecasting algorithm with another. The comparison of algorithms is possible in the table view in the forecast.html page of exfoliation. Now there is a ForecastAnalysis service for weewx that provides a sliding window for a single algorithm.

This is an example of actual data (purple) plotted against a Weather Underground forecast for temperature, updated every 3 hours. In this example, uncertainty in a forecast shows up as variance in predicted values. Take a look at the temperatures predicted for the 19th.

 

Forecast Template

forecast-for-weewx is a package that includes icons and a template file. This should make it easier to display forecast data if you do not want to bother with the exfoliation skin. Installation instructions are in the readme.txt.

 

Another sticky bearing

There are now two data points that say 2 years is the point at which the bearing in the Fine Offset anemometer starts to get sticky. I started seeing no wind whenever the wind would drop below about 8 knots. Remove the anemometer cups, apply a few drops of BoeShield T-9 to the bearing, press the cup assembly back on, and all is well again.

 

Computer Monitor for weewx

cmon is a weewx service that tracks CPU load, memory use, disk use, and network traffic. The results can easily be displayed using the plotting capabilities of weewx.





Installation instructions are in the comments at the beginning of the file.

Why not use a proper monitoring tool? Usually I use nagios to monitor the computers I manage. However, sometimes the overhead of nagios is not warranted, or the computer I want to monitor is not accessible to the nagios server.

 

How to improve rain readings

If you have one of the older Fine Offset rain buckets, you can improve the accuracy of the rain readings (especially in windy conditions/locations) by adding a wall around the bucket. In this case we simply used aluminum tape on the collector itself.

 

How to increase the WS2080 range

The antenna in the WS2080 console is just a wire. Drill a hole in the back panel of the console, mount an antenna with 5-minute epoxy, solder the wire to the antenna, and you can increase the range between console and sensor cluster. Are there any RF gurus out there who can say what shape/length of antenna is best for 433MHz?

 

Inside the Anemometer

The reason for the flaky anemometer behavior turned out to be bad bearings. The Fine Offset stations use a cheap bearing in the anemometer cup. After two years in salty sea air, the bearings started to sieze. At first it was intermittent and not noticeable. Eventually a 10-15 knot wind was required to move the cups. A shot of Boeshield T9 and some manipulation freed things up. The cups come off simply by prying them from the anemometer base - it is a press fit onto the bearing.

Newer Fine Offset models include a slightly modified anemometer, as shown below.


Old Design
2012

New Design
2013

 

WS2080A behavior

The first thing I noticed about the WS2080A is that it is much better at maintaining communication with the thermo/hygro sensor unit. The test installation is a sensor mounted about 250 feet away from the console, with a 2 foot thick granite wall partially obstructing line-of-sight and an ASUS RT-N16 router with 3 9dB antennas beneath the console. The WS2080 would drop the connection every two or three days and reconnect on its own only after a day or two. The WS2080A is having no issues so far.

Based on the documentation, it would appear that the WS2080A introduces the ability to calibrate temperature, humidity, and wind.

Here is a comparison of the console configuration as reported by wee_config_fousb:

WS2080A (aug 2013)WS2080 (sep 2011)
    abs_pressure: 1008.1
     current_pos: 1744
    data_changed: 0
      data_count: 94
       date_time: 2000-00-00 00:00
   hum_in_offset: 0
  hum_out_offset: 0
              id: 26600
   lux_wm2_coeff: 0
         magic_1: 0x55
         magic_2: 0xaa
           model: 4224
       rain_coef: 24580
     read_period: 5
    rel_pressure: 1009.7
  temp_in_offset: 0
 temp_out_offset: 100
        timezone: -5
      unknown_01: 0
      unknown_18: 202
         version: 32
       wind_coef: 2785
       wind_mult: 0
    abs_pressure: 1007.7
     current_pos: 9120
    data_changed: 0
      data_count: 555
       date_time: 2009-01-03 10:32
   hum_in_offset: 12816
  hum_out_offset: 769
              id: None
   lux_wm2_coeff: 0
         magic_1: 0x55
         magic_2: 0xaa
           model: None
       rain_coef: None
     read_period: 5
    rel_pressure: 1010.0
  temp_in_offset: 2305
 temp_out_offset: 0
        timezone: -5
      unknown_01: 0
      unknown_18: 0
         version: 255
       wind_coef: None
       wind_mult: 0
 

Flaky Anemometer

The anemometer stopped reporting for awhile. The failure started after we got about 0.25 inches of rain. During a calm in the storm, I tried replacing the batteries in the thermo/hygro unit. No luck. I tried unplugging the cables, cleaning the contacts, then reconnecting. No joy. I tried disassembling the anemometer itself to inspect the reed switch and contacts. Nothing amiss. I tried checking continuity across the reed switch. No shorts. I put everything back together, and still no wind speed reading. We got another inch or so of rain, then a few hours after the low pressure passed and things had dried out a bit, the wind speed started reporting again.

 

Support for weewx forecasting module

The exfoliation skin now supports weather, tide, and almanac forecasts as implemented in the weewx forecasting module. Many thanks to Adam Whitcroft for the climacon icon set, from which these icons are derived.

 

WS2080A is now available

Ambient Weather is now shipping the WS-2080A console instead of the WS-2080 console.

Base on comparison of the WS-2080 and WS-2080A manuals, it looks like the WS-2080A adds a calibration mode for temperature, humidity, and wind speed. The specifications are otherwise identical, and the enclosure has not changed.

 

Dead Pressure Sensor

After 1 year and 8 months of continuous operation, the pressure sensor died on one of the WS2080 consoles. Now the console always returns None for the pressure.

Apparently this is wreaking havoc on the rain counter as well. Each time there is a bucket tip, the value decrements instead of incrementing. Of course the fousb driver treats this as a counter wraparound, so no rainfall is detected.

 

weewx uploaders for COSM, EmonCMS, SEG

It was trivial to create uploaders for weewx to send weather data. Drop these in the weewx user directory then add a couple of configuration options to weewx.conf. See comments in each file for details.

 

LaCrosse WS-28xx now working with weewx

The LaCrosse WS-28xx driver is now working with weewx, but only just. There is still a bit to do before it is ready for general consumption. The driver is based on the work done by Eddi De Pieri. Eddi did all of the hard work - I am simply consolidating code and removing all of the Windows/MS coding patterns and baggage. After about 8 hours of work, the code is pre-alpha right now. The biggest problem is that the driver does not start delivering data until the hour, then it sometimes returns the same observations until the next hour. FWIW, this behavior was noted by me and others when using Eddi's ws28xx driver with wfrog.

 

Popup graphs in exfoliation for weewx

The skin exfoliation-for-weewx now includes popup graphs on the current conditions page. Mousover the stats to see the 24-hour graph for that observation. Now almost everything is available in one compact view, with at most two clicks to drill down to more detail. Still need to figure out how to get page sizing right for consistent display across various small-screen devices.

 

Direct Communication with FineOffset Sensors

As of August 2012, Kevin Sangeelee has a Raspberry Pi communicating directly with a FineOffset sensor cluster. He did it with the RFM01 tranceiver (about $7.00 from SparkFun, Hope, or Maplin) and a Raspberry Pi. Very promosing is Kevin's interpretation of the data:

Byte     0  1  2  3  4  5  6  7  8  9
Nibble  ab cd ef gh ij kl mn op qr st
Example a1 82 0e 5d 02 04 00 4e 06 86

  • abc: device identifier
  • def: temperature
  • gh: humidity
  • ij: average wind speed low byte
  • kl: gust wind speed low byte
  • m: unknown
  • n: rainfall counter high nibble
  • op: rainfall counter
  • q: battery-low indicator
  • r: wind direction
  • st: checksum

Kevin has a blog entry about it:
http://www.susa.net/wordpress/2012/08/raspberry-pi-reading-wh1081-weather-sensors-using-an-rfm01-and-rfm12b/
as well as a 0.3 release of c++ code:
http://www.susa.net/wordpress/wp-content/uploads/2012/08/wh1080_rf_v0.3.tgz

Now if only I can get a USB stick with RFM transceiver working...

http://shop.moderndevice.com/products/jeelink-module-fully-assembled

 

New plot types for weewx

Three new plot types for weewx: rose, rose_histogram, and flags. Hopefully these will get into the weewx codebase after the 2.2 release.




 

Adaptive Polling for the WS2080

The fousb driver for weewx now includes an option for adaptive polling. This approach uses Jim Easterbrook's pywws implementation for avoiding communication with the console while the console is reading from the sensor or writing to memory.

So far I have had mixed results. The adaptive polling approach makes a lot of USB activity once per day as it attempts to figure out the console and sensor timing. In at least one case this activity caused the console to go into USB lockup. On the other hand, once the timing is determined the adaptive polling seems to work very well.

This is what the output should look like:

Feb 10 21:11:25 saga weewx[7919]: fousb: avoid 5.87254691124
Feb 10 21:11:31 saga weewx[7919]: fousb: live_data new data
Feb 10 21:12:13 saga weewx[7919]: fousb: avoid 5.8712220192
Feb 10 21:12:19 saga weewx[7919]: fousb: live_data new data
Feb 10 21:13:01 saga weewx[7919]: fousb: avoid 5.90185809135
Feb 10 21:13:07 saga weewx[7919]: fousb: live_data new data
Feb 10 21:13:49 saga weewx[7919]: fousb: avoid 5.86853194237
Feb 10 21:13:55 saga weewx[7919]: fousb: live_data new data

When things get a little bit out of sync this shows up in the log:

Feb 10 21:56:12 saga weewx[9957]: fousb: avoid 5.97709417343
Feb 10 21:56:18 saga weewx[9957]: fousb: avoid 0.772798061371
Feb 10 21:56:18 saga weewx[9957]: fousb: live_data missed
Feb 10 21:56:18 saga weewx[9957]: fousb: live_data new data
Feb 10 21:57:01 saga weewx[9957]: fousb: avoid 5.87521791458
Feb 10 21:57:06 saga weewx[9957]: fousb: live_data new data
Feb 10 21:57:49 saga weewx[9957]: fousb: avoid 5.87381887436
Feb 10 21:58:12 saga weewx[9957]: fousb: avoid 5.69364500046
Feb 10 21:58:37 saga weewx[9957]: fousb: avoid 5.49648594856
Feb 10 21:58:42 saga weewx[9957]: fousb: live_data missed
Feb 10 21:59:12 saga weewx[9957]: fousb: avoid 5.96931314468
Feb 10 21:59:25 saga weewx[9957]: fousb: avoid 5.80708098412
Feb 10 21:59:30 saga weewx[9957]: fousb: live_data missed
Feb 10 21:59:30 saga weewx[9957]: fousb: live_data new data

This is what the log sometimes looks like. Weewx is synchronized with the console for awhile, then indicates loss of log sync, but then re-synchronizes. The console provides data, but it looks like the adaptive polling is struggling to find out when it should be polling.

Feb 10 22:51:24 saga weewx[11943]: fousb: avoid 5.87635016441
Feb 10 22:51:30 saga weewx[11943]: fousb: live_data new data
Feb 10 22:52:12 saga weewx[11943]: fousb: avoid 5.87507414818
Feb 10 22:52:18 saga weewx[11943]: fousb: live_data new data
Feb 10 22:53:00 saga weewx[11943]: fousb: avoid 5.91367316246
Feb 10 22:53:06 saga weewx[11943]: fousb: live_data new data
Feb 10 22:53:48 saga weewx[11943]: fousb: avoid 5.91234207153
Feb 10 22:53:54 saga weewx[11943]: fousb: live_data new data
Feb 10 22:54:15 saga weewx[11943]: fousb: live_data new ptr: 000170
Feb 10 22:54:15 saga weewx[11943]: fousb: setting station clock 15.3583
Feb 10 22:54:15 saga weewx[11943]: fousb: live_data log synchronised
Feb 10 22:54:36 saga weewx[11943]: fousb: live_data new ptr: 000170
Feb 10 22:54:36 saga weewx[11943]: fousb: live_data lost log sync -1778.96
Feb 10 22:54:36 saga weewx[11943]: fousb: avoid 5.73438715935
Feb 10 22:54:42 saga weewx[11943]: fousb: live_data new ptr: 000170
Feb 10 22:54:43 saga weewx[11943]: fousb: live_data new ptr: 000170
Feb 10 22:54:43 saga weewx[11943]: fousb: setting station clock 43.6311
Feb 10 22:54:43 saga weewx[11943]: fousb: live_data log synchronised

This is what the log looks like when the adaptive approach really struggles. The console still responds and provides data.

Feb 10 18:36:24 saga weewx[2060]: fousb: avoid 2.41176891327
Feb 10 18:36:26 saga weewx[2060]: fousb: live_data new ptr: 00b6f0
Feb 10 18:36:26 saga weewx[2060]: fousb: live_data lost log sync -56.8319
Feb 10 18:36:27 saga weewx[2060]: fousb: unstable read: blocks differ for ptr 0x00b6e0
Feb 10 18:36:27 saga weewx[2060]: fousb: unstable read: blocks differ for ptr 0x000000
Feb 10 18:36:27 saga weewx[2060]: fousb: live_data new ptr: 00b700
Feb 10 18:36:27 saga weewx[2060]: fousb: live_data unexpected ptr change 00b6e0 -> 00b700
Feb 10 18:36:28 saga weewx[2060]: fousb: live_data new data
Feb 10 18:36:28 saga weewx[2060]: fousb: setting sensor clock 28.2048
Feb 10 18:36:28 saga weewx[2060]: fousb: avoid 2.49223780632
Feb 10 18:37:03 saga weewx[2060]: fousb: live_data new data
Feb 10 18:37:03 saga weewx[2060]: fousb: setting sensor clock 15.9977
Feb 10 18:37:04 saga weewx[2060]: fousb: avoid 2.4927740097
Feb 10 18:37:27 saga weewx[2060]: fousb: unstable read: blocks differ for ptr 0x00b700
Feb 10 18:37:27 saga weewx[2060]: fousb: live_data new ptr: 00b710
Feb 10 18:37:27 saga weewx[2060]: fousb: setting station clock 27.6538
Feb 10 18:37:27 saga weewx[2060]: fousb: live_data log synchronised
Feb 10 18:37:48 saga weewx[2060]: fousb: live_data new ptr: 00b710
Feb 10 18:37:48 saga weewx[2060]: fousb: live_data lost log sync -38.9755
Feb 10 18:37:49 saga weewx[2060]: fousb: avoid 5.73746395111
Feb 10 18:37:55 saga weewx[2060]: fousb: live_data new ptr: 00b710
Feb 10 18:37:55 saga weewx[2060]: fousb: live_data new ptr: 00b710
Feb 10 18:37:55 saga weewx[2060]: fousb: setting station clock 55.9105
Feb 10 18:37:55 saga weewx[2060]: fousb: live_data log synchronised

This is output when the console gets into bad magic number mode. The console is responsive, but no data can be read since 0x55 0xAA is expected. Power cycling the console fixes it.

Feb 10 19:19:27 saga weewx[3968]: fousb: unstable read: blocks differ for ptr 0x000000
Feb 10 19:19:27 saga weewx[3968]: fousb: unstable read: blocks differ for ptr 0x000000
Feb 10 19:19:27 saga weewx[3968]: fousb: unrecognised magic number 53 aa
Feb 10 19:19:27 saga weewx[3968]: fousb: live_data new ptr: 00b9b0
Feb 10 19:19:27 saga weewx[3968]: fousb: unstable read: blocks differ for ptr 0x00b9a0
Feb 10 19:19:27 saga weewx[3968]: fousb: unstable read: blocks differ for ptr 0x00b9a0
Feb 10 19:19:27 saga weewx[3968]: fousb: unstable read: blocks differ for ptr 0x00b9a0
Feb 10 19:19:28 saga weewx[3968]: fousb: unrecognised magic number 53 aa
Feb 10 19:19:28 saga weewx[3968]: fousb: live_data new ptr: 00b9b0
Feb 10 19:19:29 saga weewx[3968]: fousb: unrecognised magic number 53 aa
Feb 10 19:19:29 saga weewx[3968]: fousb: live_data new ptr: 00b9b0
Feb 10 19:19:29 saga weewx[3968]: fousb: setting station clock 29.0028
Feb 10 19:19:29 saga weewx[3968]: fousb: live_data log synchronised

On the bright side, weewx makes these diagnoses much easier to deal with. For example, one can unplug the console, power cycle it, then plug it back in without restarting weewx. weewx will timeout, retry, then happily reconnect and continue to collect data.

 

Icons for weewx

Here are favicon and logo images for weewx.


weewx.ico

weewx logo
 

amphibian for weewx

Here is a weewx skin called 'amphibian' that makes weewx look similar to wfrog.


wfrog 0.7

amphibian-for-weewx

wfrog uses a lot of javascript, and the wfrog charts require a live internet connect to google, so an exact match is not possible. On the other hand, weewx does some things that wfrog cannot.

amphibian-for-weewx-0.1.tgz

 

exfoliation for weewx 0.7

This update includes the following:

  • embedded NWS extended forecast
  • eliminate stale noaa links
  • make forecast links consistent for easier use in non-us locales
  • added tides and 7-day forecast link to index page

 

exfoliation for weewx 0.5

This update includes the following:

  • barometer trend indicator on current conditions page
  • support for day/night bands
  • support for gap detection
  • minor layout adjustments
  • added a whole slew of forecast links
Each of the links on the forecast page is defined in the skin.conf file, so they can be enabled/disabled by override in weewx.conf, modification of skin.conf, or direct edit of forecast.html.tmpl.

 

Fun with Flags

NOAA 48-hour forecasts include a wind strip chart that has wind magnitude, gust, plus flags to indicate direction. I tried whipping up a flag chart type for weewx, and this is the result.


NOAA 48-hour forecast

weewx flag prototype
 

Day/Night bands for weewx

A bit of mucking about in the weewx code resulted in vertical bands to highlight day and night in time series charts.


day monochrome

week blue/blue

week blue/yellow
 

LaCrosse C86234 and weewx

A few hours of quality time with a LaCrosse C86234 were promising, but not enough time to make code stable enough for release.

The LaCross C86234 appears to be the same as the WS-28XX series. The C86234 has wireless and batteries all over the place. The rain sensor is standalone, with its own batteries. The wind sensor is standalone, with its own batteries and a solar panel to charge the batteries. The temperature/humidity sensor has its own batteries. The rain and wind sensors communicate to the temperature/humidity sensor, then the temperature/humidity sensor communicates with the console. The console (also has its own batteries) communicates with a USB wireless dongle using a proprietary protocol. Luckily the USB wireless dongle shows up as a standard USB device and requires no special magic for communication.

Unfortunately, pairing the console to the USB dongle (and pairing each of the other devices to each other) can be rather flaky. On the plus side, they seem to be self-healing - if they lose connection with each other, they retry and often (but not always) re-establish communication.

I managed to get a plug computer running debian to talk to the console, but not reliably enough to build a complete driver for weewx for this device. It is probably a 2 or 3 day project to get everything working, but all of the wireless links (and especially the flakiness of console-to-computer) make me hesitate to do the implementation.

 

new rain collector design from Fine Offset

Apparently Fine Offset has been listening to complaints about incorrect rain readings. The 2012 stations include a rain bucket with a slight change from previous models - a taller wall has been added to the top of the collector, and the drain hole is slightly smaller.

The Fine Offset rain collector reported bad readings in the following conditions:

  • wind - in windy conditions rain would blow out of the collector, resulting in under reporting
  • heavy rain - in heavy rain, water would overflow the collector, resulting in under reporting

Details and pictures are available here: http://fineoffset.com/rain_gauge_new.htm

 

Guide for weewx on plug computer with Fine Offset station

This guide is a recipe for installing weewx on a plug computer. It shows how to use a ramdisk to reduce the number of writes to flash disk.

 

exfoliation for weewx 0.4

This update includes the following:

  • include NOAA and RSS templates (same as Standard skin)
  • added station subtitle
The station subtitle makes it easier to distinguish consoles in a multi-console configuration.

 

exfoliation for weewx 0.3

This update includes the following:

  • better favicon
  • disable underlines on datum links in main table
  • improved layout of sun/moon data
  • use dash rather than 'N/A' for no data

 

config_fousb.py

config_fousb.py is now available in the weewx source tree. This simple script provides read-only (for now) access to fixed-block data in Fine Offset stations. The fixed-block data are based on the description in the document 'ws-1080-1090-2080-cal.doc' from Ambient weather (or perhaps Fine Offset?).

There are a few fixed block fields that would be very nice to have, namely model, version, and id. Unfortunately these values are garbage, at least in the WS-2080 on which I am testing. config_fousb tries to read these fields, as well as the fixed block magic numbers.

The latest pywws release (12.10_r547) was extremely helpful in the process.

http://code.google.com/p/pywws/

The latest fowsr release (1.0-20110904.tar.gz) includes similar capability of reading the fixed block. It looks like the fowsr code was based on pywws.

http://code.google.com/p/fowsr/
http://code.google.com/p/fowsr/wiki/FineOffsetWeatherStationReader

 

fousb.py update to r329

This update to fousb.py addresses the following issues:

  • fixed rain calibration (weewx expects cm, station provides mm)
  • used a value of 2mm instead of 10mm for 'sane rain reading' to avoid counter bounce
  • added more checks and handling of bogus readings from station
  • do None checks in a more pythonic way
  • minor adjustments to logging - info vs debug
These issues are still outstanding:
  • station reports absolute pressure but this is recorded as barometric pressure in weewx

 

exfoliation for weewx

Exfoliation for weewx is a simple skin for the weewx weather system. It includes pages for current conditions, day/week/month/year summaries, almanac page, and forecast.

To install, simply expand the exfoliation-for-weewx tarball into the skins directory, then modify the weewx.conf file like this:

[StdReport]
    ...
    [[StandardReport]]
        skin = exfoliation
If you would prefer to leave the standard skin in place, just add exfoliation as an additional report like this:
[StdReport]
    ...
    [[ExfoliationReport]]
        skin = exfoliation
        HTML_ROOT = public_html/exfoliation
        [[[Extras]]]
            radar_img = http://radar.weather.gov/ridge/lite/N0R/BOX_loop.gif
            radar_url = http://radar.weather.gov/ridge/radar.php?product=NCR&rid=BOX&loop=yes
Unless you live in the Boston area you should change BOX to the appropriate code for your region, or comment the radar_img line.

Outstanding issues:

  • templates have a hard-coded path in them for including the .inc files
  • forecast page is not yet implemented
  • the favicon is just a placeholder for now

 

fousb.py update to r318

This update to fousb.py addresses the following issues:

  • fix wind speed calibration (weewx expects km/h, station provides m/s)
  • added retries and logging for bogus block pointer reads
These issues are still outstanding:
  • station reports absolute pressure but this is recorded as barometric pressure in weewx

 

fousb.py update to r311

This update to fousb.py addresses the following issues:

  • retry when the reads of the current block address fail
  • better handling of reads with bad block size
These issues are still outstanding:
  • wind speed units are incorrect - station reports in metric but weewx interprets as US with incorrect conversion
  • station reports absolute pressure but this is recorded as barometric pressure in weewx

 

fousb.py is working

The FineOffset-USB module for weewx has been running for a few days now and seems to be ok. weewx should make it much easier to sort out the bad magic number issues and USB lockups.

Copy the module to the weewx installation:

cp fousb-r308.py bin/weewx/fousb.py
Add a section in the weewx configuration file weewx.conf:
[Station]
    ...
    station_type = FineOffsetUSB
[FineOffsetUSB]
    # The station model, e.g. WH1080, WS2080, WH3081
    model = 'WS2080'
    # The python driver code for FineOffset stations
    driver = weewx.fousb

The station model is only for display purposes. FineOffset stations have one of two data types - the smaller '1080' data format, or the expanded '3080' data format. The fousb driver will automatically figure out which data type to use. But you can specify the model name so that the weewx display will report the actual station you are using. Unfortunately there is no way to automatically determine the station model, and there are many repackaged/relabeled versions out there.

 

FineOffset module for weewx

I did a quick merge of pywws and weewx and managed to get weewx to work with a FineOffset WS2080. Unfortunately there are still many rough edges. Rain calculations are way off, wind direction needs some work, and windchill, heat index, and dewpoint are not calculated correctly. The initial implementation does everything as live readings - it ignores the fact that the WS2080 logs data.

Unfortunately there are some random data corruption issues that seem to be USB-related. But with weewx I should be able to track them down and possibly implement workarounds...

It took about 5 Doctor Who episodes to hack the initial implementation.

Someone should hack/reverse engineer the wireless protocol for the Fine Offset stations so that a computer with wireless hardware can speak directly with the Fine Offset sensor bundle and avoid all of the USB flakiness of the Fine Offset consoles.

 

weewx

weewx is a python tool patterned loosely on wview but without all of the cruft. So far only the Davis Vantage and the Oregon Scientific WMR are supported, but I think we can fix that...

 

Still no Radio Controlled Clock, still USB Problems

The WS2080 still does not get a lock on the radio controlled clock. I suspect hardware issues, as this is noted as a reason to return the unit at the ambient wiki.

I still see periodic problems with the USB connection - the computer is unable to read data, and the only way to fix the problem is to power cycle the WS2080 console. That means removing the batteries and unplugging the USB cable. As a result, the WS2080 is useless for locations where it must go unattended for months at a time.

I could make a daemon to reboot the WS2080 when wview cannot communicate with it, but there is no way to programmatically power cycle the WS2080.

And FineOffset/Ambient still do not offer any firmware updates to fix the USB problems.

 

Missing Data

It took me awhile to notice this, but when the WS2080 console loses the connection to the sensor unit, wview will not parse the output. The console data are still just fine, e.g. for interior temperature and humidity. The result is gaps in the wview data, even though the console is working just fine.

One of our consoles is about 200 feet from the sensor unit, with the corner of a stone foundation and single pane glass window between it and the sensor unit. Most of the time it gets the sensor signal just fine. But on really foggy and windy (?) days the connection drops. A lot.

Someday I'll have to figure out the difference between a "we have a connection to the sensors" output and "no sensor data" output, then submit a patch for wview.

 

Install 5.20.2 on Ionics Plugs (Nimbus and Stratus)

Did a couple of installations on some ionics devices. Here is the process:

# install pre-requisites
apt-get install libsqlite3-dev libgd2-noxpm-dev libcurl4-openssl-dev \
  mysql-client libusb-1.0-0-dev gawk php5-sqlite

# build and install radlib
tar xvfz radlib-2.12.0.tar.gz
cd radlib-2.12.0
./configure --prefix=/opt/radlib-2.12.0 --enable-sqlite
make
make install

# build and install wview
tar xvfz wview-5.20.2.tar.gz
cd wview 5.20.2
CFLAGS=-I/opt/radlib-2.12.0/include LDFLAGS=-L/opt/radlib-2.12.0/lib ./configure --prefix=/opt/wview-5.20.2
make
make install

# fix file locations (prolly could do this in configure)
cd /opt/wview-5.20.2/var
mv wview wviewmgmt /var
ln -s /var/wview .
ln -s /var/wviewmgmt .

# adjust the configuration and skin
/opt/wview-5.20.2/bin/wviewconfig
/opt/wview-5.20.2/bin/wviewhtmlconfig

# modify startup script
cp examples/Debian/wview /etc/init.d/wview
emacs /etc/init.d/wview

# give the apache user permission to do stuff
adduser www-data sudo
visudo

# modify the apache configuration
emacs /etc/apache2/conf.d/wview.conf

# configure logrotate
emacs /etc/logrotate.d/wview

# configure syslog
emacs /etc/rsyslog.d/wview.conf

# configure ramdisk for logs and images
emacs /etc/fstab
mount -a

# start the new wview
/etc/init.d/wview start

Here is the snippet for /etc/init.d/wview

export LD_LIBRARY_PATH=/opt/radlib-2.12.0/lib:/opt/wview-5.20.2/lib:/usr/local/lib:/usr/lib
CONF_DIRECTORY=/opt/wview-5.20.2/etc/wview
RUN_DIRECTORY=/opt/wview-5.20.2/var/wview
WVIEW_INSTALL_DIR=/opt/wview-5.20.2/bin
RADLIB_BIN_DIR=/opt/radlib-2.12.0/bin

Here is the apache configuration /etc/apache2/conf.d/wview.conf:

Alias /weather /var/wview/img
<Directory /var/wview/img%gt;
  Options FollowSymlinks
  AllowOverride None
  Order allow,deny
  Allow from all
</Directory%gt;

Alias /wviewmgmt /var/wviewmgmt
<Directory /var/wviewmgmt%gt;
  Options FollowSymlinks
  AllowOverride None
  Order allow,deny
  Allow from all
  DirectoryIndex login.php
</Directory%gt;

Here is the syslog file /etc/rsyslog.d/wview.conf:

:programname,isequal,"radmrouted" /var/log/wview.log
:programname,isequal,"radmrouted" ~
:programname,isequal,"htmlgend" /var/log/wview.log
:programname,isequal,"htmlgend" ~
:programname,startswith,"wv" /var/log/wview.log
:programname,startswith,"wv" ~ 

Here is the logrotate file /etc/logrotate.d/wview:

/var/log/wview.log {
  weekly
  missingok
  rotate 52
  compress
  delaycompress
  notifempty
  create 644 root adm
}

Here is the fstab file /etc/fstab

wviewimg /var/wview/img tmpfs size=10M,noexec,nosuid,nodev 0 0
wviewlog /var/wview/log tmpfs size=10M,noexec,nosuid,nodev 0 0

 

Update to 5.20.2

Finally got around to doing an update to the wview software from 5.19.0 to 5.20.2. Did the standard update process:

# build and install radlib
tar xvfz radlib-5.12.0.tar.gz
cd radlib-5.12.0
./configure --prefix=/opt/radlib-5.12.0
make
make install

# build and install wview
tar xvfz wview-5.20.2.tar.gz
cd wview 5.20.2
CFLAGS=-I/opt/radlib-2.12.0/include LDFLAGS=-L/opt/radlib-2.12.0/lib ./configure --prefix=/opt/wview-5.20.2
make
make install

# shutdown wview
/etc/init.d/wview stop

# backup data
cd /var
tar cvfz wview-yymmdd.tgz wview
tar cvfz wviewmgmt-yymmdd.tgz wviewmgmt

# point symlinks to data
cd /opt/wview-5.20.2/var
mv wview wview-orig
mv wviewmgmt wviewmgmt-orig
ln -s /var/wview .
ln -s /var/wviewmgmt . 

# adjust the skin
cp -p /opt/wview-5.19.0/etc/wview/skins /opt/wview-5.20.2/etc/wview
cp -p /opt/wview-5.19.0/etc/wview/wview-conf.sdb /opt/wview-5.20.2/etc/wview
/opt/wview-5.20.2/bin/wviewhtmlconfig

# modify startup script
emacs /etc/init.d/wview

# start the new wview
/etc/init.d/wview start

 

exfoliation updated to 0.7

This update to exfoliation for wview skin includes the following:

  • support for the 'plus' configuration, i.e. solar radiation, UV, and ET.
  • copy static elements in the post-generate.sh script to accommodate ramdisk configurations
  • improved layout for the phone page

 

Review of WS2080 for amazon.com

Here is the review for amazon of the WS2080.

 

The WS2080 RCC is rubbish

It has been about a week now and the time on the WS2080 is still not correct. Of course I could manually enter the date and time, but then what is the point of having a radio controlled clock? The manual says it could take up to a couple of days for the unit to lock in on the radio signal. But a week?

 

exfoliation for wview

First of all, kudos to Mark Teel for all the work he has put into wview. The underlying bits work really well, the system is easy to configure, and the architecture makes it easy to use as much or as little as you need.

Of course I could not leave the default wview look and feel untouched. wview 5.19.0 ships with two skins: chrome and classic. Both of them look very much like something from the mid-90s, and the html is horrendous, packed with font tags and random styles. I stripped the templates and created a new skin, called exfoliation, of course. It weighs in at about two-thirds the size of the chrome skin, it is far easier to modify, and it is easier on the eyes, IMHO.

What does it look like, you ask?

chrome classic exfoliation

To use exfoliation, do something like the following to an existing wview installation:

# stop the wview daemons
/etc/init.d/wview stop

# make a copy of the existing configuration
cd /etc/wview
tar cvfz html-yymmdd.tgz html *.conf *.sh

# extract the exfoliation bits and put them where they belong
cd /tmp
tar xvfz exfoliation-for-wview-x.y.tgz
cp exfoliation/static/* /var/wview/img
cp exfoliation/standard/*.htx /etc/wview/html
cp exfoliation/standard/*.incx /etc/wview/html
cp exfoliation/standard/*.sh /etc/wview
cp exfoliation/standard/*.conf /etc/wview

# start the wview daemons
/etc/init.d/wview start
Of course, if you have done any pre/post mods you'll have to put those into the appropriate generate scripts. There is no plus variant at this point.

The wview documentation says that you can simply restart the htmlgend daemon when you make changes to the conf files like this:

kill -s HUP `cat /var/run/htmlgend.pid`
In my experience this does not work - new templates get generated, but the images.conf changes are ignored. This always works:
/etc/init.d/wview restart

Beware the exfoliation skin is still pretty crude - lots of redundancy in terms of information content, not terribly space-efficient, and the implementation still uses tables for the layout rather than div elements. At least it should be easier to modify than chrome or classic.

 

check_wview

I created a Nagios plugin called check_wview that will check to see whether wview is working. It compares current time to the mtime of the wview database or the index.htm to determine whether wview data collection or html generation is ok, warning, or critical. It can report weather observations as performance data.

Run it via NRPE with a simple entry like this in /etc/nagios/nrpe_local.cfg:

command[check_wview]=/usr/lib/nagios/plugins/check_wview
Then something like this on the Nagios server:
define service {
  use lan-service,graphed-service
  host_name weather
  service_description weather
  display_name Weather
  check_command check_nrpe!check_wview
}

 

wview installation

I did a manual installation of wview that looked something like this:

# hardware: ionics nimbus 101 plug computer
# RAM: 512MB
# disk: 512MB
# cpu: 1.2GHz ARM
# OS: debian 6
# wview: 5.19.0
# radlib: 2.11.3

# install prerequisites
apt-get install build-essential
apt-get install zlib1g-dev libpng12-dev libreadline5-dev gawk
apt-get install libsqlite3-dev sqlite3
apt-get install libgd2-noxpm-dev libssl-dev
apt-get install libcurl4-openssl-dev
apt-get install libusb-1.0-0-dev
apt-get install apache2 php5 php5-sqlite

# build and install radlib
cd /home/build/radlib-2.11.3
./configure --prefix=/opt/radlib-2.11.3 --enable-sqlite
make
make install

# build and install wview
cd /home/build/wview-5.19.0
CFLAGS=-I/opt/radlib-2.11.3/include LDFLAGS=-L/opt/radlib-2.11.3/lib \
  ./configure --prefix=/opt/wview-5.19.0
make
make install

# configure wview
/opt/wview/bin/wviewconfig
/opt/wview/bin/wviewhtmlconfig

# configure rc stuff
cp examples/Debian/wview /etc/init.d
chmod 755 /etc/init.d/wview
update-rc.d wview defaults 99
postfix is already installed and configured, so no need for sendmail or sendemail. wview does not use much disk space, but the stuff required to build it does. Before the installation, disk use was at 59%. After the installation, disk use was at 76%. The disk size is 512MB.

Since this is a plug computer with flash, I configured a ramdisk for wview by putting this in /etc/fstab:

wviewimg /var/wview/img tmpfs size=10M,noexec,nosuid,nodev 0 0
The /var/wviewmgmt and /var/wview directories contain local state, but some of it is disposable and some is not. The img directory sees the most changes as files in that directory are modified by the html generator daemon. In my configuration, /opt/wview/var is a symlink to /var/wview. I should have used the localstatedir option in configure when buildling wview.

The other parts are pretty standard. Here is the apache conf snippet in /etc/apache2/conf.d/wview.conf:

Alias /weather /var/wview/img
<Directory /var/wview/img>
  Options FollowSymlinks
  AllowOverride None
  Order allow,deny
  Allow from all
</Directory>

Alias /wviewmgmt /var/wviewmgmt
<Directory /var/wviewmgmt>
  Options FollowSymlinks
  AllowOverride None
  Order allow,deny
  Allow from all
  DirectoryIndex login.php
</Directory>

Here is the logrotate snippet in /etc/logrotate.d/wview:

/var/log/wview.log {
  weekly
  missingok
  rotate 52
  compress
  delaycompress
  notifempty
  create 644 root adm
}

Here is the syslog snippet in /etc/rsyslog.d/wview.conf:

:programname,isequal,"radmrouted" /var/log/wview.log
:programname,isequal,"radmrouted" ~
:programname,isequal,"htmlgend" /var/log/wview.log
:programname,isequal,"htmlgend" ~
:programname,startswith,"wv" /var/log/wview.log
:programname,startswith,"wv" ~

Here are the logwatch snippets:

/etc/logwatch/conf/logfiles/wview.conf

LogFile = /var/log/wview.log
Archive = wview.?
Archive = wview.?.gz
*ApplyStdDate =

/etc/logwatch/conf/services/wview.conf

Title = "wview"
LogFile = wview

/etc/logwatch/scripts/services/wview

wview

 

Goodbye wfrog, Hello wview

I decided to install wview (again) to let it run awhile in an effort to diagnose the USB issues. wview has been running for a full day now, and guess what? No dropouts! I even had wfrog and wview running concurrently for awhile. wfrog would frequently stop collecting data, while wview chugged along without missing a sample. We'll see what happens when (if?) the WS2080 locks up the USB connection, but so far it looks like the collection algorithm/implementation in wview is much more robust than that of wfrog/pywws.

If only the look and feel of wview did not suck so much...

 

Two different problems?

There appear to be two different data collection failure behaviors: (1) WS2080 drops the USB connection so wflogger stops logging data, (2) wflogger stops logging data even though the USB connection is ok.

WS2080 USB dropout

When this happens, doing

python TestWeatherStation.py
fails with the message
Device or resource busy
There is nothing in the wflogger log file to indicate a problem, even when log level is set to debug. The only way to fix it is to power cycle the WS2080.

wflogger dropout

When this happens, the wflogger.log file contains a message from pywws about live_data being late:

2011-11-16 08:48:07,835 INFO [pywws.weather_station] live_data log 11.304s late
then there are no more messages from wflogger, even when log level is set to debug. Restarting the wflogger process fixes it.

This is for a WS2080 connected via USB to a Nimbus101 plug computer. Plug computer details:

  • Linux saga 2.6.34.9 #1 PREEMPT Thu Jun 23 10:15:49 PHT 2011 armv5tel GNU/Linux
  • Debian GNU/Linux 6.0
  • pywws-11.10_r429
  • wfrog-0.8.1 with modifications to csvfile.py, wh1080.py, aggregator.py
  • python-usb 0.4.22-2+b2
  • python-yaml 3.09-5
  • python-lxml 2.2.8-2
  • python-cheetah 2.4.2.1-1
  • python-pygooglechart 0.3.0-1
  • usbutils 0.87-5squeeze
  • libusb-dev 0.1.12-1
  • libusb-0.1-4 0.1.12-1

 

better comparison of weather stations

Here is a pretty comprehensive comparison of weather stations:

weather station comparison guide
It looks like the Fine Offset units (e.g. WH1080, WH1090, WH2080) are repackaged by many vendors, including ambient weather. Unfortunately there are no firmware updates at the Fine Offset website to address the USB dropouts (or any other issues, for that matter).

 

WS2080 clock takes forever to synchronize

Supposedly the WS2080 has a radio clock. The documentation says that it can take multiple days for the unit to get its time from the radio signal. This is probably because the radio clock signal broadcasts at 60Hz, so that means not many bits can be transmitted.

nist radio clock info
Apparently the synchronization algorithm in the WS2080 only tries to get a clock signal every two hours so that it does not interfere with the sensor radio signal.

 

WS2080 USB dropouts continue

The USB connection to the WS2080 drops out periodically. Sometimes it happens after a couple of hours of operation, sometimes after a couple of days. The only way to restore the connection is to remove the batteries from the WS2080 console, remove the USB cable (so the console does not get power from the computer), wait for a minute or so, then plug things back in.

A WS2080 may cost 1/4 of a Davis station, but if the WS2080 cannot be left to run on its own, that is not really money saved. If I cannot figure out a way to get the WS2080 to reset (reset the usb port? shut off power to the usb port?) then we might have to spring for a Davis. At the very least I'll have to install wview to see if the problems are specific to pywws and wfrog.

I tried doing a usb reset by sending USBDEVFS_RESET. No joy. The OS indicates that the reset was successful, but the device does not respond. It looks like only a power cycle will reset the WS2080. FWIW, doing lsusb shows the WS2080 as this device:

Bus 001 Device 004: ID 1941:8021 Dream Link WH1080 Weather Station / USB Missile Launcher
Use the idVendor and idProduct to match up the devices subdirectory with the lsusb output. In this case idVendor is 1941 and idProduct is 8021.

I tried suspending power to the usb port, but no joy:

echo suspend > /sys/bus/usb/devices/1-1/power/level
-bash: echo: write error: Invalid argument
The only values that work are 'on' or 'auto'. Trying to put anything else into level results in the Invalid argument.

Doing 'lsusb -v' shows that the root hub (the usb port on the nimbus101) is supposed to support power switching:

  wHubCharacteristic 0x0009
    Per-port power switching
    Per-port overcurrent protection
    TT think time 8 FS bits

I tried using hub-ctrl to power cycle the root hub. This caused the weather station to get a new usb device number, but it did not restore communication with the weather station.

 

check_wfrog

I created a Nagios plugin called check_wfrog that will check to see whether wfrog is working. It compares the current time to the timestamp of the last entry in the csv file to determine whether wfrog logging is ok, warning, or critical. It reports the values as performance data.

Run it via NRPE with a simple entry like this in /etc/nagios/nrpe_local.cfg:

command[check_wfrog]=/usr/lib/nagios/plugins/check_wfrog
Then something like this on the Nagios server:
define service {
  use lan-service,graphed-service
  host_name weather
  service_description weather
  display_name Weather
  check_command check_nrpe!check_wfrog
}

 

wflogger stops recording data

wflogger stopped recording data. The wflogger.log file did not indicate any problems. The wflog.csv file was not updated. The wflog-current.xml file was not updated. The wflogger process was still running. Doing a 'python TestWeatherStation.py' returned proper results.

 

Another WS2080 installation

Installed another WS2080 weather station. This one is paired with an Ionics Nimbus101 plug computer. The plan is to use this one for testing and development. I already modified the wfrog 0.8.1 release so that it will record interior temperature and humidity to the csv file, and so that the wind direction and magnitude are recorded properly. Now I need to make it graph the interior data.

My modified wfrog files are here (changes relative to a generic 0.8.1 installation):

csvfile.py
wh1080.py
aggregator.py
These changes fix the units/scaling problems (a bit more robustly than what is in the wfrog repository as of 12nov2011) and record the interior temperature and humidity to the csv data storage. The data storage changes are a short-term hack so that I can record data. I have not analyzed the wfrog system to make these changes general enough for database or other storage mechanisms, or even to understand what other parts of wfrog will be affected by the csv changes.

 

WS2080 is not good for unattended use

Around 00:30 on 09 November, wflogger stopped updating the current weather conditions and the history. It looks like the WS2080 has borked the usb connection again. Unfortunately there is no one nearby to unplug, turn off the WS2080, then replug. And no one will be able to until April. sigh.

When I try to test the connection using

python TestWeatherStation.py
pywws gives errors of
usb.USBError: could not claim interface 0: Device or resource busy

The last time this happened, the only way to fix it was to turn off the WS2080, which meant disconnecting the USB cable and removing the batteries.

 

A week of unattended collection

In the past 11 days, the rain collector indicates no rain most days, but one day shows 75.0 mm and another shows 15.9 mm! We probably need a funnel around the collector otherwise the high winds will blow the rain away before it collects.

The wind direction was a bug in the wfrog data collector. See this thread:

http://code.google.com/p/wfrog/issues/detail?id=86

More discussion about WS2080 issues (including the 'None' values when radio link is lost and the poor design of the rain collector) are in this thread:

http://groups.google.com/group/wfrog-users/browse_frm/thread/bcbfc4101b14535a?pli=1

The interior temperature and humidity are, in fact, captured by wfrog, but they are not logged. wfrog displays only the latest values, not a history. We are quite interested in the interior conditions just as much as exterior to ensure that the solar heating and other systems are working. I had to hack aggregator.py and wh1080.py, but now we are recording interior temperature and humidity.

The ionics nimbus 100 plug works really well. It is running nagios, wfrog, its-electric, and apcupsd. It is running debian 6, which is what it came with.

The dreamplug is useless, and it is running only wfrog. After a few hours it stops responding to pings or ssh, but oddly enough any existing ssh connections continue to work. After a day or two wfrog stops collecting data. Most infuriating is that there is nothing in the system logs to provide any indication of why the machine locks up. We are using only the wireless interface of the dreamplug, and we are still running debian 5, which is what it came with. Both wired interfaces are disabled as is the bluetooth.

 

A few days with the data collector

When the console does not receive data from the weather station, wfrog simply craps out instead of providing useful diagnostics. I mistook this for a problem with the USB connection and wasted hours chasing the wrong problem.

Apparently wfrog does not know what to do with interior temperature/humidity data. Out-of-the-box it displays the exterior sensor data just fine, but it does not display the interior data.

It looks like wfrog displays the wind direction 180 degrees from where it should. If someones says "south west wind at 20 knots" that means the wind is coming from the south west at 20 knots. wfrog displays this as a north east wind. At the very least wfrog should have a preference for whether wind direction should be displayed as "from" or "to".

Unfortunately wfrog uses its own python-based web server. That might be ok for simplicity, but I'd like to manage the web pages using apache and emacs, especially when I already have an apache server running on the machine on which wfrog is running.

The WS2080 console is not terribly robust. After a week of running, it suddenly stopped communicating with the plug computer. A power cycle brought it back, but that meant losing data.

The WS2080 console sometimes wedges when it cannot communicate with the weather sensors. A power cycle brings it back.

Obviously power cycling to fix the console is not acceptable - there is usually no one around to flip the switch!

 

Data collection options

We considered the following open source solutions for collecting and displaying data:

systemnotes
wfrog
0.8.1
Nice to look at, and easy to customize. Quite a few dependencies, but they are all python modules and easy enough to install even on debian 5. The dependency on google charts is probably going to be a problem for us.
pywws
11.10_r429
This is a set of building blocks by Dave Wells with which you can collect data, build web pages, send data to a weather aggreator, etc. wfrog uses pywws for the USB connection to the WS2080.
wview
5.19.0
Fairly comprehensive system. The dependency on radlib is annoying. The information display is ugly, busy, and looks like something from 1995. The web-based configurator works well, but the underlying php is pretty rough. If you have to implement a configuration UI, why on earth would you copy the linksys UI?

Our biggest constraint is that we must run on a plug computer, so that means minimal CPU and memory as well as control of where/how the data are stored. Since the system uses flash memory for its disks, we need to save long-term data to a separate file system on a USB or SD device.

 

First impressions

The console display is reasonable, but not amazing. The backlighting is not evenly diffused. It would be much better if it changed based on ambient lighting, especially when the device is plugged in via USB (and thus drawing power from the USB). Each of the different data readings is displayed about the same size as the others. I would prefer having temperature and wind speed/direction more prominant and the other elements smaller. The 'forecast' is useless (it just tracks the pressure trend then displays a clound or sun icon) and occupies far too much space. The pressure history is annoying - it cycles through its display "to prevent screen burn-in" instead of showing a constant display, so one must wait until it repaints the history to see the history. It is the equivalent of having blinking text - not only is it annoying, but it actually makes the information harder to grok.

Like most of these devices, the information layout is lame. Each bit of data is given as much relevance (in terms of size and layout) as any other. But not each bit of data deserves as much relevance. For example, the trend icon is huge, but it is pretty much useless. The time and date are least important (we have watches and clocks for that), yet they are just as prominent as the weather data. The wind rose stands out, but the marker for actual wind direction is barely discernable. With just a little thought the display could be much better, i.e. it would be easier to digest in a quick glance and would require less learning every time one looks at it.

The console is configured by cycling through a 'menu' button and an 'enter' button, with up and down arrows to change values. I would prefer a separate button on the back of the device so that end-users do not inadvertantly (mis)configure the console.

Wireless range is marginal, and hopefully it will not degrade much as the lithium batteries in the temperature/humidity sensor start to fade. One console is 150 feet from the weather sensors, with a single brick wall between. The second console is 200 feet from the weather sensors, with 4 wood-and-sheetrock walls between. In this configuration the consoles only get about 50% of the samples they attempt (according to the wfrog logs).

The sensor package is made of cheap plastic. Time will tell how well it holds up to UV, ice, and extreme temperature changes.

The rain sensor is a self-bailing see-saw device in a plastic enclosure. Hopefully we do not end up with spider nests blocking the mechanism. There is no heater attachment for measuring snowfall. There is no baffle around the collection tray, so in windy conditions it will probably under-report the rainfall.

The temperature and humidity sensor package contains the wireless transmitter. This package is connected to the rainwater and wind sensors by two telephone wires - one is 2 conductor (rain sensor) and the other is 4 conductor (wind speed and direction). One could easily increase the distance between sensors by using standard phone wiring. We placed the wind sensors and rain sensor on the included mast at the end of the roof ridge on an out-building, then ran wires down under the eave to the temperature/humidity sensor under the eave on the northwest corner of the building.

The weather direction sensor dances a lot in both light and heavy air.

 

The configuration

This is the system we are working with:

  • Two WS2080 consoles
  • One WS2080 sensor station (wind speed, wind direction, rain gauge, temperature, humidity)
  • One Ionics nimbus 100 plug computer connected via USB to a WS2080 console
  • One DreamPlug plug computer connected via USB to a WS2080 console

 

What are the options for weather monitoring?

These are our requirements for weather monitoring:

  • Open data format and open interface for aquiring data.
  • The data will be aquired and stored on a linux-based server.
  • Robust startup/control - must start up automatically after power failures; must not require periodic restarts.
  • Must survive extreme temperature (-30 - 110 F), humidity (10-100%), wind (0-70 mph), ice, and salt spray.
  • Must support multiple consoles for a single weather station.
  • Wireless communication between weather station and consoles.
  • Distance of 200 feet between station and consoles, with some wood and brick obstructions.

These are the options we considered in the Summer of 2011.

systemnotes
ambient weather WS2080
$159.95

Features:

  • USB port for easy connection to your PC
  • All the weather data from the base station and weather history data with user adjustable measuring intervals can be recorded and uploaded to your PC
  • 4080 data points with adjustable (5 minute minimum) sample interval
  • Free PC software for transfer weather data to PC
  • Rainfall data (inches or millimeters): 1-hour, 24-hour,one week,one month and total since last reset.
  • Wind chill and Dew point temperature display (B0F or B0C)
  • Records min. and max. wind chill and Dew point with time and date stamp
  • Wind speed (mph, m/s, km/h, knots, Beaufort)
  • Wind direction display with LCD compass
  • Weather forecast tendency arrow
  • Weather alarm modes for: a) Temperature b) Humidity c) Wind chill d) Dew point e) Rainfall f) Wind speed g) Air pressure h) Storm warning
  • Forecast icons based on changing barometric pressure
  • Barometric pressure (inHg or hPa) with 0.1hPa resolution
  • Wireless outdoor and indoor humidity (% RH)
  • Records min. and max. humidity with time and date stamp
  • Wireless outdoor and indoor temperature (B0F for B0C)
  • Records min. and max. temperature with time and date stamp
  • Radio Controlled Clock with Time and date with manual setting
  • 12 or 24-hour time display
  • Perpetual calendar
  • Time zone setting
  • Time alarm
  • High light LED backlight
  • Wall hanging or free standing
  • Synchronized instant reception
  • Low power consumption (over 2 years battery life for transmitter)

Specifications:

  • Outdoor temperature range: -40B0F to +149B0F
  • Indoor temperature range: 32B0F to +122B0F
  • Humidity range: 10% to 99% (1% resolution)
  • Wind speed: 0-112 mph
  • Measuring range air pressure: 8.85inHg - 32.50inHg
  • Alarm duration : 120 sec
  • Transmission range up to 300 feet line of sight, 100 feet under most conditions
  • Power consumption: a) Receiver: 2 x AA alkaline batteries (not included) b) Sensor WH7: 2 x AA alkaline batteries (not included)
  • Transmission frequency: 433MHz
  • Console Dimensions (LxWxH): 4.5 x 6.75 x 1.5"

LaCrosse WS-2810U-IT
$249.99

Features:

  • Included PC Interface
  • Wind Chill, Direction, Gust and Speed
  • Solar Powered Wind Sensor
  • Rain Data
  • Forecast w/ Tendency
  • IN/OUT Temp
  • IN/OUT Humidity
  • Weather Alarms w/ Storm Warning

Davis
Vantage Pro2 6152
$535.00
USB WeatherLink 6510USB
$148.00

Features:

  • Outdoor Temperature
  • Outdoor Humidity
  • Wind Speed
  • Wind Direction
  • Rainfall (Precipitation)
  • Barometric Pressure
  • Indoor Temperature (console)
  • Indoor Humidity (console)