NetReceiver is a free cloud service for collecting and monitoring remote sensor data ("sensing") and controlling devices ("actuating"). NetReceiver clients are called NetSenders.

NetReceiver is based on the idea of keeping clients very simple and putting as much processing as possible into NetReceiver. Read about these principles here.

Read about the ESP8266 Arduino client here.

Objects

NetReceiver defines the following objects:

Protocol

NetReceiver clients communicate with the service by means of HTTP requests which return JSON data. Devices are identified by their unique MAC address. Devices in turn have one or more pins, which are either associated with sensors which convert physical measurements or actuators which perform physical actions.

NetReceiver JSON values are described here.

Pins

Despite the name, pins represent more than just physical device pins. In NetReceiver, pins represent different data types. Scalar pins represent non-negative integers, with -1 signifying an error. Scalar values are sent as URL encoded parameters in the HTTP request. Vector pins represent everything else, i.e., any value that cannot be represented as a single non-negative, number. Vector data is sent in the body of a HTTP POST request.

Pins may be further classified as real or virtual. Real pins, such as analog or digital (GPIO) pins, are attached to a physical device. Virtual pins, on the other hand, provide an additional layer of abstraction. For example, while a DHT22 combined temperature/humidity sensor is physically connected to a single GPIO pin, extracting useful data requires additional computation. In this case, two scalar "X" pins suffice to convey computed temperature and humidity values. A more complicated example is video where a single "V" pin represents one channel of video data.

Pin types

Name Type Description Example usage
A Scalar Analog value A0 for battery voltage
B Vector Binary data B0 for binary accelerometer data
D Scalar Digital value D0 for alarm power
S Vector Sound (audio or sonar) data S0 for first audio channel
T Vector Text (log or GPS) data T0 for log data
V Vector Video data V0 for first video channel
X Scalar Software-defined sensor value X10 for ESP8266 raw analog value

Pin ordering

By convention, pin numbers start from 0, not 1. For example, A0 is the first analog output (or only analog output in the case of the ESP8266), V0 is the first video channel, etc. Log data is typically reported as the first text output, namely T0, and GPS data, if present, is reported as T1.

Configuration

Clients obtain configuration data, i.e., the list of sensor and actuator pins and WiFi network info, from the service by means of config requests, which are sent at startup and subsequently in response to configuration changes. Clients bootstrap themselves by communicating initially via a WiFi Network named "netreceiver" with WPA password "netsender" (the default network). The clients identify themselves by means of their unique MAC address via the "ma" parameter. NetReceiver issues each client a device key, the "dk"parameter, which must be supplied with config requests. Requests for which the MAC address does not correspond to the device with the given device key are rejected. The exception is the initial config request when the client does not yet know its device key. Finally, clients may optionally supply their LAN IP address via the "la" parameter.

The format of config request is as follows: 

  /config?ma=01:02:03:04:05:06&dk=12345678&la=1.2.3.4
Whenever the user has made a configuration change, the service returns a response code "rc" of 1, for example: 
  {"ma": "01:02:03:04:05:06", "rc": 1}
Note that any response can include the "rc" parameter, not just config responses. The client then reissues a config request. Clients also issue config requests periodically if no input or output pins are currently defined.

Device settings

Devices are configured under Settings. The ESP8266 client is configured 100% this way, however the Raspberry Pi client requires some pre-configuration via /etc/netsender.conf. Specifically, the MAC address (ma) and device key (dk) must be manually specified in /etc/netsender.conf before the client can run as a NetSender client. Other parameters are updated automatically.

Parameter Description Example
di Device ID (when sent as a variable it is "id") "esp-10"
ma MAC address (required in netsender.conf for Pi) "0A:0B:0C:0D:0E:0F"
dk 8-digit device key (generated by NetReceiver; required in netsender.conf for Pi) 01234567
wi Comma-separated SSID and key "MySSID,MyKey"
ip Comma-separated input pin names "A0,D1,X22"
op Comma-separated output pin names "D2,D12,X10"
mp Monitor period in seconds 60
ap Active period in seconds (for ESP clients only)
tg Software version tag (a Git tag) "v1.2"
rc Response code See device actions
hw Client-specific hardware specifier (not used by NetReceiver but saved by clients) "dhtPin=22"

Sensing

The poll method sends sensed input values from the client to the service. For example, if analog port 0 (A0) reads 100 and digital port 2 (D2) reads LOW, the poll for the device with MAC 01:02:03:04:05:06 and device key 12345678 is: 

  /poll?ma=01:02:03:04:05:06&dk=12345678&A0=100&D2=0

Actuating

The act method requests values to write to output pins, e.g.: 

  /act?ma=01:02:03:04:05:06&dk=12345678
which returns JSON with pin output data, e.g., if D1 is to be set HIGH: 
  {"ma": "01:02:03:04:05:06", "D1": 1}

State

In addition to a site's configuration information, users may define variables to represent state. The vars method requests such state, e.g.: 

  /vars?ma=01:02:03:04:05:06&dk=12345678
returning JSON variable name/value pairs: 
  {"id": "MyDevice", "MyDevice.Light": "on", "MyDevice.Camera": "off", ...}

Device actions

Response codes included in NetReceiver's response are used to initiate device actions, shown below with their corresponding NetReceiver status icon.

The status icon remains in effect until the action completes, whereby it reverts to the ready state (). The reboot status is displayed until the device is updated.

Upgrading is a 3-step process as follows:

  1. Update the device tag to the desired Git tag, e.g., from "v1.0" to "v1.1".
  2. Once the device is ready, initiate the upgrade.
  3. Once the device is ready, reboot the device for the upgrade to take effect.
If the device remains stuck in the upgrade state, it means the upgrade has failed. If this happens, updating or rebooting the device will reset things.

Variables

A NetReceiver variable is simply a string. Boolean variables are variables for which a value of "1", "true" or "on" is treated as logical true. A value of "0", "false" or "off" is logical false, as is a variable that is not defined.

Variables can be global, i.e., Var, or local to a device, i.e., Device.Var. Global variables are shared across all devices, whereas local variables are only accessible to the given device.

When variables are mapped onto actuator pin ervalues, logical true is sent as integer 1 and false as integer 0. Other integers are sent as is.

The var sum, denoted "vs", is a 32-bit CRC-style checksum of the current variables and their values which is returned as part of each poll or act response. Clients are responsible for comparing the latest "vs" value with their cached value to determine if their variables are out of date. Clients then reissue a /vars request.

Special variables

There are two special variables which are always included, namely id and mode. The former is the ID assigned to the device by the user (not to be confused with the MAC address). The latter represents the client's operating mode. Clients may set arbitrary values for the mode but conventionally Normal signifies the default mode and Paused signifies that the client is paused in some way (case sensitive). Clients should therefore check the mode value and act accordingly.

While most variables flow one way from NetReceiver to the client, clients may communicate mode changes in the other direction. This is done by adding the md parameter to the /vars request. For example: 

  /vars?ma=01:02:03:04:05:06&dk=12345678&md=Special
Upon success, the /vars response always includes the new value for the mode. Also, as for other variables, changes to mode can fire triggers.

Sensors

Sensors transform raw pin values into useful measurements, including binary data. NetReceiver includes a number of built-in functions to make this easy. These generally take comma-separated values of floating-point numbers. Additional custom functions can be defined in code, in which case the single argument is the name of the function that performs the custom transformation.

Name Args fn(x)
none x
scale a a * x
linear a,b a * x + b
quadratic a,b,c a * x2 + b * x + c
custom fn fn(x)

Triggers and crons

Triggers perform actions resulting from specified sensor conditions, e.g., a voltage is less than a specified value. Triggers have optional preconditions, which are simply one or more boolean variables which must all be true in order for the trigger to fire. Triggers with preconditions are evaluated prior to those without preconditions (in the same order as displayed). Triggers executed previously can set variables which are read or modified by subsequent triggers.

Crons perform actions based on specified times of the day. To perform an action for an interval of time requires one cron to start and another to finish.

For example, one trigger or cron may set the "Alarm" variable to "on" (true) and another may set it to "off" (false). Further, other triggers may specify "Alarm" as a precondition and will only fire when "Alarm" is true.

Crons may use either a numeric time such as 09:45 or a symbolic time, namely one of Midnight, Noon, Sunrise or Sunset. Midnight and Noon are calculated based on the site's timezone and Sunrise and Sunset are calculated based on the site's location. The current values for Midnight, Noon, Sunrise and Sunset are displayed on the Settings page. Crons may also use a repeating time, represented by asterisk (*) followed by 1 to 4 digits. For example, *30 means repeat every 30 minutes, and *800 means repeat every 8 hours (not 800 minutes).

Notifications

NetReceiver sends notification emails from NetReceiverService@gmail.com, so ensure this address is not treated as spam by your email provider.

Client

The client is known as NetSender, which is also the name of the client library. The default client is gpio-netsender. There is an Arduino C++ version for the ESP8266 microcontroller (based on the Adafruit HUZZAH ESP8266 breakout) and a Go version for the Raspberry Pi. You may install multiple clients at multiple locations at your site. Note that a WiFi connection is required and the ESP8266 only operates in the 2.4Ghz WiFi band. Read about the special features of the ESP8266 Arduino client here.

Setup

You will need to know the client device's MAC address in order to register it with NetReceiver. Once you have this information, go to your site's Settings page and create a Device object with the given MAC address. Also specify the WiFi network that you wish to use. The other parameters can be specified later.

When a NetSender powers up it first tries to connect to the default network, which is a WPA/WPA2 WiFi network named "netreceiver" with the password "netsender". Any easy way to do this is to temporarily set up your smart phone's hotspot (tethering) this way. Do not use the default network as your device's permanent WiFi settings as it is only intended to be used as the fallback network.

If a NetSender cannot connect to the last saved network it will attempt to connect to the default network and will continue to do so indefinitely.

Note that NetSender does not require a WiFi connection that is either permanent or 100% reliable. NetSender will keep trying to reconnect until it is successfully able to connect.

Monitor Period

The monitor period for a device is the number of seconds between monitor/actuation cycles.

Extending the Client

The NetSender library and default client, gpio-netsender, has support for reading values from analog (A) and GPIO digital (D) pins. For sensors that require more work than simply reading a pin, e.g., reading a data sequence, counting interrupts or a software-defined sensor, you'll need a custom client. NetReceiver makes this possible with the concept of external pins prefixed with an "X", i.e., "X0", "X1", etc. You simply map these virtual pins to your custom functions. NetReceiver treats external pins like regular pins, allowing you to define sensors and triggers, etc. See temp-netsender and weather-netsender for examples.

External pins

External pins are "X" followed by 1, 2 or 3 digits. The first digit of a 2-digit pin (or first two digits of a 3-digit pin) denotes the device type and last digit denotes the quantity. No more than 10 quantities can be assigned to a single device type, however a second type can be assigned to the same physical device if more than 10 quantitites are required. While digits representing devices are unique, digits representing quantities are not. For example, temperature may be quantity #0 for one type of temperature sensor or quantity #2 for a different type of sensor. Single-digit external pins, such as X1 or X2, apply to all devices.

Device type assignments

Device Type
ESP 1
Pi 2
SDLWR 3
DHT11 4
DHT22 5
DS18B20 6

External pin assignments

Device Number Input quantity Output quantity
Any 0 Download/upload size (bytes)
Any 1 Download bandwidth (bps)
Any 2 Upload bandwidth (bps)
ESP 10 Analog (A0) value
ESP 11 Alarm (bool)
ESP 12 Alarm count
ESP 13 Boot reason
ESP 14 Suppress pulse (bool)
Pi 20 CPU temperature
Pi 21 CPU utilization
Pi 22 Virtual memory
Pi 23 Aligner median error
Pi 24 Aligner error std dev
Pi 25 Aligner signal strength
Pi 26 Aligner link quality
Pi 27 Aligner link noise
Pi 28 Aligner link bitrate
Pi 29 Aligner reference angle
SDLWR 30 Wind speed
SDLWR 31 Wind gust speed
SDLWR 32 Wind direction
SDLWR 33 Precipitation
Pi 34 Turbidity
Pi 35 Salinity
Pi 36 RV bitrate
Pi 37 Dissolved oxygen
DHT11 40 Temperature
DHT11 41 Humidity
DHT22 50 Temperature
DHT22 51 Humidity
DS18B20 60 Temperature

Temperatures are reported in tenths of a degree Kelvin. Humidity is reported in tenths of a percentage. Raw temperature sensor values can be converted to degrees Celsius with a linear(0.1,-273.15) transformation and raw humidity values can be converted to a percentage with a scale(0.1) transformation.

Experimental methods

Binary and text data

Binary and text data can also be sent to NetReceiver using the poll method, using pin types B and T respectively. Binary data is useful sending sensor data more compactly than text formats, such as JSON.

Recent data can be downloaded using the data and text methods respectively. For example, to download binary data B0 originating from a device named DEV: 

  /data/KEY/DEV.B0
KEY is either the site key or an application-specific password granted by the site's administrator.

Here is another example for text data for a specific date/time range: 

  /text/KEY/DEV.T0?ds=START&df=FINISH
or for a duration from a start time: 
  /text/KEY/DEV.T0?ds=START&dd=DURATION&du=UNITS
START and FINISH are times are ISO-format date/time strings, i.e., "YYYY-MM-DD HH:MM", or a time string "HH:MM", DURATION is a number and UNITS is one of minutes (default), hours or days.

Video and audio

NetReceiver's previous experimental support for video and audio has now migrated to Video Grinder, which is written in Go rather than Python. Source code can be found at bitbucket.org/ausocean/vidgrind.