Plug and play sensor nodes
At the moment to get a system up and running you need to do quite a bit of manual firmware configuration, setting unique node-id's selecting different emontx sketches for different functionality, hard coding the emonglcd to get different screens. To add a second emontx you need to add several lines of code to the emonbase and the basestation needs to know the data structure of the sensor nodes.
What if you could upload a standard sketch to each of these nodes, plug your base station into your router, power up an emontx and see emontx 1 appear in a node list in emoncms. Then as you add more nodes they automatically appear in emoncms, no additional base station configuration needed. In time the software could develop to allow remote configuration of nodes such as emontx config and emonGLCD pages: beginning with page template functions as described in the last blog post: emonglcd template based displays
There are several people I know of that are developing these ideas including Mark Campbell who I have been in contact with by email and is developing base station based auto ID assignment (similar to IP addresses) , eeprom storage of auto ID's, MAC address type unique ID's on the nodes.
Pcunha has also developed as part of his larger 1-3 phase sketch remote ID assignment, emontx config including setting calibration variables.
I put together a MultiNode NanodeRF sketch a couple of weeks back that works with the new version of emoncms, its on github here: https://github.com/openenergymonitor/NanodeRF/tree/master/NanodeRF_multinode
So I think we're making some good progress on these ideas.
The following is a barebones concept that demonstrates auto id assignment and receiving data from multiple nodes.
The main idea is that commands and data packets are sent in a variable width char array.
The first character in the array details what kind of packet it is:
d - 2 byte integer data i.e int power1, power2, power3, voltage.. i - set node id
this packet type character determines the packet deconstructor to use.
Effort is made to keep the packets as lightweight as possible, for example sending a set node id command only uses 3 bytes:
1st byte is the packet type 'i' for set node id. 2nd byte is the target node id, 3rd byte is the new id.
char data[] = {'i',1,autoid};
Decoding the 2 byte integer data also allows for different packet lengths, which makes it possible to add a variable to be sent on the sensor node without chaning the basestation.
Here's the full code:
Concept 1: Auto ID's and Multiple nodes sketch.
TX code
#include <JeeLib.h> int mynode = 1; #define freq RF12_868MHZ #define group 1 typedef struct { char type; int power1, power2, power3, battery; } PayloadTX; PayloadTX emontx; unsigned long timer; void setup() { Serial.begin(9600); Serial.println("tx"); rf12_initialize(mynode, freq, group); } void loop() { //------------------------------------------------------------------ // RX data //------------------------------------------------------------------ if (rf12_recvDone()) { if (rf12_crc == 0 && (rf12_hdr & RF12_HDR_CTL) == 0) // and no rf errors { int node_id = (rf12_hdr & 0x1F); char type = rf12_data[0]; if (type=='i') { int target = rf12_data[1]; int newid = rf12_data[2]; if (target==mynode) mynode = newid; rf12_initialize(mynode, freq, group); } } } //------------------------------------------------------------------ // Transmit data //------------------------------------------------------------------ emontx.type = 'd'; emontx.power1 = 520; emontx.power2 = 1209; emontx.power3 = 1000; emontx.battery = 3300; if ((millis()-timer)>1000) { timer = millis(); int i = 0; while (!rf12_canSend() && i<10) {rf12_recvDone(); i++;} rf12_sendStart(0, &emontx, sizeof emontx); rf12_sendWait(0); } }
Base station code
#include <JeeLib.h> #define MYNODE 20 #define freq RF12_868MHZ #define group 1 typedef struct { char type; int power1, power2, power3, battery; } PayloadTX; PayloadTX emontx; unsigned long timer; //int idmap[30][1]; int autoid = 1; char instr[30]; int pos = 0; void setup() { Serial.begin(9600); Serial.println("Base"); rf12_initialize(MYNODE, freq, group); } void loop() { //------------------------------------------------------------------ // RX data //------------------------------------------------------------------ if (rf12_recvDone()) { if (rf12_crc == 0 && (rf12_hdr & RF12_HDR_CTL) == 0) // and no rf errors { int node_id = (rf12_hdr & 0x1F); char type = rf12_data[0]; if (node_id==1) { autoid ++; char data[] = {'i',1,autoid}; int i = 0; while (!rf12_canSend() && i<10) {rf12_recvDone(); i++;} rf12_sendStart(0, data, sizeof data); rf12_sendWait(0); } if (type=='d') { Serial.print("DATA: "); Serial.print(node_id); Serial.print(" "); byte n = rf12_len; for (byte i=1; i<n; i+=2) { int num = ((unsigned char)rf12_data[i+1] << 8 | (unsigned char)rf12_data[i]); if (i>1) Serial.print(","); Serial.print(num); } Serial.println(); } } } }
It can be downloaded from github here:
https://github.com/openenergymonitor/FirmwareDev/tree/master/01
If your interested in the development of these features, would be great to hear what you think
Re: Dev: Plug and play sensor nodes
Looks like a good idea. The main issue I can see at the moment is there is no way of defining what each datavalue is according to the above. So you either have to have fixed entries based on hardware type or guess. You also have the potential to increase the number of transmissions when you have multiple datatypes since I think you'd need one transmission for type d and another for type i.
How about expanding on this along the same lines so you have 3 entries per datapoint: TYPE, NAME, VALUE.
Then you send packets like this:
{'i',1,autoid,'d',235,'Vrms','d',397,'Power1'}
A slight increase on bytes but allows this to be passed directly through to emoncms with input names.
Expanding a little further on this. Since a power saving is acheived by not transmitting floats temperatures are multiplied by 100 we could use a more detailed data type, either as a charactor or better as a series of defines in emonlib: For example:
EMON_TEMP: Temperature*100 (divide by 100 to get true value). Type: Int (-32,768 to 32,760). 32,761-32,767 are reserved for sensor error reporting (eg. 32767 = no reading from sensor). Maximum reading 320C (you might define a temperature with type long for things that must go higher like wood stoves).
EMON_POWER_KW: Power(KW)*100 (divide by 100 to get true value). Type unsigned Int (0-65528). 65529-65535 are reserved for sensor error reporting (eg. 65535 = no reading from sensor). Maximum reading 655.28KW.
EMON_VOLTAGE: Voltage (V)*100 (divide by 100 to get true value). Type unsigned Int (0-65528). 65529-65535 are reserved for sensor error reporting (eg. 65535 = no reading from sensor). Maximum reading 655.28V.
You then specify your RFM12 string:
{EMON_VOLTAGE, 235, 'Vrms', EMON_POWER_KW, 397, 'Power1'}
This way any receiving device and emoncms has all the information it needs to use the data and if you include a node id in the string where it has come from too.
Re: Dev: Plug and play sensor nodes
Yes, That's exactly what i have in mind.
I Have some points to consider:
- I Think we have to deal with ACK's. In my code i use them to reply status and pass returns back. I have read all the jeelabs documents about it, and i can say that the rfm code can be extended a little to provide a better solution (but thats not the point here)
- I like "Keep it simple" but like "keep it single" more. I dont like multiple small packages when we can pass one large package.
in my case i pass all the variables to the Emonbase using almost all bytes that the rfm can transmit. It can be a big battery problem (or not) i'm not shure, but it can be a problem with EU regulations too ( is that correct?). so it can be optimized. The EmonTX can have 2 methods of sending data: 1. All data collected (As in my case) or 2. Selective data (just total power by example).
- I Like Computer Browser Interfaces, so it's possible to use the emonbase as a bridge and use a Java/Javascript application to drive and configure all the stuff on the nodes. ( That's evil ).
-Compatibility for extensions: My main concern is be possible to do modifications without to brake all the software. Ex: i want to add a gas meter emonTx. I Can create a new code that will use all the features of the network without major problems, or i need to add a cummulative KWH variable (as i do) on the Emontx, without breaking all the comunications and CMS, etc.
Thanks a lot,
Re: Dev: Plug and play sensor nodes
Didn't I suggest doing something similar to this back in June?
http://openenergymonitor.org/emon/node/722
As the previous posts above, I think this would only be a good move if you also include the type of measurement and its unit of measurement - this way GLCD can automatically know that its a PV energy reading or a hot water temperature probe etc.
The GLCD would also need to combine several readings into potentially several screens/groups, you could have a couple of emonTx units sending data which are related to each other - do we need timestamps or a time sync broadcast message from the GLCD once in a while ?
As someone who has personally designed and worked with these sorts of protocols in the past, please also include a version number in the packet somewhere to allow you to expand the format without breaking all the existing code!
Re: Dev: Plug and play sensor nodes
Thanks guys, so datatype definition needs to be a key feature. I wasnt sure weather datatypes should be sent each time or if there could be a way to send it the first time and not repeatedly, thinking that if we could avoid sending datatype and dataname data then we could keep packet size to a minimum.
But thinking about it, provoked by your ideas there with defines mattwire and I think its exactly what your proposing in the linked post stuart - sorry I didnt catch that earlier!, I can see that datatypes could be sent with each transmission with only a very small bit of extra weight.
If we used a definition list as you suggest stuart:
Then the datatype could be a single byte giving us 255 possible datatypes and that single byte could define everything we need to know such as:
the variable name in emoncms could be constructed from this such as node1.ct1.realpower
If the datatype was a single byte that came before its data bytes such as:
1 byte (datatype), data bytes, 1 byte (datatype), databytes...
Then the datatype byte can tell the deconstructor how many of the following bytes belong to the data..
..
Ok so as Im writing this I can see that the datatype byte is or can be the same as the packet type character I was suggesting above. But instead of only being the first one it can be as stacked as you suggest.
So a section of our 255 available datatypes could be commands such as set node id.. and another section may be actual datatypes such as Voltage..
Nice!
Can you see any problems with referring too all data and command type defenition in a single byte..?
Re: Dev: Plug and play sensor nodes
I've had a go at writing a packet constructor and deconstructor for the 1 byte (datatype), data bytes, 1 byte (datatype), databytes packet format idea, I think its looking good so far, I have put the code up on github here: https://github.com/openenergymonitor/FirmwareDev/blob/master/packet_builder/packet_builder.ino
Would be great to hear your thoughts.
Re: Dev: Plug and play sensor nodes
A single byte might be limiting in the long run, but I also agree that keeping the packets small is important.
How about "reserving" a bit on the byte to indicate that this is a 2 byte data type and then the receiver can use that ?
You could also have some sort of broadcast "here I am" message when a node is first powered up (and perhaps every 100 packets or 10 mins) with this data in it if you didn't want to send this all the time - this might be a better way of doing things.
Additionally, I think you many need to include a description/definition of what you are trying to send back to base.
Its no good sending 10 temperature measurements from varios devices if you have no idea where the sensors are or what they are measuring !!
For instance:
Does the RFM12 device do hardware level checksum calculations? Do we need to bother with that ?
Think security/encryption of the data has also been mentioned in the past.
Re: Dev: Plug and play sensor nodes
The RFM12packets include CRC checksum, no need to handle this additionally.
Encryption may be an overkill, but it is already available for those who need it, see examples here:
https://github.com/jcw/jeelib/blob/master/examples/RF12/crypSend/crypSend.ino
https://github.com/jcw/jeelib/blob/master/examples/RF12/crypRecv/crypRecv.ino
Re: Dev: Plug and play sensor nodes
I also wanted to share some ideas for the auto node id assignment:
1. The NanodeRF sends out a time packet every time it receives incoming one (this is so that the EmonGLCD is updated)
Re: Dev: Plug and play sensor nodes
mharizanov, think you might be taking a slightly different approach to what I was originally suggesting.
Thoughts so far:
1. Why does a node id matter at all ? Why not broadcast every packet ?
2. You could have several devices transmitting data for a grouped set of readings (e.g. solar pv, current, temperature, voltage, power levels etc.) so don't think that a single node will be sending all those readings together in a single packet or even same time span!
3. If the data packets are self describing then no changes are needed to GLCD or emonCMS to handle the packet
4. Not everybody want's or uses emonCMS - its an extra expense to host/power that some people won't want
Re: Dev: Plug and play sensor nodes
How about not only making the EmonTX nodes plug and play, but also the inputs. Auto detect if a CT, pulse or temperature probe has been connected to the EmonTX and automatically start sending data.
I'm going to have a try to combine above with the FirmwareDev packet routines.
Re: Dev: Plug and play sensor nodes
Probably worth doing this on startup only to avoid the overhead. Add the new sensor and simply reset the emonTx and it discovers it.
Re: Dev: Plug and play sensor nodes
You could have several devices transmitting data for a grouped set of readings (e.g. solar pv, current, temperature, voltage, power levels etc.) so don't think that a single node will be sending all those readings together in a single packet or even same time span! http://gurjeetguri.blogspot.com
Re: Dev: Plug and play sensor nodes
Any progress with this?
Did you see JeeLabs recent thoughts on the same issue?
http://jeelabs.org/2013/01/14/remote-node-discovery/
http://jeelabs.org/2013/01/15/remote-node-discovery-part-2/
http://jeelabs.org/2013/01/16/remote-node-discovery-code/