As i am new to the board and the OpenEnergyMonitor hardware i found it very interesting with the possibility of this platform. And also i find the threshold a bit high for people who really want to use this platform, but lack the computer skills.
I spent some time thinking about how to make a small easy setup system from the Nanode that does not require any programming of the Arduino/Nanode board and can be delivered pre programmed from the emon shop. Just a plug and play user experience.
As of today, the API key has to be put into the software. And since this API key is user specific each unit needs to be programmed by the user himself. This gives the system a high technical threshold. The whole arduino programming and github thing scares most people away.
However, if each unit had a ID chip on the one wire bus like this unit:
http://www.maximintegrated.com/en/products/digital/memory-products/DS240...
And the emoncms had a feature for finding Nanodes from the same IP as the user who is logged in, then it would be possible to link Nanodes to a specific user with no programming. Just one simple link/connection to the DS2401 unique chip ID instead of the API key.
In the emoncms database, the DS2401 ID will be the post that replaces the API key, or the ID will be part of this key.
Basicly if a user connects 10 of these Nanodes on the same network, and logs in to the emoncms, he will find 10 units each identified by the DS2401 in the emoncms account. And emoncms knows they belong to this user because they are all logged in from the same IP address. This is a first time setup only.
After the first setup, emoncms knows that these units belong to this account, and all logging from these unique IDs will be on this account.
If this unit has inputs for only one CT, One optical pulse input and some Dallas sensors on the one wire bus it will be a really cheap system that anyone can set up. It might be inside the technical boundaries of the Nanode alone. And the production cost will be very low. If it can handle 3CTs, 2 pulse inputs, and some Dallas sensors it will be more then most people will ever need.
One unit can be used to monitor a heatpump, or the ventilation, or houshold electricity.. Just about anything.
Of course this system will not be as good and flexible as the current wireless systems. But it will come in at ~ £30 and be all that the small, first time user needs. If there is no network connection, a cheap RJ45-->WiFi bridge can be used for another ~ £15.
WiFi unit (running linux and can be reprogrammed for ease of use) price < £15
Is there any technical limitations in the emoncms that will not allow this idea to work?
Thanks
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Thats a really neat idea to check the unque ID against the logged in IP of the user! I had been thinking of the idea of preprogramming a serial number to the basestation which was also printed on the case, that the user could then enter in emoncms to link the account to the device, but your suggestion is simpler both from a configuration point of view on the shop side and for the user.
I cant see the needed changes to emoncms to make this work being an issue either, nice idea!
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Thanks Trystan!
I am an engineer in energy and construction (buildings), i really dont know much about software and hardware stuff but sometimes i can figure it out ;)
Im always obsessing with ease of use and price, because i know that these two points are always the ones that makes my customers back off. No matter how much money they can save from energy monitoring, they don't care if it means they need to be learning new things or spending big €€. They don't want to learn, they don't want to spend money.. They just want it to work straight out of the box.
This means alot of time goes into setting a system up for the customer. This is expensive, so then it is back to square one.
Ease of use puts the price down for installation, as well as producing and maintaining. Not to mention support.
Programming different IDs into each unit with the arduino software will be time consuming or require some kind of special equipment i guess, if nobody wants to make it manually. But just adding this ID chip will fit straight into current production methods and will not require any modification to production lines other then specification for components and maybe small alteration to the board-drawing and the basic sketch. After that it's just the same work flow as you have today. Some smaller changes in the emoncms will be needed also.
I hope this can work. It would be nice with a really small and simple unit that can compete with the basic systems that are on the market today.
Simple systems avalible:
http://www.eliq.se
http://www.efergy.com
http://www.theenergydetective.com
They are all simple to set up, and cheap. But they are not very flexible, and they all work pretty much the same.
It is obvious to me that people who design these systems in general don't know so much about buildings or energy usage in buildings. And people who do, don't know so much about data processing and electronics. And this is making all the systems above crippled in some way.
Anyways. Hopes this idea can be some small seed to a cheap and simple energymonitor to get people started.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
From the DS18S20 datashet: (http://datasheets.maximintegrated.com/en/ds/DS18S20.pdf)
64-Bit Lasered ROM Code
Each DS18S20 contains a unique 64-bit code (see
Figure 8) stored in ROM. The least significant 8 bits of the
ROM code contain the DS18S20’s 1-Wire family code:
10h. The next 48 bits contain a unique serial number.
The most significant 8 bits contain a cyclic redundancy
check (CRC) byte that is calculated from the first 56 bits
of the ROM code.
From the DS2401 datasheet: (http://datasheets.maximintegrated.com/en/ds/DS2401.pdf)
Unique, Factory
Lasered and Tested
64-Bit Registration Number
(8-Bit FamilyCode +48-Bit Serial Number + 8-Bit CRCTester)
Are these two similar enough that the same thing could be done with a DS18S20 vice the DS2401, with the added benefit of available temperature data?
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Bill!
That has been also on my mind, and i did not research price difference maybe its the same price anyways.
To find the correct answer i think many aspects should be analyzed.
- Price
- Ease of use
- Ease of use in the emoncms
- Powerusage on the arduino board
- Total length of the ID tag for both chips, and if they are totally unique
Will it be a good idea to have a Dallas sensor mounted on the board, does it make any use there? Will it be a good idea to have it on cable, in case something happens and it needs to get exchanged, what will happen to data in emoncms?
However, the idea to use Dallas sensor instead of ID chip can be very smart. And will suit many users just as well, or even better. In terms of standardizing a product i am not so sure.
Can there be more pros/cons with ID chip vs Dallas sensor?
Best regards
Johannes
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Hi Johannes,
Comparing the two components, we get:
Price
2401 ~2USD
18B20 ~4USD
Ease of use
There shouldn't be much difference here. Both the 18B20, and the 2401, use the 1-Wire protocol.
Ease of use in emoncms
This is more of an "ease of use on the nanode" kind of question, as display of temperature on an emoncms dashboard is something that users have done for quite some time.
Power consumption
This is one area where the 2401 has the 18B20 beat. 18B20 standby current is ~750nA, whereas the 2401 has zero standby current requirements. 18B20 operating current is typically 1mA. 2401 operating charge is 30nC. The emonTH battery life is spec'd at ~16 months for a unit with only the DS18B20 sensor, that takes a temp reading once every 60 seconds. http://shop.openenergymonitor.com/emonth-433mhz-temperature-humidity-node/
Total length of the ID tag for both chips, and if they are totally unique
The difference here is the 2401 has a guaranteed unique serial number, where the 18B20 is advertised as having a unique serial number, but no mention is made of a guarantee that it's unique. Since both chips have 48-bit serial numbers, it appears the odds of any given serial number being truly unique should be very good.
However, the idea to use Dallas sensor instead of ID chip can be very smart. And will suit many users just as well, or even better. In terms of standardizing a product i am not so sure.
Both components are from Maxim, and the 18B20 is already an emonTH option, so it makes sense to use a part that's "already in the pipeline," and can supply a serial number. The only down side I can see, is if some of the serial numbers aren't unique.
Bill
Edit - changed 18S20, and relevant data, to 18B20
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Thats a nice comparision!
The difference in price is almost nothing, and the serial seems unique enough.
The power drain difference can be controlled in the software for the Arduino, if there is no need for temperature measurements the software can be setup to only check the ID at startup. And then never check back on that one wire protocol. I dont know anything about the programming but i think that can be done.
In total, this would be a nice feature to add to coming versions, and i will avoid the API key programming problem and the boards can be delivered with pre programmed software that will work direct with emoncms.
Thank you Bill for supporting this idea with your thoughts and ideas.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Really interesting! and nice idea to use the DS18B20. This could also be used on a raspberrypi based basestation for easier setup
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Trystan, it's even better to use it for the raspberry as well.
Maybe I should change topic for this thread. It's a bit wrong for the discussion.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Hi J,
Thanks for the nice comment!
Will it be a good idea to have a Dallas sensor mounted on the board, does it make any use there? Will it be a good idea to have it on cable
When I was an Electronics Technician for the US National Weather Service, part of the job involved repairing weather station equipment supplied by Vaisala, RM Young, Davis Instruments, and Peet Brothers.
One thing in common was a cabled temperature sensor.
All of the manufacturers mentioned above recommend housing an outdoor temp sensor inside a solar radiation shield to prevent false temp readings as a result of direct exposure to the Sun. Here's a commercially available example: www.ambientweather.com/amwesrpatean.html.
Here's an example of a DIY rad shield: www.weather-watch.com/smf/index.php?topic=38652.0.
A board mounted sensor would pick up heat from the electronics, unless the housing the sensor is in, is aspirated, or the sensor is located away from the electronics i.e. on a cable. So a cabled sensor is advantageous for indoor use too.
Im always obsessing with ease of use and price, because i know that these two points are always the ones that makes my customers back off. No matter how much money they can save from energy monitoring, they don't care if it means they need to be learning new things or spending big €€. They don't want to learn, they don't want to spend money.. They just want it to work straight out of the box.
Sadly enough, that's a common attitude in the US too. Just the mention of the word "Linux," or "software" scares away lots of people.
B
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
"A board mounted sensor would pick up heat..."
That's a complaint that was made about the GLCD. The temperature sensor was at the top of the board when the GLCD was standing in it's normal 'on edge' orientation and several users claimed it was over-reading as a result of heat coming off the processor.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I noticed that too, and filed it away as one of those "I'll do it later" projects of putting the sensor on a cable to get it away from the PC Board. Haven't gotten 'round to it, yet. Oh Well...
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Ok, thanks for all the input guys!
I am still thinking about what suits the users best here.
Using the Dallas will give more features to the package with no extra cost. But mounting it on the board will give bad readings, and mounting it on a cable can cause problems in the emoncms later if the sensor needs to be exchanged, as all the old data will be linked to the old sensor.
Is this conclusion correct from what was pointed out earlier in this thread?
Is there any clever way to overcome this?
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Just to be awkward, what happens when you have multiple sensors on one node?
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Hello Robert!
I think that one solution can be to keep one data bus for one unit for retrieving ID, and never connect multiple units on this data bus.
There can be another data bus for other sensors.
If its done this way, i think it will be very easy to write the code in such way that the ID number can be retrieved on startup of the unit. Just by addressing this bus with a query for ID.
I am just trying to think about how to do it in such way that the units can be shipped with no software modifications needed. Just hook it up and create account in the emoncms, and there is all your units on the same IP. Each unit can be given a name, but will be identified from the ID on this data bus in hardware.
Maybe this is a loss of one data bus, but it sure will be easy for beginners to set up.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I was thinking in terms of a substitution list in emonCMS, which would (probably as an option) allow the user to tag the ID or IDs with the chosen node number or a "friendly" name. That would allow ID device and node replacements (you just edit the table) and assuming multiple devices on the bus with the IDs being returned in random order, with the node software could send whichever it liked - first discovered, lowest number - and CMS would recognise all as equivalent and allocate the data correctly.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
A Microchip 24AA02E64 MAC EEPROM is available at ~€0.20 with 64 bits unique ID. As it is I²C it uses up two I/Os, of which one could be reused for user porposes.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Now i found DS2401 for less then 1$ incl. shipping on ebay.
If one buys 100pcs i guess the price will be even less.
I think that using the one wire protocol is better then using I2C, because the I/O is still usable for other stuff and the ID chip can be mounted on the PCB as standard.
Still the idea with Dallas sensor is pretty good, if it can be done in such way that will not make problems in the emoncms if it needs to be exchanged or something like that.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Using the Dallas will give more features to the package with no extra cost. But mounting it on the board will give bad readings, and mounting it on a cable can cause problems in the emoncms later if the sensor needs to be exchanged, as all the old data will be linked to the old sensor.
Still the idea with Dallas sensor is pretty good, if it can be done in such way that will not make problems in the emoncms if it needs to be exchanged or something like that.
Even if the 2401 is used, the same problem exists if it fails. i.e. you now have a copy of emoncms tied to a serial number from a dead chip. Seems like the temp sensor is the way to go. Same comm protocol, 48-bit serial number like the 2401, and temp data available for use if desired.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I don't want to say the MAC EEPROM is the better solution, but it is much cheaper and any two (or one, see above) I/Os can be used with a bit-banged minimal I²C emulation. For a board with ethernet it can additionally be used for MAC address.
BR, Jörg.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I've always liked the idea of each "node" either nanode or emonTX/TH and the base all having unique ids/addresses.
This way the base can be modified to just listen for the devices it wants. I'd still like to see each of the sending devices communicating using a generic protocol to deliver their readings rather than having to configure all the nodes in the base and what each of there readings mean.
It would be very cool if the base station (Pi?) "discovers" new sensors/nodes and simply asks the user to confirm they want to receive data from them - especially if the input configuration was automatic as well.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Some notes:
- Arduino with ATMEGA328 already has a 1k eprom on chip.
- Ethernet developer boards usualy have mac address configured by the host cpu. That means it gets set from the arduino firmware and must be saved there.
You could implement a simple random serial id generated by pressing a button at arduino boot that gets saved on its internal eprom for further usage. One firmware fits all.
There is no assurance that the serial id is unique across devices.
I personally dont like the fixed serial id, it's like old intel Pentium III Processor Serial Number, rastreability of devices is a privacy issue that may have legal concerns on some countries.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Chaveiro, thanks for your input.
Do you have another idea, except from a ID chip, that will allow the emon hardware to run with no alterations at all to the firmware by the user and still be unique in that way that the emoncms can differentiate between each unit?
I am open to all kinds of solutions that allows the user to just plug the unit, log into emoncms and have the unit located there with no tampering in hardware or software. It does not need to be a chip.
I think to most users privacy is not a major concern, and if someone manages to log into the emoncms, they can pretty much get all the data they want even if the IDs are randomized. The data always ends up one place, its just a matter about how easily it finds its way there.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Hmmm, I really don't see the problem. Every ethernet node already has a unique MAC address (or 'serial number'). As have many RF nodes which are pre-configured by the manufacturer. I am using a lot of Homematic devices and each has a unique serial number, too. So it is very easy to automatically find and register the nodes in the system.
Why not re-use things that work perfectly in other (very similar) environments?
Jörg.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Well i see 2 scenarios :
A - Ethernet/wifi nodes that have a local server used for ip / ssid configuration where you input your apikey.
B - RF write only nodes that has no mean to receive data
For A it's partially solved and easy implemented, connect to node config page, set apikey that gets saved on eprom.
For B it will have to write to a local hub via rf so you can implement this nodes with a switch on one cpu pin to act as node configuration that increments it's node ID each time its booted with the pin pressed. Or switch to a configuration mode with led blink status report for setting the ID that gets saved on eprom also.
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Here is another very simple idea.
If using the emonPi module, you could just read the RaspberryPi serial number from /proc/cpuinfo and use that as identification.
This number should be unique and there is no additional hardware required. Any thoughts?
Ondrej
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
This was a really good idea!
It will work very well with all people who are using only one board anyway, for people who are using multiple nodes i think that it could be useful with ID on each unit.
Since this is just a ID number anyway i dont see why both can be used?
I am interested to know if this idea will be implemented in the next version?
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
After some more carful thinking around this i came to some simple conclusions.
If the raspberry pie identifies with the serial as ondrei suggested, and all the EMON units have an ID chip for unique serials. This will make it very easy to organize data in the emoncms.
It can also make room for a very simple setup process for the users.
Today setting up the pie requires tinkering in terminal. This will scare 99% of all normal users away. A simple GUI for setting up the Pie on the web can be done like this (one of many alternatives):
This can create some problems with read/write privileges or stuff like that. Or might not be possible du to other reasons. But as i see it, this will be a very easy way to configure the Pie for users that are not very well oriented in unix / commandline based environment. Configfile will be downloaded/rewritten from the openenergymonitor.org. That will make for possibilities to create nice GUI, and make simple changes for all users no matter if they are on the same network as the Pie.
If user wants to remove one unit from their account, it should be a simple "release" button to release that unit.
If more units are added, it should be very simple for the user to just login to the emoncms.org from the same IP address as the new unit is added, and the emoncms.org will automatically prompt the user also when adding more units.
I also suggest that emoncms.org should have a feature to add names to all the nodes in the network. So that they are presented with names inside the emoncms, that will correspond to their unique ID. This way various locations can be logged with no problems, and multiple sites can be within one account.
Maybe they should be organized with a tree structure or something like this, so that it is visually easy to determine what nodes are under what hub. That will be a later matter, but very useful for people who use multiple Raspberrys on multiple sites, all logging to one account.
Ex as follows:
Main user account.
"Villa" = Raspberry Pie ID 1277545312
"Garage" = Raspberry Pie ID 2231231231
"Summerhouse" = Raspberry Pie ID 1245545312
I think this can make all the setup process very simple for people with only one unit, and people with many units.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I do not have static IP address. If I reset my router, my ISP allocates a different IP address to me. So I am the same user, with the same hardware, but not the same IP address. How will that work?
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
Hi Robert!
This is a idea for a first time setup (its actually just two steps):
Now the raspberry and your web session towards the emoncms.org will have the same IP, because this is happening at the same time. This makes for simple association between the two units, that cannot be hijacked so easily by others.
Inside the server, automatically this will happen:
So the emoncms.org will now notice that there is one unidentified raspberry connected, and one user logged in from the same IP. The emoncms.org will prompt the user who is logged in and ask if the user wants to associate this new raspberry pie (ID) with his account.
After this first time setup process, which is very simple and requires no typing/input of API key or serial numbers for devices, this raspberry will be associated with the account you have setup on emoncms.org. Even if you move it to other IPs all the data will still be delivered to the same account on the emoncms.org server, because its based on the serial from the raspberry, and not the IP or API key. You could say that the API key is replaced by the serial of the raspberry, and instead of putting the API key in the config file, the emoncms.org server gets the API key from the raspberry.
This is just an idea, and if implemented it will make emoncms.org and openenergymonitor.org the most easily setup system on the market.. For everything i know. Efergy, eliq, smappee and the others have more complex setups not to mention the professional systems on the market.
This will make a really good situation for the openenergymonitor.org, because now they can sell a complete system that does not need any configuration at all from the user. It works straight out of the box. The new user just plugs everything in and registers a new account. No API keys, no config files, no linux command prompt.. Nothing. Just pure pleasure and energy monitoring ;)
If all new accounts that are registered like this comes with a standard setup of pre-configured feeds for the emonTH and EmonTx, the user will not even have to make the setup for the feeds. There can be standard feeds pre configured for each new account that is associated with this kind of process. Removable for professionals ofcourse.
If the emonTH and EmonTx gets ID chips on board, they don't need to be configured with node IDs in the firmware. This allows for mass production of this product, with no special configuration what so ever. Just power up, login in and go!
And also there can be a simple dashboard page, that is pre configured for the first time user that shows power now, energy consumption, temperature or stuff like that. Just a simple nice page that does not require any setup with feeds or dashboards, working only with the standard configuration emonTH and emonTX.
If this were to become reality, emoncms can be a product that my mother could set up in 15 minutes and start using. Right now it takes a long time to set this system up, understand how it works, and start using it. Reducing this threshold is important if this system is to attract new less experianced users.
All this is possible but requires some job on the emoncms server side..
Myself i really think this is a good idea, and that if the skilled guys on this forum gives it a go it will be doable.
All the time that can be saved, and all the new potential users for this project will really be good. And from there its not to far to start competing with the commercial systems. Its only price and visuals from there.
I think this idea can be polished but this is one way to go.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I quite like the idea of the while logged in to emoncms using a "search for other devices on same IP" to then add them to an authorized list BUT I think emoncms should pass the apikey (auth token) to the device so that you are not using a "non-changeable if compromised" serial number, it doesn't restrict users to the Pi club and it circumvents any "personal data" issue attached to reading and retaining a devices serial number.
Paul
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
That could also be a way to do it, actually its just as good. And the way you suggest can allow existing system to with API key to keep on going with no changes, only that the raspberry downloads the API key.
One way to solve the association problem could also be putting names on each unit, so that all the feeds from one unit will still be possible to transfer to the replacing unit just by simply giving it the same name.
The same goes for the ID chip i suggested in the EmonTH and EmonTx. If this ID chip is used it will be very easy for the system do differentiate between units. And there will be no need to put different IDs to keep them apart. Its all included in the design. And that way, even if you swap the EmonBase, all the Nodes will still have the same numbers and keep feeding to the same feeds even if there is a new base.
All my thinking around this is actually around making this system simple enough for ordinary people. Today we are a board full of Linux/PhP/Electronics/Arduino gurus.. But in order to reach the big markets and more users it has to be just as simple as smappee, eliq and efergy. I think we can do that.. But better =)
But before its implemented, i really think that everything should be well planned, so that in case units gets swapped or needs to be exchanged for various reasons, all the data in the emoncms should be kept intact.
I am so much hoping for someone to pick this idea up and implement it in the emoncms.
//J
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
I'm not convinced a per node recognition is required in emoncms unless they are devices posting direct in which case they will be connected via the same ip, have thier own mac address and could be altered to accept an apikey from emoncms. this option is going to become less frequent as most devices will either log to a local emonhub or in time post to a remote mqtt server which has its own encryption and authentication built-in.
I think concentrate on making the emonhub to emoncms connection both easier and more secure as well as improving the current node to hub authentication. the hub would be responsible for providing emoncms with pre-authenticated channels (nodes) via it's own authentication.
Paul
Re: Using Chip ID instead of API key, simple setup energy monitor, less skills required.
About security i don't know much so i cant contribute to that discussion.
When you mention API key download i get another idea too..
I'm thinking that it might be a good idea to have a preferences/settings page in the emoncms, that creates a config file for the emonhub.
This way no one would ever have to go into the unix grotto to configure the unit. All the settings could be stored in the emoncms and downloaded automatically to the base. This could make a really nice interface to setup the pie. Stuff like launching the wunderground interfacer and all other kinds of scripts that is in the emonbase.
However, that's another thread for that discussion. And i dont even know if it's possible.
//J