Where to start for a new house

Hi,

 

This is my first post here, I discover the project thanks to jihele article on linuxfr.

First of all, sorry for my english, I'm french :)

This looks like a great project and something that I want for our house project. We are waiting some land in the city we want to build an house. This is planned for 2014 (lands are published beginning of year, building will follow just after)

So I have read some of the wiki and articles (not all, there is so much :D) and forums posts, but I want to share and asks some points here. If the answer is somewhere, please forgive me to have not look well or not finding it, just give me a link or some clues it's already a lot.

 

So here are my questions :

 

1- Is it possible (now or in future plans) to use ethernet and not RF communications ? (as we are going for building the house, will have RJ45 everywhere, and if possible to avoid radio communications I prefer like many people I think) (I see on the wiki there is a list ethernet but no link, here http://openenergymonitor.org/emon/buildingblocks in Networking) I know Arduino can connect by ethernet and for exemple Bretagne Telecom gives clues for ethernet module reading french electric provider (http://departements.telecom-bretagne.eu/rsm/logiciels/smartb/particuliers/) Also ethernet module for Arduino have an option for PoE, making it perfect to not use any additional cable or battery for power)

2- I read a post about sending data with python script to oemcms. Doesn't seems to much complicate to what I read (I'm not really software engineer, mor system so my level in scripting is not good) So it's not complicate to use sensors we already have ? (plan is to get a raspberry in each room already using them for multiroom audio streaming, network webcam and must be also for touchscreen for access domtique software and maybe other usages, one of them is temperature sensor, for this I have ordered DHT22 so will be perfect if can send the data to oem)

3- Is it complicate to use oem for reading its data ? As a good way, I prefer to use twice a data than pulling twice the information. In an home automation, temperature for exemple is usefull in scenarii, or light exposition. So if oem already have the informations, better using them in the home automation software than pulling them from sensor in home automation software.

4-Is there already some work done for interfacing oem with domotique software ? For now I still not sure which one to choose. Have a shortlist of domogik and freedomotic (second one look more active) The sure thing is I will use IPX800 modules for heart of automation.

 

If you have any usefull advices or reading in addition, if you have read all until here you must have a good idea of my project :)

 

Thanks

 

Cédric

TrystanLea's picture

Re: Where to start for a new house

Hello Cedric

Good to hear your interest in building a system and your plans for the house, it sounds like a big project! 

All main configurations that we've been using end up as ethernet at some point whether its the raspberrypi that connects to LAN via ethernet or the NanodeRF which is an ethernet enabled arduino. If you dont want RF at all (RF would probably be cheaper than having an ethernet basestation for sensors in each room) then the nanode without the rf chip with an emontx shield might be your best bet

http://openenergymonitor.org/emon/emonBase/NanodeRF

http://openenergymonitor.org/emon/emontxshield

For your second and third question, Emoncms should do what you want and it has a good API which you could use to get the data out of it. I would experiment with it first to get a feel for how the system fits together, start by getting one sensor logging to emoncms and build up from there.

 

Mattia Rossi's picture

Re: Where to start for a new house

Hi Cedric,

I am new to the forum too, but have been reading it for some time (just received my first order consisting of a couple emontx, a raspberry and a bunch of sensors) so I'll try and give a shot at your questions but don't feel too comfortable with the answers :)

1 - On the transmitter side I think it should definitely be possible using emontx shields and an ethernet/poe capable arduino, you would have to modify the emontx firmware to transmit the data through an ethernet connection instead of an rf one directly to your emoncms (or other data aggregator) instance

2 - Yes, as long as you are able to get the data from the sensor in the raspberry in some way (usb/serial/rfreceiver) then it should be trivial to add that feed to emoncms (especially if you already practiced with the emontx modifications to send data to emoncms over TCP/IP)

3 - If by oem you mean emoncms, then reading data directky from it is one mysql client connection/http query away (or pushing data from it is one emoncms plugin away) , but since in your scenario you would need to write your own feed generators I think it would be simpler having the transmitters feed both emoncms and the home automation system than having to write a bridge between emoncms and the home automation system (be it a custom external program or an emoncms plugin).

On a second thought since on the raspberry you can use whatever language you are comfortable with (and possibly some already written libs) while on the arduino you would be pretty much on your own, maybe a better approach would be to write a small proxy that runs on the raspberry and that feeds whatever data to whatever collector (emoncms/home automation system) whenever it is received from the emontx sensors

4 - Don't know :) you should probably be better off reseearching the possibility to interface your raspberrys to the home automation system, and from there decide how to handle the data exchange

 

Regards

Mattia

 

lunarok's picture

Re: Where to start for a new house

Hi Trystan,

 

For what I think, I don't need ethernet basestation for sensors. As I I have one Raspberry in every room, I want to use DHT22 sensors (they do temperature and humidity) directly connected by I2C to each Raspberry. And also the IPX800 can have additionnal sensors (temperature, humidity and light) So there is maybe only light sensors I'm missing for additional rooms (but a way to do it via raspberry will be nice)

 

So I want at least the emontx for power usage. But they can be all at same place (in fuse box) right ? So, if all power use can be monitor from here, network switch will be there, direct connexion possible.

So yes I will look at your 2 links to see how get a ethernet shield on emontx. There is also this article (in french, but maybe you understand it ;) ) :

http://departements.telecom-bretagne.eu/rsm/logiciels/smartb/particuliers/

The emonmeter is able to send data directly to emoncms, maybe it's possible to use part of their work to get the emontx send by network directly to emoncms ?

 

 

lunarok's picture

Re: Where to start for a new house

Hi Matt,

 

1- Yes I will look at links given before and the link from Bretagne Telecom that is using an arduino to monitor french power fuse box to send directly by ethernet connexion to emoncms.

2- In my case, I think my primary use of Raspberry it's not as a emonbase but more as a emontx (with sensor only) My emoncms will be on a server (where I already have all my other web products like UPNP, streaming ....)

3-Feeding both can be done yes, but it's better feeding only one and sharing the feed :) But if it's I will do with it. My server where will be the emoncms is also the one will have the automation suite. So sharing in this case is not a network thing, the data transit only once.

I have better feeling with Raspberry like you said I can use the language I want. But will be used only for sensors. For all arduino related stuff (emontx for power use and other pule use, and maybe emonmeter or some arduino reading telemeter for power usage) And arduino is new for me, I have wanted to get involved long time before but never get a good reason, now I have :D But as it's new, modifying firmware will be easy ?

4- Yes I look also what home automation suite can be the best on my side :) Need to have one with a url plugin maybe to read data by API or I don't know

 

Thanks

 

Cédric

lunarok's picture

Re: Where to start for a new house

Just want to add some things.

I have install emoncms (just fresh) on my server, was quick :) Like these I have the API doc under my noise.

I will try make input from DHT22 as soon I receive my sensor (so from one Raspberry first, as soon it's successfull I order 3 more for 2 raspberry and one Cubieboard) For these sensors, before knowing about emon, I already planned it (order DHT22 3 weeks ago) and was excpecting to use my Zabbix for follow up first.

 

And important : if I want to try get a emontx using ethernet, I order an emontx from openenergymonitor and what else ? (as I say, newbie for arduino)

Mattia Rossi's picture

Re: Where to start for a new house

Cédric,

it depends

if you feel more comfortable with a full arduino solution (as suggested from Trystan) then you will need a nanoderf and an emontx shield to plug directly into the nanoderf, then you will need to modify (or find one that suits your needs) the nanoderf firmware to be able to directly send the data from the sensors through tcp/ip instead of wireless

if you don't then you could refer to this forum post:

http://openenergymonitor.org/emon/node/2327

in which there is a brief explanation of how to connect the emontx (or multiple stacked emontx modules) to a raspberry through the serial interface, in this case the arduino firmware should already be available, and you would have to write the code on the raspberry to receive the data and store it in emoncms

In order to be able to start working with an emontx or a nanoderf what you need are a 5v power supply and a miniUSB cable, an FTDI cable to be able to program the nanoderf/emontx  firmware, an AC/AC voltage adapter if you plan on measuring voltage accurately and as many ct/temperature sensors as you need .. anyway, the store is really well organized and you have all this information in the page detail (what is included, what is not and is needed, and what needs soldering) while you can find all the links to the relevant firmwares in the site under the various modules pages

 

I believe there's also a short explanation of how to work with the arduino IDE and how to load firmwares in general, but I haven't tried that yet and can't really point you to it at the moment

From what I have seen of the arduino firmwares so far, if you are moderately competent with writing software and with the technologies involved (serial communications/tcp communications and so on) then you won't have any problem modifying the firmwares to suit you needs. If your experience with writing software (not using software written by someone else, but really writing it to suit your needs) is poor, then I'd suggest to stick with standard solutions at the cost of using the wireless modules, in order not to put too much stuff on the plate and not to risk buying a lot of stuff that you won't be able to put together as you like.

 

Personally, I do have cat6 cables running around the house, and a lot of experience in writing software, and I chose anyway to use standard emontx modules and using the wireless transmission to a raspberry. This way I can put one wherever I need it (one near the fuse box, one in the garage near a couple of pumps) without having to bother to run lan cables to them but only providing power (and if I really wanted I could use batteries and not worry about the power either).

Also, with the raspberry RFM12Pi board I'll be able to directly interface some weather sensors that use the same frequency (if not the same protocol) and store in emoncms also weather data without having to add other hardware. You already have the raspberrys, so all you would need is a single RFM12Pi module and you would be up and running.

 

 

lunarok's picture

Re: Where to start for a new house

Hi Matt,

 

Thanks for this link about serial connexion between Raspberry and EmonTX. That's a good way and I think the one I will like. Yesterday, I come to think that getting EmonTX with ethernet make me scared, as there is a more work to modify and they will send datas each. A way with a base is better so I get only the emonbase to send data (so easier for management) And same maybe this base getting data from DHT22 in first step and not them sending.

As the link speak of serial connexion, do you think it's possible to do it with a normal computer ? (in fuse box, it will be more the NAS/home server than a simple raspberry for me)

For exemple, I get the same mind about emonMeter (link I send before). As in the fuse box I can have already Raspberry and other emonTX, having an arduino connected directly to network for telemeter ... Don't know if it's good.

 

And yes, I keep RF as a way. But more for outside of house (less wireless in home) as you say maybe for water (don't know where it will be) and also weather stations like Oregon use this technology (and for outside sensors they are cheaper way than building an arduino outdoor station like http://arduinoweather.sourceforge.net/)

 

Ok, I see in the shop RFM12Pi it's about 20 euros, worst it for a first step with emontx and see after what can evolve.

I see also AirQuality Egg shield. 75 including CO² measure look cheap and it's interesting. But it's speaking of connect it to a NanodeRF, that I don't need if using RFM12Pi on a RPi, so is it working with the Pi as emonbase too ?

Mattia Rossi's picture

Re: Where to start for a new house

Hi,

As the link speak of serial connexion, do you think it's possible to do it with a normal computer ? (in fuse box, it will be more the NAS/home server than a simple raspberry for me)

I imagine so, the added value of using the pi is taht you already have the code for the serial communications and the hardware connections ready and working

I see also AirQuality Egg shield. 75 including CO² measure look cheap and it's interesting. But it's speaking of connect it to a NanodeRF, that I don't need if using RFM12Pi on a RPi, so is it working with the Pi as emonbase too ?

It is a shield, so it must be connected to an arduino (the nanoderf is an arduino platform with added stuff)

Jérôme's picture

Re: Where to start for a new house

Salut Cédric.

Welcome to OEM. Nice to see you here. (I'm the one that wrote the article on linuxfr.org)

Long posts. I went through that quickly.

1/ From OEM point of view, one raspi in each room is overkill. But if you have them for a good reason, why not. If you're buying a few, you may want to have a look at Olinuxino instead.

Yes, it is possible to merge emonTX and emonBase so that the TX sends data over ethernet. This is how I started with an Arduino and a emonTX shield.

2/ I maintain the python gateway script. There is a post here from someone else wanting to do something similar : using a sensor directly from the Pi, no TX involved. The gateway script can be adapted to do that. Either by adding the code that manages the sensor, or, to make it generic, by running a code to manage the sensor, then store the data in a file, then let the script monitor the file. I like the second way better because it means you can keep on pulling new versions of the gateway script without merge issues, and it makes it generic : any sensor can use it. See that other thread for details.

I haven't done any move in that direction in the script, but honestly, it shouldn't be difficult. The only issue bothering me was about the ability to name the inputs (you may want to name each sensor with the name of the room). This wouldn't be easy with the current API, but the API may evolve and anyway, it is not necessary, just a convenience.

3/ I think you can access any value through the API, without even using the database directly. If I'm wrong, then it is a good feature request for the API.

4/ As I wrote in the article, I don't think much work was done regarding automation. It is a wide (and wild) area. But I'm sure people here will be interested.

Good luck. Enjoy the WE.

 

lunarok's picture

Re: Where to start for a new house

Thank you Jérôme for your answer (and your article in the first place)

1/ Yes, I completly agree one RPi in every room is overkill for OEM. But like I say, I will have them for home automation purpose so, as they are here if they can also get some sensors, it's pure bonus.

Thank you for Olinuxino, I didn't know these products. Will try them for sure (the fact they have also touchscreen in 4 and 7" avalaible is very interesting, as I was thinking of using RPi for automation monitors as well)

2/ Yes the second way looks less intrusive, so better for me too. So on the RPi used as the gateway, get all additionnal value in a file as you described look nice. Like this can have something generic for ouput and multiple input (I mean call X RPi/olinuxino/Cubie ... for their sensors and also the IPX800 for sensors)

Maybe you can bring me light on the gateway : do I need to have my emoncms on the RPi getting the gateway ? The article here :

http://emoncms.org/site/docs/raspberrypibuild

It's not clear if I can avoid emoncms on RPi and just install the python gateway, and in this case I think I need to add the RPi module for emoncms to my emoncms instance on server right ?

3/ I have install emoncms on my server, the API doc let me think that yes every value is exposed so no need to do SQL request, just using the API.

https://myserver/emoncms/feed/value.json?id=0

4/ I have ask on domogik mailing if look possible for a plugin in this way, first return is that seems easy if API can give a list of feed and after value of a selected feed (and the API can)

Ask same for freedomotic and waiting answer. For me a couple OEM+automation software can be great. OEM for what I see is already a great tool for energy and sensors, so let it take care of all this and a good automation software must be able to handle any feeds or action, maybe how quickly and easy integration of OEM will be in domogik or freedomotic can be my answer to which one to choose :)

 

In these few days I start to read and see what is possible with OEM, I'm just amazed how it seems easy and well thinked in architecture. Congrats for this.

Is it possible to use wiki maybe for exemple to share what people are doing with OEM ? Or any other ways ? Can be great to see what people have done with it, what they have connected ... And in case like me for starters, can be a good way to let expain and maybe add a picture for sharing what expected and have ideas from people with knowledge ?

Jérôme's picture

Re: Where to start for a new house

Hi.

2/ I'm questioning the pertinence of using a buffer file as it means regular write accesses on the SD card, which wears off the cards with time.

Indeed the gateway script needs an emoncms instance to host the RPi module that sets the radio parameters. This is for historical reasons, but it makes sense to change that.

For those matters, see discussion here:

https://github.com/emoncms/raspberrypi/issues/30

The wearing off issue is discussed here:

http://openenergymonitor.org/emon/node/2292

http://openenergymonitor.org/emon/node/2312

4/ I think I heard of Domogik once on lunuxfr.org but then forgot about it. I hate to recreate the wheel, and it would make sense to interface Domogik or any equivalent to OEM instead.

I didn't take the time to dive onto it (yet) but from what I know, home automation becomes complicated when it comes to interfacing with proprietary stuff. Domogik uses xPL. I have no idea how widespread this protocol is.

Is it possible to use wiki maybe for exemple to share what people are doing with OEM ? Or any other ways ? Can be great to see what people have done with it, what they have connected

As you saw, I do that on my profile page, but most people don't. I use that profile page as a draft, a way of sharing info, and a personal memo.

 

lunarok's picture

Re: Where to start for a new house

Hi Jérôme.

 

2/ Yes, readly quickly your gateway, I think reading in a flat file is not worthing value. If the gateway must receive a new way to get data, it must be by listening on an IP. I'm not pythonist, but it's common in first exemples of python coding to show this is simple (to make easy publicity :D)

I have comment the github link.

I think the gateway as a standalone service is a good way for future. You have already init script, the main script so give it a setting files where we can set the emoncms(s) to send data and where to listen for datas incoming.

Like this maybe :

[emoncms]

serverip=xxx.xxx.xxx.xxx

apikey=*****

 

[emoncms]

serverip=xxx.xxx.xxx.xxx

apikey=*****

 

[ethernet]

listenip=xxx.xxx.xxx.xxx

 

[radio]

...

 

Like this, in future including the radio access maybe can work on other hardware that RPi

 

4/ Domogik is using xPL as a main bus. Plugins are a way to take data from other sources and send them in the xPL bus (more or less) This makes it also perfect with the xPLduino project to come (teams are working together)

The main dev has answer me, for him reading json is easy in python. There is already some plugin started for XBMC (XBMC use also an API with JSON files) They are willing to help, so as soon I have a emoncms "working" I will try with their help to start a plugin. (I have order RPi RF module and emontx with pusle sensor and current sensor today, already send. And already receive DHT22 for my Pi, waiting the other sensors)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.