Archived Forum

Hi Allen

The OEM gateway is emonHub's predecessor and there are several benefits to upgrading, but OEMG was a solid performer and therefore I couldn't say it was essential, however the IPE-R1 image that most OEMG installs were on does cause issues and is not supported, that was one of the primary reasons emonHub came about.

So if using IPE-R1 read-only I would strongly recommend the jump.

Sorry there is no emonhub installer with a built in OEMG uninstaller as there were some variations and most users had IPE, so the additional work to accommodate the possible variants for a very few non-IPE users wasn't valid.

I can help you uninstall if you wish or there are some pre-configured OEM SDcard images with emonHub on, tell us about your set up etc for a recommendation.

Paul

Hi Paul,

Thank for getting back so quickly. I'm pretty sure my setup on the Raspberry PI is standard. I downloaded the image and updated the config file with my API key. But other than that it's a normal read-only OEMG installation.

I'm not sure what IPE is, or how I can tell if I have it or not. Would it be easier just to load the SD card with a new image?

I have attached a hard disk to my PI with at some point I want to start using for a local emoncms, but I've not got round to that yet, but if I can set that up at the same time as the move to emonHub, that would be useful.

Thanks for you help.

Allen

Hi Allen

Sorry I didn't reply yesterday I was pondering over what to recommend and didn't really reach an answer so here are my thoughts.

"normal read-only OEMG installation" sounds very much like it will be IPE-R1 based.

The direct replacement for what you have now would be original emonhub as a forwarder only on a read-only Raspbian OS image. current SDcard images are read-only Raspbian and do have emonhub set up by default.

However where as earlier versions of the "previous" pre-emonPi SDcard image had low-write emoncms disabled by default, I believe this image and onwards has low-write  emoncms enabled by default. so emoncms would need to be disabled to "match" what you have now

Neither the "current" post emonPi or "previous" pre-emonPi SDcard images provide a direct route to a full OS and emoncms HDD solution, so do not help with setting up the HDD at the same time.

The emoncms install guide for linux would mostly work for the Pi but it's noy an easy route and the Raspbian OS needs to be installed and booting to the HDD first.

There is an old guide I wrote many moons ago in a thread that could be followed to use an HDD on a Pi first,

I do also have a emoncms install script but it may not work first time if it's not up to date, I never know until I try it as it isn't linked to any emoncms changes.

None of this is particularly difficult it's just not as easy as "download this" or "follow that" and I'm starting to think it maybe easier to guide you through an install rather than try and guide you through the guides, at least that way other users may find it of use too.

Paul

Hi Paul,

I'm happy to go through an install. My only worry is that I don't have a spare Pi, so really I need to get this done in one go, so as not to lose monitoring data. But it hung again last night, so the sooner the better.

Thanks for your help.

Allen

Setting up the hdd and installing emonHub is the easiest parts so if we do that first then you are back up and running while we install and set up emoncms.

Firstly Download the latest version of Raspbian OS and install to the HDD and a (spare?) SDcard and

follow Creating a local EmonCMS installation on a Raspberry Pi up to the reboot only, then install emonHub

git clone https://github.com/emonhub/dev-emonhub.git ~/dev-emonhub && ~/dev-emonhub/install

and then add emonHub command line options,

Run emonhub -c to open emonhub.conf to add your apikey and add the hash to " #url = http://localhost/emoncms " (commenting out the localhost reference will use the emoncms.org default). If neeeded edit the rfm2pi details while you are in there (this should look quite familiar to you as a OEMG user).

You can also set "loglevel = DEBUG" if you want verbose logs to check everything is ok using emonhub -dl.

That should be all you need to get you running emonhub from the HDD. 

Read-only isn't a mandatory requirement and is used predominantly on sdcards, possibly because users with HDDs are doing other stuff on there perhaps? RO can still be installed as a safety measure if you prefer, 

Paul

Hi Paul,

Everything was going fine. Ihave the Pi booting from HDD and I have installed emonhub. I editted the emonhub.conf to include my apikey. But when I start with emonhub -dl I get this output:

2015-07-07 20:21:07,500 INFO EmonHub Pre-Release Development Version (rc1.2)
2015-07-07 20:21:07,503 INFO Opening hub...
2015-07-07 20:21:07,511 WARNING Setting emonCMS apikey: obscured
2015-07-07 20:21:11,526 WARNING Device communication error - check settings
2015-07-07 20:32:40,525 INFO Logging level set to DEBUG
2015-07-07 20:32:40,529 INFO Setting emonCMS url: http://emoncms.org
2015-07-07 20:32:40,532 INFO Setting emonCMS apikey: null
2015-07-07 20:32:40,535 INFO Setting RFM2Pi frequency: 868 (8b)
2015-07-07 20:32:41,539 INFO Setting RFM2Pi baseid: 10 (10i)
2015-07-07 20:39:30,405 INFO Setting emonCMS apikey: set

My config is like this:

[reporters]

# This reporter sends data to emonCMS
[[emonCMS]]
    Type = EmonHubEmoncmsReporter
    [[[init_settings]]]
    [[[runtimesettings]]]
        #url = http://localhost/emoncms
        apikey = <apikey obscured>

Hi Paul,

Panic over! I hadn't realised that emonhub -dl just output the log. Just running it normally and tailing the log shows that it is receiving and outputting data. My emoncms is being updated.

One question. Will emonhub start automatically if I reboot the Pi, or does it need to be started manually ?

Thanks,

Allen

Hi Allen 

What baud does your rfm2pi operate at ? I assume 9600 if from the OEMG era.

Is that defined in the RFM2Pi settings in emonhub.conf?

[[RFM2Pi]]
    [[[init_settings]]]
        com_baud = 9600
    [[[runtimesettings]]]
        etc etc

Paul

It will auto start at boot if you used the install script, however it is always best to test these things just to confirm rather than finding out at a later date.

emonhub -dl does tail the log, it looks like something upset the serial port baud detection on your first attempt, defining the baud explicitly will disable the auto detection and ensure a good connection everey time.

Paul

Hi Paul,

Everything seems to be working fine with the emonhub, so the next step is to start looking at setting up emoncms. However, before I do that I'd like to take full advantage of my complete USB external drive. At the moment I have this setup:

(parted) print list free
Model:  Mass Storage Device (scsi)
Disk /dev/sda: 80.0GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
        32.3kB  4194kB  4162kB           Free Space
 1      4194kB  62.9MB  58.7MB  primary  fat16        lba
 2      62.9MB  3277MB  3214MB  primary  ext4
        3277MB  80.0GB  76.7GB           Free Space

So you can see there is a large amount of free space. Would it be best to create another partition to use this space and install emoncms on that, or would it be best to extend the existing ext4 partition to use this space (in which case what is the best way to do that) ?

Thanks,

Allen

The best way to configure the hdd space is to separate the filesystem and data files. I usually install emoncms to the same partition as the OS but move the mySQL and data folders to a "data" partition like the waty it's done on the pre-configured images. This can (should?) be done regardless of whether you go for read-only OS or not.

It's entirely up to you if you use ALL the space in one go or not, It can make things a little easier to manage to keep the partitions smaller and increase the size as needed but it's a minor preference. If it were me, I would just create another small(ish) partition leaving plenty of room to expand the existing one eg a 5GB partition starting 20GB into the hdd ending at 25GB.

​fdisk is probably the easier tool to use, gparted will also do it but I'm less familiar with it.

In fact most of this is covered in the SDcard "built it yourself" guides, using fdisk will be the same (different size and position) but you won't need to format ext2 since it's not on a SDcard.

​I've been meaning to retest my installer on a spare Pi for you, it makes installing emoncms a one line command, I will try and do it over the weekend.

Paul

Hi Paul,

I have now followed the remainder of the instructions on this page http://openenergymonitor.org/emon/node/5092. I now have emoncms running on my Pi and a new data partition. How can I move the mySQL database to my data partition ?

Thanks,

Allen

Hi Allen

I have answered your other Newbie issues with emoncms thread about the emoncms versions etc. 

If it were me I would start again, setting up the hdd and reinstalling emonhub (you could keep your existing emonhub.conf file) will be less painful than chopping things around and provide a known state to build the correct emoncms version on.

As mentioned above "The emoncms install guide for linux would mostly work for the Pi " if you wish to install step by step but there is a change in the server root folder and it doesn't automatically install to 2 partitions.

I have had a look at my installer script and although I would like to make a couple of adjustments I believe it will work as is ( I've not tested recently)  to test it I need to free up an sd card to test it as they all currently have partial projects on them.

The script will automatically move the SQL and other data files to "~/emonData" your data partition should be mounted to that point before running the installer. (this is the default and can be altered)

Paul

Hi Paul,

Your script seemed to be working okay, until it tried to create the database:

Cloning into '/var/www/emoncms'...
remote: Counting objects: 8958, done.
remote: Total 8958 (delta 0), reused 0 (delta 0), pack-reused 8957
Receiving objects: 100% (8958/8958), 4.59 MiB | 916 KiB/s, done.
Resolving deltas: 100% (5283/5283), done.
ERROR 1006 (HY000) at line 1: Can't create database 'emoncms' (errno: 2)

The password used by emoncms to access database is 6335f188ead949c7b380dfc715ab297f

PHP Warning:  Module 'dio' already loaded in Unknown on line 0
Channel "pear.apache.org/log4php" is already initialized
PHP Warning:  Module 'dio' already loaded in Unknown on line 0
log4php/Apache_log4php is already installed and is the same as the released version 2.3.0
install failed

Do you know what might be causing this?

Thanks,

Allen

I found a page that suggested a restart of mySQL. When I tried, this is what I got:

pi@raspberrypi:~$ sudo service mysql stop
Stopping MySQL database server: mysqld.
pi@raspberrypi:~$ sudo service mysql start
df: `/var/lib/mysql/.': Too many levels of symbolic links
df: no file systems processed
/etc/init.d/mysql: ERROR: The partition with /var/lib/mysql is too full! ... failed!

So I tried this:

pi@raspberrypi:~$ df -hk
Filesystem     1K-blocks    Used Available Use% Mounted on
rootfs           6172412 2693876   3177516  46% /
/dev/root        6172412 2693876   3177516  46% /
devtmpfs          218636       0    218636   0% /dev
tmpfs              44584     264     44320   1% /run
tmpfs               5120       0      5120   0% /run/lock
tmpfs              89160       0     89160   0% /run/shm
/dev/mmcblk0p1     57288   19448     37840  34% /boot
/dev/sda5        9948012   22632   9413380   1% /data

And this:

pi@raspberrypi:~$ df -hk /var
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/root        6172412 2693876   3177516  46% /

When I just try this:

pi@raspberrypi:~$ df -hk /var/lib/mysql
df: `/var/lib/mysql': Too many levels of symbolic links
df: no file systems processed

I get a similar message to the one when starting the mySQL. The symbolic link is like this:

pi@raspberrypi:~$ ls -l /var/lib/mysql
lrwxrwxrwx 1 root root 11 Jul 13 23:01 /var/lib/mysql -> /data/mysql

When /data is my new data partition on the hdd.

The symbolic link on /data is:

pi@raspberrypi:~$ ls -l /data/mysql
lrwxrwxrwx 1 root root 11 Jul 13 23:34 /data/mysql -> /data/mysql

​Something looks wrong!

Hi Allen,

Something certaily isn't right there. somehow you have a symlink pointing to itself  "/data/mysql -> /data/mysql", and since you also have a symlink "/var/lib/mysql -> /data/mysql" that raises the question where is the mysql folder?

Going by the number of messages containing "already" in that latter part of your post before last, I would hazzard a guess and say either you have run this script more than once or there were reminants of the last install still hanging around.

Did you start a fresh or uninstall the apt-get installed emoncms?

Was the data partition mounted when you first run the script?

If you add the data partition to the fstab it will be mounted automatically on boot, is /dev/sda5 correct? you have 5+ hdd paritions or does a new id being allocated to the same partition?

Paul

Hi Paul,

I've solved it. I was because I ran your script a second time and didn't move the mysql directory back from my /data partition.

Allen

excellent!

Hi Paul,

I have downloaded the data from emoncms.org using the backup.php script. However, when I try to visualize one of the feeds I get this error:

Warning: fopen(/var/log/emoncms.log): failed to open stream:
Permission denied in /usr/share/php/log4php/appenders/LoggerAppenderFile.php on line 99

I think this is because in your script it has:

sudo touch /var/www/emoncms/emoncms.log
sudo chmod 660  /var/www/emoncms/emoncms.log

But this location for the emoncms.log file does not match the error message I got. I also had to chmod 666 to allow access to the log file.

The error I am getting is this:

PHPFina:get_meta meta file does not exist id=26295

When I looked back at the output from backup.php I can see these lines:

Create feed 26295
PHPFINA: 26295
Cannot open local metadata file

The metadata files are on my /data partition (although I'm not sure it's using them). The look like this:

drwxr-xr-x 2 www-data root   4096 Jul 13 23:59 phpfina
drwxr-xr-x 2 www-data root   4096 Jul 13 23:59 phpfiwa
drwxr-xr-x 2 www-data root   4096 Jul 13 23:59 phptimeseries

​Although I have noticed I have the same directories in /var/lib. My emoncms settings.php is definitely pointing to the /data directories and in the backup.php script I set:

$link_to_local_emoncms = true;

Edit - wrapped long lines - BT

I had to do what was suggested in this post http://openenergymonitor.org/emon/node/5495 and chmod to 777. It's a shame that old post doesn't mention if they fixed the problem.

Hi Allen

The logging was changed recently in v8.5 so that's why the address is different(to the script), although /var/log's default permissions will probably hamper any direct logging, the normal practice is to use a app specific folder with appropriately set permissions. This has probably been done to suit the read-only configuration for the low-write version. This will need a better solution long term, both in the source and then any installer.

The meta file permission issue is more likely to be that the user "pi" isn't able to access the meta-files when the php script is run, Have you tried using "sudo" ? setting 777 just opens all the permissions to everyone.

Paul

Hi Paul,

Using sudo fixed the permissions problem (with the directories set back to their previous permissions).

Thanks,

Allen