Emoncms - SD cards / usb flash / spinning disk recommendations?

There has been a lot of discussion on the forum as to the suitability of SD cards due to the limited number of writes it can do, but the individual experience seems very varied.

I have 18 raspis running emoncms since January and have so far had 5 SD card failures (corruption which is not fixed by reimaging) - I'm using Sandisk Ultra 8GB (a class 10 card). One of the cards lasted less than a month. I'm told that some SD cards have better 'wear leveling' than others, anyone have any recommendations for replacement?

Barry Broom has recommended using a non-journaling file system such as ext2 to decrease disk writes, so I'm giving that a go on a couple of replacements now - will feedback results.

I've also seen recommendations on pi forums to move the swap and log files to a usb flash, obviously they have similar limited write problems, but are cheaper (so an identical spare could be left with each pi) and with better write leveling, couldn't that extend SD life dramatically? Are there other things we could do in software such as increasing cache size or even removing caching?

I understand the ultimate solution would be a read only boot partition on the SD and a usb spinning disk for data logging - but most of the drives available are overspec'd (250GB+) and too expensive to roll out over multiple installations. Does anyone have recommendations on cheaper hard drives? The ubiquitous BIFRA are the cheapest general usb but seem poorly made with secondhand disks. I'm also concerned about power draw from the pi as I don't want to use seperate power leads or powered USB hubs, anyone got something already working?

Sorry for the barrage of questions, I feedback results as I get them.

Mx

Ian Eagland's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hi

I don't think you will have any success without a powered USB Hub. I tried to get a hard disk running without a hub and ran into problems. The Pi kept hanging. I had another Pi with a powered USB hub that also powered the Pi off a very short gold USB lead. This worked perfectly without a hard disk. I then added the hard drive and the Pi started hanging. I had to end up with 2 power supplies. One for the hub and one for the Pi. I was trying to use a single power supply as my setup runs off a 24V solar system and I wanted to use a single 24V to 5V  DC to DC adaptor. Having searched online it seems many others have found that for reliable operation you have to use a powered hub on a supply  independent to the Pi.

The one thing I intend to try is running the hub and Pi of a single DC DC adaptor each with their own individual connection as the DC DC adaptors I use handle up to 2.5amps.

Regards

Ian

 

glyn.hudson's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Interesting to hear your experiences Matt. It's a good question you have raised. I know this would defeat the point of using a Pi but it would be possible for the Pi to relay the data straight to emoncms.org or another emoncms server without also logging locally. This will require an edit to the Raspberry Pi emoncms module. 

Jérôme's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Yes, this is the approach I was suggesting here : http://openenergymonitor.org/emon/node/2292

This might make the Pi less interesting for a large scale usage, if a Nanode is a simple solution.

But it still has pros : possibility to use it for other tasks as well, remote access through ssh to work on it, even reprogram the RFM2Pi since v2.

I could try to propose a patch, but I think it would make sense to rethink the logic of the module and its GUI (see the other thread mentioned above), so I wanted to have other opinions before diving into it. Besides, it is quite possible that I'd need a bit of help with the php. If Trystan is interested, I'd like to have a design discussion before going down to the code. Perhaps github is a more advised place for these discussions.

BTW, I heard about Olinuxino, which could be a cheaper alternative to the Pi with equivalent features. Not sure what it implies for the RFM2Pi. After all, it is just serial communication.

matt-carbon-coop's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Thanks Ian, that's really useful information, so any spinning disk model needs to factor in powered usb hub cost and socket availability. It certainly makes the system more intrusive in the living room.

Jérôme this is a very interesting proposal, I'm also doing the data collection and analysis on a remote server, though personally I like the local server 'backup' of feeds for when the internet connection goes down (alarmingly regularly in some cases). I agree even without this there are some advantages of the pi over the nanode. Olinuxino is an interesting prospect if only for its greater openness.

Just spoke to Trystan about this, it seems they've not had problems with SD card corruption using a lower cost Sandisk than the one I'm currently using - which is positive. He also noted that v5 of emoncms executes far fewer mysql queries which may help. One suggestion he made was that perhaps all logging for apache, mysql etc should be turned off by default with documentation on how to enable them for troubleshooting.

With wear leveling on an oversized (8GB) card should be lasting a good deal longer - if we're looking at 2 years+ as people on the raspberry forum suggest, that wouldn't be a problem. In terms of Sandisk SDs only the 'Extreme Pro' specifically mentions wear leveling, there have been questions on the Sandisk forum about which models support wear leveling and specific requests for documentation but they have not received replies.

I understand the swap file is not used greatly under raspbian, but I'd like to start looking at that and perhaps moving to a swap partition on usb flash - as it's faster, cheaper, better documented (in terms of static wear leveling capabilities) and could wear out without immediately bringing the system down.

Thanks for you replies, I recognise that my use case - multiple geographically disparate installs without local support - is a bit marginal, so I really appreciate you taking the time.

Ian Eagland's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

As an update to this I have just modified my DC DC converter and wired in a chopped off micro usb to power the Pi. So I have 2 connections. One is plugged into the PI and the second is plugged into the USB hub. I have a wireless Wifi, a pen drive and a hard drive all on the hub. It has all booted up without any problems and is running as I type this. I will report back if I have any long term problems. It suggests there is a market for a Wallwart  with 2 leads, 1 a plug and 1 a microUSB.

Or maybe a small junction box with multiple connections.

Regards

Ian

Paul Reed's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Ian, I bought one of these which works quite well.

Paul

TrystanLea's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hello, As Matt and Jerome mentioned above I think the best way to start is to disable all the logs as default, apache access log, apache error log, mysql, pi gateway script log, this should reduce writes as much as possible and preserve precious disk space, then as you mentioned Jerome have the option for remote forwarding only without any local inputs being written to, although I think this happens already if no inputs are configured (apart from the initial write that creates the input). We may be able to reduce further writes with continued optimization of input processing to reduce number of mysql queries which will be good for performance in general.

There is also the complicating factor of what happens when a pi is powered up and down without a full halt, restart, could the corrupted cards that are going after a short time be due partly to this? We corrupted a card here today by accidently shorting the pi 2 or 3 times. The SD card did work again though after a fresh image was put on it...

Jerome: I thought Id remove the default logging from the php gateway script tomorrow, I wonder whats the best way to configure it to turn logging on if needed? shall we work on this and then update the documentation with steps for turning off all logs as default and enabling only for troubleshooting purposes. Id be happy then do an an updated image with these changes.

Trystan

Ian Eagland's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hi Trystan

My 2 SD failures (after several months in each case) occurred on a Pi that has never been powered down as far as I know. Running from a battery solar system and the controller logs have no power shutdowns recorded. 

Regards

Ian

matt-carbon-coop's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hi Trystan,

Data corruption due to power loss is an interesting point, I was looking at that very issue yesterday when assessing whether to turn off Journaling on the ext4 file system - which would reduce writes but make the data far more vulnerable in the case of power loss. I was also reading that types of static wear leveling can make the data more vulnerable on power loss but I don't think these apply in our case, very readable explanation of some of the issues and tested cards here.

I can't be assured that pis haven't had the power pulled, but on 2 of the cards re-imaging the cards gives disk errors and on a third the SD fails again after a few hours, which makes me think it's a hardware issue rather than just dataloss. None of this is more than anecdotal though I'm afraid...

Mx

TrystanLea's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Made some progress on disabling all logs as default, just creating a new ready to go image for download at the moment.

So for existing installations:

Turning off apache logs

I've documented all the steps in the raspberry pi build from scratch guide here http://emoncms.org/site/docs/raspberrypibuild

There are 3 different files that need to be edited to turn off all apache logging:

1) In apache2.conf:

sudo nano /etc/apache2/apache2.conf

Replace the ErrorLog line:

ErrorLog ${APACHE_LOG_DIR}/error.log

with:

ErrorLog /dev/null

Comment out the line:

# LogLevel warn

2) In /etc/apache2/conf.d/other-vhosts-access-log

sudo nano /etc/apache2/conf.d/other-vhosts-access-log

Comment out:

# CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined

3) In /etc/apache2/sites-enabled/000-default

$ sudo nano /etc/apache2/sites-enabled/000-default

Comment out:

# ErrorLog ${APACHE_LOG_DIR}/error.log
# LogLevel warn
# CustomLog ${APACHE_LOG_DIR}/access.log combined

Restart apache to complete:

sudo /etc/init.d/apache2 restart

Turning off rfm12piphp logging:

I've updated the bash script to take an argument that turns logging on and off so the first step is to download the latest raspberrypi module from github

cd /var/www/emoncms/Modules/raspberrypi
git pull origin master

to update exisiting module installed via git.

Next re-copy the rfm12piphp bash script to /etc/init.d:

sudo cp /var/www/emoncms/Modules/raspberrypi/rfm12piphp /etc/init.d/
sudo chmod 755 /etc/init.d/rfm12piphp
sudo update-rc.d rfm12piphp defaults

To complete restart the rfm12piphp script:

sudo service rfm12piphp restart

Check that the logging is all off

If you call any of the following they should either report a missing file or if the file already existed before turning logging off, no new entries should appear in the terminal window:

tail -f  /var/log/rfm12piphp.log

tail -f  /var/log/apache2/access.log

tail -f  /var/log/apache2/error.log

tail -f  /var/log/apache2/other_vhosts_access.log

You may want to delete any exisiting logs to save on disk space:

sudo rm /var/log/rfm12piphp.log

Both the apache access log and the rfm12pi log are frequently updated logs so turning these logs of should reduce disk writes significantly and also freeup disk space. 

Jerome, would you be able to implement this feature in the python gateway and document how to update and turn off the logs?

Il update when the ready-to-go image is up.

Trystan

 

Ian Eagland's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hi Trystan

Which directory do you need to be in when you do the git pull origin master  command?

Regards

Ian

TrystanLea's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

a yes:

cd /var/www/emoncms/Modules/raspberrypi

Il copy that in above

Jérôme's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

> Jerome, would you be able to implement this feature in the python gateway and document how to update and turn off the logs?

In the init file, there's this line:

DAEMON_ARGS="--logfile /var/log/rfm2pigateway/rfm2pigateway.log"

If replaced by

DAEMON_ARGS=""

the logs go to standard output. I suppose this is /dev/null if launched as a service. We could check that.

Besides, the .py gateway is designed with more granularity in the logging. I was planning to add a multiple-choice selector in the GUI to allow for the choice of the logging level : DEBUG, WARNING, INFO, CRITICAL, NO, etc.

I haven't done it yet because it may be confusing since the php gateway does not have this feature. We could consider all messages in the php interface are of level DEBUG and modify the php script accordingly...

Maintaining both scripts not only means  more work but sometimes it can slow down the pace, as each modification will have to be done for both and were the logic differs, it can be painful.

The other reason it is not done yet is that I'd like to propose a new design for the Raspberry Pi module, with

- a part for the radio config and emonGLCD sending and other general settings

- a part for the emoncms target servers (any number of servers, local or remote treated the same way, local being just a particular case)

- a window for the log to be displayed, as you suggested

I think this is worth doing in a dev branch.

I should try to produce a pseudo-screenshot of what I'm thinking of and we could discuss this before I start coding.

I'm being out of topic. Where should we discuss this ?

 

TrystanLea's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Hello Jerome, I've opened an issue in the raspberrypi github repo to discuss further https://github.com/emoncms/raspberrypi/issues/30

sebastian.gallehr's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

We have tried a lot of different SD cards incl. class 10 SanDisk on a non gateway Raspi based emonCMS. We want to have local dashboards so it will not be sufficient for us to use the gateway approach. After app. 3 month all SD cards where dead exept one:

Transcend Ultimate-Speed SDHC Class 10 UHS-1 8GB

Petrik's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

 

After changing the polyfuse I am able to run a 3.5" 500mb usb hard disc without a need for extra PSU.

 

TrystanLea's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Petrik: thats interesting, could you share some pictures of what you did?

Petrik's picture

Re: Emoncms - SD cards / usb flash / spinning disk recommendations?

Changed polyfuse to 3A version, if I remember correctly the original is 700mA - polyfuse resistance increases when current limit is reached which in practise means that voltage is dropping too low for disc to operate. With 3A polyfuse disc seems to be operating as the polyfuse current limit is not reached.

Bought from ebay link below, these are a bit bigger than original - but could be fitted.

http://www.ebay.com/itm/350680862214

Here is a piccie...

 

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.