Dropbox backup

----------------------------------------------------------------------------------

UPDATE 3/4/16 - see this thread for updated script
_______________________________________________

For a week or so, I've been testing a script to backup my emoncms data & database dump to Dropbox for free, safe off-site storage, and it seems to be working extremely well.

The bash script is written by Andrea Fabrizi and is very easy to install and which guides you in setting up the Dropbox API which is normally quite difficult, but with this script I was up and running in 10 minutes.

Once the script is configured, it is run with a simple syntax;
$ ./dropbox_uploader.sh upload <source directory> <remote directory>
which can be run from the command prompt, by a cron job or (as I do) by using node-red, to run the script at defined times.

Uploading to Dropbox is quite slow, and takes me about 10 minutes to upload 60Mb of data, so I'm using a 'staging directory', where data is stored awaiting upload.
The feed & database dump is copied to the staging folder by using the 'usefulscripts' backup and mysql_backup tools which again are run at regular intervals by node-red.

This could be further developed;

  • Combining the scripts to create an emoncms cloud backup app
  • Version control, to compress the backup files into an archive, with a time/date filename, so several version are available to roll-back.

Maybe something similar to this.

Paul

pb66's picture

Re: Dropbox backup

Nice find!

I was using copy.com but that service is ending in May, dropbox was never that easy to use from a Pi so I will give this a whirl. Shame they are not as generous with their free space as copy but I guess I shouldn't grumble when it's free.

Paul

Paul Reed's picture

Re: Dropbox backup

Me too, that's why I'm looking for an alternative.

borpin123's picture

Re: Dropbox backup

I'm guessing you compress the files? They are just text, so compression should be significant.

Don't understand the 'dropbox slow' comment.  You just save a file to the folder you have told DB to sync, and it should just do it.

If you have a folder setup for DB to sync, could you use the emoncms backup script and save the resulting file to the DB folder?  You just need a backup strategy to rotate the file names.

Not sure if the emoncms backup script compresses the resulting files or not.

I'd thought of this (I used a similar idea to do it for my weather data) but had not got around to trying it.  The script to set up DB should help :)

borpin123's picture

Re: Dropbox backup

Actually got around to reading up on the backup module.  I think it would be best to run the backup module script  from a cron job, configured to send the backup to a dropbox sync folder.  You might also need to write something to clean out old backups.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.