Showing posts with label Synology. Show all posts
Showing posts with label Synology. Show all posts

October 11, 2016

Quick and Dirty Install SyncThing in Debian with a Systemd Service

Sync
I really like BTSync (aka “bittorrent sync” aka “Resillio”). It was a god-send considering a wanted to sync files between multiple computers and NAS, but did not want to store them in the cloud (read as: These are not he DropBoxes you are looking for.) I first tried AeroFS for a short stint, but the vendor soon shifted focus from individual to team/business oriented support. BTSync was released during this time.

I’ve been using BTSync for 2 years now. I stayed at v1.x all this time. BTSync is exactly what i want and has been working great.

When I recently had my work notebook replaced, I decided to “upgrade” BTSync to v2. I seem to likeed it more – the interface worked a little better, so i decided to upgrade BTSync on my Synology Diskstation to v2 also, but it failed to run at all . I seem to be stuck with v1.4.11. Let me be clear, v1 is not bad; It does work. But sometimes, i just can’t live with older, obsolete, and unsupported software. Also, I don’t believe the bittorrent team are freedom-hating, but it is certainly closed-source.

I’ve researched the file-syncing subject many, many, times. On several occasions, I was super close to trying open-source (freedom-loving) Syncthing, but at various times it was not quite ready. Well this time, after looking at it again, it was ready. And so was I.

So here is the quick and dirty install for debian/ubuntu with a systemd service:
(Note: If you are not using systemd, or are using an alternate OS, then there are other service options. )
# install syncthing as per http://apt.syncthing.net/ instructions
curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
echo "deb http://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
sudo apt-get update
sudo apt-get install syncthing

# create new service file via syncthing's systemd "system" file:
sudo curl -o /etc/systemd/system/syncthing@.service https://raw.githubusercontent.com/syncthing/syncthing/main/etc/linux-systemd/system/syncthing%40.service
sudo curl -o /etc/systemd/system/syncthing-resume.service https://raw.githubusercontent.com/syncthing/syncthing/main/etc/linux-systemd/system/syncthing-resume.service
# enable and start the syncthing per-user service user=$(whoami) sudo systemctl daemon-reload sudo systemctl enable syncthing@${user} sudo systemctl start syncthing@${user} systemctl status syncthing@${user} #launch web-UI xdg-open http:127.0.0.1:8384

To install on Synology DiskStations, use the SynoCommunity package repository.

I won’t go into the setup, but some key things are the following:

First, be sure to set a “GUI Authentication User” and “GUI Authentication Password” via the Actions>Setting menu, then restart the service.

Next, it is important to understand that a Syncthing relationship is different than BTSync. Rather than a single key that enables syncing as with BTSync, in Syncthing you will add “devices” to each system and specific share folders. You are required to add these “both-ways“. For example for two systems to sync, you will add device A to your device B, and vice-versa. If you wish to sync between three devices, device A will need both device B and C added; device B will need both device A and C added; and finally device C will need both device A and B added. During this, you may need to create the folders before-hand, and pay special attention to the folder ID (key) provided by Syncthing, as it must be the same on each end-point.

Also, I was certainly having some sync/connection issues, and after much troubleshooting, it seemed IPv6 was stopping IPv4 from being the preferred connection method. There is no documentation stating a defacto way to disable IPv6, but after reading https://docs.syncthing.net/users/config.html, i tried setting my “Sync Protocol Listen Addresses” to simply tcp://0.0.0.0, which seems to be the equivalent of saying “use IPv4 on all interfaces.” I am still uncertain if this is necessary or even proper, so use with caution. In fact, I'd recommend do not do this yourself.

With both BTSync and Syncthing, the software repeatedly scans files for hashes, and if changed, will sync updates.  These computations are a heavy burden on an ARM based NAS such as the Synology. Syncthing's default scan interval is 60 seconds.  Every 60 seconds it will scan a sync-folder for changes -- this caused my Synology to continuously thrash the disks.  In an effort to reduce this heavy activity, i went to the advanced settings, chose my folder(s) and set rescanIntervalS to 900 seconds (15 minutes). Since my Synology is more of a backup device and I do not edit these files directly, this is a much better sync-interval.

You should certainly visit the Syncthing Documentation for your needs. Also, You may or may not need to open or port-forward TCP 22000 and UDP 21027 depending on your network environment.

As always, Good Luck!

- - -


Please consider crypto tipping:
  

October 03, 2010

The inevitable happened - total data loss



You've heard it a thousand times. "Backup your data!" Well when you have a 400Gb harddrive filled and nowhere to backup to, you're screwed.

One fine day I figured I'd play with my BIOS settings for no apparent good reason. I'm searching around and see this nifty drive chipset setting that was disabled. The options were Disabled, RAID, and SATA. Well I had 3 SATA drives plugged in, so why is this thing on "disabled"... hmmm let me change that. I rebooted and my machine hung. I reset, my machine went a little further but kept complaining that a device was not ready "(Ready=0)". Well let me disable, I figured. I rebooted and again it hung. What I had done was mistakenly enabled SATA power to my drives when they already had MOLEX power supplied. It turned out that this can be (and definitely was) a very bad thing.

Luckily my 320GB drive that stored "My Documents" among other things was safe, but my drive containing 4 years of collecting Martial Arts videos was dead. I also had quiet a few CD backups (ISO) -- about 360GB of data is lost. I tried several software recovery options intermittently over a few days; however, it was a complete fail.  I checked into data recovery options and the rates are phenomenally and outrageously expensive. It's a "no go" -- I'm not paying a grand to restore what is 80% re-downloadable and truth be told, i'd never have the time to review all those martial videos anyhow. I'll just find the important ones again. Yes, some of the data is irreplaceable. I had a backup ISO containing files my father created. His "My Documents" and CadKey drawings. My father passed away in 2004. I backed up the data and wiped the computer. It was irreplaceable data that I'd preferred to keep.

So this post was written for two reasons: One, to frighten you into backing up your data! Two, offer some backup solutions.

I had, in the past, wanted to get a "real" NAS (Network Attached Storage) device. I had a FreeNAS running, which is very nice, but it required me to keep an old Pentium 4 1.2Ghz PC powered on 24/7 with only 80GB+30GB storage -- not nearly enough. I could have gotten a replacement drive, whether internal or external, but what happens when that fails? Total loss again -- and drives do fail. There is also services like carbonite.com offering unlimited backup for $55 a year, but how many continuous days do you think it would take to upload hundreds of gigs of data?

I was aching to get a NAS. There are many brands.  Some too cheap, some too expensive. I kept researching newegg.com. I had some limited experience with iOmega's ix stores.  Very nice devices, but they were a bit overpriced and I understood the drive replacements can only be bought from iOmega. I ultimately decided on a Synology product that had good feedback on newegg.com and has some good features. The product I wanted had to have a high capacity and fault tolerance. I did not want a single drive solution. I wanted to mirror or RAID. I could have chosen a two drive solution for mirroring, but I felt that expandability was necessary. My final choice, although still very pricey, had room for growth and steered clear of the truly over-priced vendors.

My latest purchase is a Synology DiskStation DS410j. It is a 4 bay personal NAS unit supporting up to 8TB, yes eight terabytes by using four 2TB drives. Although in the past I've always opted to buy Western Digital drives, Newegg's customer feedback reported many dead-on-arrival WD's, and Seagate's, and just about every other brand. The best feedback for 2TB drives was for Samsung. I don't particularly like Samsung, but the feedback spoke for itself. My new drives are the Spinpoint F4 2-terabyte, eco-green, 5400, 8.9ms drives.  They are not the fastest, but i knew this was a for the NAS unit which was mostly limited by the network speed.  So I am a proud owner of a Synology 410j with two 2TB mirrored drives -- I just couldn't afford four drives. Of course, I can always add a drive or two, and as I understand it, seamlessly upgrade to RAID 5.  The 410j was discounted to $329 and the drives were $119 each.  StorageReview.com has a nice write-up on the DS410j.


For those of you that in no way can afford to drop nearly $600, I'll share some other options with you.  As mentioned prior, Carbonite.com allows unlimited online backup for $55 with the caveat of requiring the time of uploading.  This can be hellish on DSL or cable.

For those wanting a free online solution, I found Adrive.com which allows 50GB of free online storage, with the caveat of uploading via a java interface on a web-page.  There is also Megaupload.com with 50GB free online storage that also allows a browser toolbar or windows application for uploading.  Again, with these services, upload/download speeds are limited by your type of internet service.

The cheapest local storage option would simply be to purchase an external USB drive, but of course there is no fault tolerance.  Although unlikely both your computer's harddrive and your external drive would fail at the same time, it is possible.  But for that matter so is failure of both drives in my NAS.  For me, I am more comfortable with having a dedicated unit for backups.  It also allows me to setup my family's machines to auto-backup -- A great convenience to save invaluable information.


Please consider crypto tipping: