For most of my parents lives, most of what they had to worry about keeping track of or safe was contained in a filing cabinet and a photo album. As a 25 year old, just starting my own journey, I already have over a terrabyte of hard drives in a drawer, as a temporary "archive". My current "active set" of data is well over 300GB at this point, and with media that number swells to over 1TB.
One of my first avatars that I used online included the phrase "I Live Online". I've always considered myself amoung the first generation to grow up online - I can barely remember a time without a modem in the house, and the web was fully integrated when I entered the workforce. This got me thinking - how can I best set myself up for keeping all of it over the next 50+ years? I want to be able to show people pictures from my climbing career, which are already a decade old, and I nearly lost already. I also want to get my data to a point where I finally know where all of it is, make sure I have ubiquitious access to it, and make sure I have been backing it up. I've spent the last month or so re-thinking how I maintain my digital footprint, and have ended up with the below.
The Core
The hub of my digital footprint, The Core is the set of data and services that run my life:
- Contacts
- Tasks
- Calendar
- Financial
- Media
- Digital Assets (i.e. code, notes, documents)
- Printing
- Whatever else may come up
These data silos are unlikely to change, and form the core of what I will want to care about on this time scale. Right now, my main apps are all cloud apps - Google services, Evernote, Wunderlist, Dropbox - and The Core versions are mainly acting as backups. I am slowly moving to something closer of a 50/50 mix, at getting each service to the point that it could serve the full role. This has the added benefit of getting everything into the most basic and transportable formats.
The goal is not to use 100% self-hosted services. I am not doing this out of a fear of bad actors going after my data, but rather to figure out a personal infrastructure for the next 50 years. I plan on using Evernote until the day they turn off the lights. I like some of the features of GDocs. I won't use a service without the ability to export from it, so that I can keep it backed up, and migrate to a different/self-hosted sercice.
The Hardware
Right now, most of this is going to be handled by the server described in A More Personal Cloud. That server is currently a VPS at Linode, but the long-term plan is that this role would be filled by a server at home. Since apartment life makes that fairly sub-optimal, VPS it is.
The Media duties are currently being handled by a raspberryPi, running Raspbian, which has a 2TB drive attached to it. I've gotten most of my media duplicated to it at this point, but haven't gone as far as to delete the originals yet. I still need to setup the final backups for it (which will be outlined below), and I dont know if I want to fully nuke it until I get the media to their final home.
Since things like Bit Rot are actually non-trivial on the scale we are describing, one part of this is getting files onto filesystems that have built-in integrity checking. My motto has been "ZFS ALL THE THINGS", and until I get that bare-metal server in the basement, where I can stick a FreeNAS box for some RAID fun, I'm going to rely on backups if anything really chaotic strikes.
When I can move to the home server setup, it will take over all of the above ,aside from email. I still really like Fastmail.fm, and have no real plans of moving. It does run offlineimap to have a live backup of all my mail, in case I do need to move for some reason. It (or something else on the rack) will also gain some home automation/stats/infrastructure as well.
There are still two pieces I need to figure out - what to use for tasks, and what to use for note. Currently, they live in Wunderlist and Evernote, and are getting exported and backed up. I'm fine relying on the cloud services, but I still want an analogus service in place. I also need to move my printer into the office, and put it onto the samba share.
The Network
It's worth spending a minute talking about the network. By day, I spend part of my time as sysadmin. This has started to leak into my home life, and then I found the Homelabsubreddit. I've landed on the opinion that there is no reason to not approach my home network like it is a SMB network. So, rather than just setting up a 802.11 router and calling it a day, I have moved everything in my apartment that can be hardwired onto Cat6. This included running 75' of it from the router to the switch in the office, where it then goes to my desktop, Steph's desktop, and the raspberryPi. Somehow, she is cool with it.
The raspberryPi is going to be serving as the VPN entery into the network, which will then enable access to all of my media from anywhere. (Currently, it is limited to the internal network, because I wanted to go to a barbeque rather than setup a VPN last weekend). Since the rest of the data lives on the VPS, I already have remote access to all of it.
The long-term plan is to host a rack in a server closet at home, which will manage the network (firewall, switching, patch, routing, VPN), which will then run WiFi APs for the mobile devices. Long, long term, I would like to have ethernet run next to every outlet, which will all have the covers with built-in USB ports.
Backups
Backups are the first line of defense for the digital native. I am paranoid as hell about losing data, after it happend once in college. Currently, my main backup tool is CrashPlan. CrashPlan has a sweet feature that lets you backup to other computers, which is an easy way to get your data backed up to other locations. Currently, I back everything (servers, computers) up to my desktop PC, which in turn gets backed up to the CrashPlan cloud. I also regularly dupe the drive in the machine, and keep that in an off-site location.
I backup any service I care about enough to include in The Core as well. This currently includes Evernote, Wunderlist, and my Google account.
Since "2 is 1 and 1 is 0", I also hedge my bets. I use tarsnap on my servers, and run my main file systems out of Dropbox and Google Drive. My core files are also on my fileserver.