FreeNas build for medium loads.
I build this NAS as a central file and backup server for a small business environment.
The actually build was pretty straight forward. The case is modular so that you can get the complete drive cage out to access the side fans and create more room for the board and PS installation. The drive cage has a backplate with power distribution and additional fan headers. Not sure where to find the space to put more fans but if you do its just plug and blow. To create more space for cable management I removed the front IO with two USB and Audio in and out. For one there is only one header for two USB 2.0 on the board and I needed that one for the Boot USB drive. I didn't want the Boot drive to be plugged in the back or worse the front. So I got a 9pin header to two USB 2.0 cable for that. I also removed the 2.5" drive tray in the back to make more room for the cables. Everything will fit with the tray but not as elegant. You can physically lock the power button in this case so you have to open the front door and press the button underneath. I went one further and used the reset button as power button so you can't reach it at all without opening the door.
If you want to build this config a couple things I realized along the way:
The airflow in the case is good but you should consider a CPU fan if you are running heavy loads. I reach 30-35 degrees C with low to medium loads in a cold (15C) environment.
There was a silicon problem with the C2xxx Intel Atom Series that would brick the board after about a year of use. Make sure to check that you have revision V1.03 to be clear of that. Mine might be affected so im currently gambling. AsRock support is super helpful and worked out a extend warranty with Intel to replace affected boards. But if you buy the board now you should get the new and unaffected stuff.
clearly mark your drive bays and make photos with the drive SN and Bay number. It helps a lot setting everything up and replacing drives along the road. 8 can get confusing trust me...
RAM is the backbone of ZFS. If you want raw performance consider 64GB. Or at least populate only two slots with 32GB so you can expand later on.
before deciding on drive size and number decide on your best stripe/raid config and lern about how to expand vdevs in FreeNAS. My all in approach might not be the best/cost effective for you.
always calculate in the cost for a UPS if you don't have one. I totally forgot that in my initial calculation.
To the actual operation:
It runs FreeNAS 11 with all drives in one big Raid Z2 vdev. So I have two Disks redundancy and about 22TB of usable Space. (ZFS is a bit tricky with counting overhead plus you shouldn't go over 80% utilization so the real wold usable space will be smaller) I run different datasets for different purposes as well as a zvol for a ISCSI Target. I did not add any cache and the performance relies on the 32GB of ECC RAM. The main focus was reliability and versatility and not raw throughput. So far all my workloads are handled with ease.
I run Syncthing in a jail to sync all business files on all office machines. That works very well so far and is a low maintenance solution. For my private use I run Plex in another jail with its own Dataset. Im not a super heavy Plex user so the processor is handling that fine.
Every Dataset has a individual Snapshot routine depending on the use. Long term media Storage gets one every week, production critical and daily changing data gets more with the amount reducing over snapshot lifetime. (every hour for two weeks, every day for two month etc.)
Offsite Backup is Amazon S3 at the moment for all production critical data. Integration into FreeNas is easy and reliable. I have different cloud sync tasks for different datasets. Im syncing on a file level so all ZFS benefits are gone. Long term solution is a second FreeNAS build with replication tasks and only super critical data in a AWS Glacier storage.
Things I haven't touched at all are VM's event though they are supposed to work better in FreeNas 11.2.
In my opinion this build is the absolute sweet spot for Price per GB per Functionality. Or in other words every component is very well utilized and there is not one resource wasted or under-utilized. If you want to save money you could go with a smaller cheaper Power Supply. What ever you do don't cheep out on RAM. Its always your performance bottleneck. Don't invest in a cache drive until you understand how your workloads translate into the ZFS filesystem. The chance that more RAM is the solution seems more then often the case.
If there are any more questions fire away!
PS: More pictures coming soon