ZFS powered NAS, ultimate alternative to Drobo + Droboshare [Complete Guide]

There is a newer version of this build. Check it out!

FreeNAS Performance
NAS performance is often tricky to measure because everything is relative to the size of the file you are transferring and the caching technique used to facilitate the transfer. In my previous post about using the TonidoPlug as an alternative to Droboshare, I’ve discovered that these underpowered devices lacked sufficient amount of memory to cache large files. Therefore, the burst speed (transferring small files, say 5mb) is a lot faster than the sustained speed (transferring large files, say 2GB). The Droboshare gives 6mb/s of sustained speed when transferring anything over 300MBytes, read up on a quick review from Engadget.

After testing my FreeNAS rig, I copied a 4GB file via Gigabit Ethernet and the throughput sustained at 25MBytes/s, which is 4 times the speed of the Drobo + Droboshare! That’s not bad at all!

Reading speed is 35MBytes/s, not great but good enough. The bottleneck here seems to be the Intel NM10 chipset bundled with the new Atom chip. I have seen people with regular desktop chipsets that can do 70-80MBytes/s read and 30-40MBytes /s write.

System Tweaks
The out of the box performance wasn’t exactly topnotch, but it’s nothing that can’t be fixed without a little tweaking. Since we use ZFS and Samba, let’s dig into the details on these two beasts.

ZFS
First, make sure you are running the latest nightly build FreeNAS that has the latest vesrion of ZFS – version 13.
Secondly, ZFS is a very memory intensive file system, largely due to the caching. The more caching done, the faster the speeds.
To change the cache size, login to the FreeNAS web GUI, Advanced->File Editor and load /cf/boot/loader.conf (if you are using the embedded FreeNAS version). Otherwise, load /boot/loader.conf

Copy and paste the following code and save the file.

kern.hz="1000"
vm.kmem_size="1524M"
vfs.zfs.arc_min="256M"
vfs.zfs.arc_max="768M"
vfs.zfs.txg.timeout="10"
vfs.zfs.vdev.max_pending="10"
vfs.zfs.vdev.min_pending="4"

This tweak is best for systems with 2GB of RAM. The vm.kmem_size defines the total amount of RAM the kernel can allocate for ZFS. It should be set to about 50%-70% of your total RAM size. The vfs.zfs.arc_max size should be about half of vm.kmem_size.

Samba
The folks over at FreeNAS have done a great job of keeping Samba up to date, especially with the latest nightly builds. Most of these experimental builds have been rock solid.
The secret to having a speedy Samba server is to enable Asynchronous IO. This feature has been built into the latest nightly build starting from FreeNAS 0.7.2 (5191).
Simply go to your Samba configuration page in the web GUI, and enable AIO and use the default value of “1″ for both AIO Read and Write size.
Also do not forget to enable the “Large Read/Write” feature for  Samba, and enable keneral tuning. (Go to System->Advanced – enable Tuning)

That’s it, after a reboot you are done. Most of the optimizations have been enabled by FreeNAS out of the box, but there are some experimental features that you have to dig out on your own. You should test the ZFS RAM size and its impact on performance. Also be aware that the memory size allocated cannot be too large, or your system may become unstable.

That’s it guys, this concludes my NAS build. I apologize for the long wait, but the end result is very satisfying.

It is possible to build your own NAS with Drobo abilities for half of the price and 5 times the speed.  =p

Please leave comments and questions.

 

  • eric

    Hi Frank,

    I think you missed my comment.

  • http://www.frankleng.me Frank Leng

    oh sorry about that. I got the case from NCIX.com
    they are Canadian but also have a US site.

  • eric

    Thanks. I never ordered anything from NCIX before, but I’ve heard of them before.

    Thanks for letting me know :)

    I think I might just get the Fractal Design Define R3 :)

  • Pingback: Review – HP Microserver « Chris Swan's Weblog

  • gcs8

    i got feedup with my drobo about 3 months ago when i ran out of space (95%) an it slowed down a lot. so i build my self a freenas box that now holds 20 2TB drives in a raidZ2 with ZFS over iscsi for 33.9TB of storage and a lot better RW speeds. let me know what you think i will leave some links if you dont mind, its my new baby. http://gcs8.org/m http://gcs8.org/rack great wright up BTW.

  • Concord

    Not sure if you want it for personal use or corporate use.  In any case, for business, I’m using a couple of aberdeen inc. (abernas) RAID6 standard NAS servers for 4-5 years.  They came out with ZFS server using NExenta software in 2010.  Like any rack server, the fans are real noisy, at least on the ones I have.

  • Izrada web sajtova

    This is a
    great and informative post.Well, the points given were facts that could never
    go wrong. I have visited so many pages with the same discussion and topics

     

    Izrada web sajtova

  • Polovni automobili

    Great job here. I really enjoyed
    what you had to say.           

    Polovni
    automobili

  • http://www.day-traders.com/ day traders

    I really love reading your comments guys.I learn a lot from you.

    Thank you so much.

    Regard,

    Epoyjun

  • mobi

    Thanks for a great article, I’m now fully signed up to freenas! I’m using the Asus E35M1-M PRO
    http://www.asus.com/Motherboards/AMD_CPU_on_Board/E35M1M_PRO/

    but I fail to see how the GPU is going to help with data copy throughput. Can you explain? Maybe freebsd or zfs makes use of the GPU??

  • Mbwjoe

    I think all you’ve done here is prove just how great the drobo is.

  • Burzukk

    I am a video designer and work with numerous and huge files that I wish to keep all this as safe as possible, fast and easy to access from my win7 os

    My questions:
    1.How will this virtual machine interract with windows 7. Via ftp!? 
    2. Is it possible to make it a DAS communicating directly with windows (recognized as just an external disk via esata). Would the VM OS just sort all this as the Drobo seems to?
    3. In its current state (NAS) can I just throw files from my ntfs drives at it whitout any concern and vice versa?
    4. Do you think there is any way I could work directly on the NAS from windows? Without using a ftp!? (edit files, open after effects projects from it, render directly on it, etc)

    I know you probably don’t use windows as and OS may not know all the answers concerning windows, but just trying to find answers, can’t find anything related to my questions on the web. Thanks!
     

  • http://www.frankleng.me Frank Leng

    It was never meant to be a drobo killer. drobo has its place. but for the people who want the best performance/cost ratio, this is it.

    most importantly. if your drobo ever breaks down like mine did. your only bet is to buy another drobo. which can be quite costly for most people here

  • http://www.frankleng.me Frank Leng

    Hi. forget about FTP. I never mentioned anything about it in the guide, nor are we ever going to be using that with this setup.

    The machine you end up building uses standard Windows file sharing protocol – SMB aka CIFS. This means that Windows can detect the server on the network directly. You will access the file just like you would with any standard Windows network share.

    And yes, you can drag and drop any file, any size of files you want from your existing NTFS partitions. 
    You will want to use fast hard drives. the WD Greens I used will not do you much good.

  • http://frankleng.me Frank

    the GPU is not going to help. I was referring to the main chipset which has a GPU embedded in it.

  • explorer

    I’ve been using Drobo Gen1 for more than 3 years and I’m now looking at Drobo S. I’m a video pro, and now with RED cameras accessible and SpeedGrade (which works with DPX sequences) available in Adobe CS6 I need more storage, peace of mind that my data is safe, and most of all speed. Do you think your solution will work?

    I was planning to use Drobo S eSATA connection to access my files. Rememberm that working with video is very different that simply sending files over FTP using UDP protocol.

  • http://www.frankleng.me Frank Leng

    If your goal is to get fast data access, then I would say go with this setup but connect the NAS to your workstation with a crossover Gigabit ethernet, so even 10Gigabit ethernet connection.
    From what I could remember, the Drobo and Drobo S are slower than most DIY setup. mostly due to the way it writes file into its proprietary file system.  Maybe it’s improved now… but I would not bet on it for speed. only simplicity.

    A friend of mine did a simple ZFS setup for storing RED footages and he used 10k Raptor drives. That worked really well for him, but he said the machine needed a lot of RAM for ZFS to really perform. 8GB-16GB are recommended. which is about an additional $100 to spend.

  • Podne obloge

    Oh
    my goodness! an amazing article dude. Thank you
    Podne obloge

  • Rob Blasdel

     http://scottkelby.com/2012/im-done-with-drobo/

  • marcopolojetlag

    It was really great to find this resource via Google by searching for “Drobo Alternative”.  Is the thread still active?  I would like to synch this device with http://www.crashplan.com.  Has anyone done that?  Will it work?

  • http://www.frankleng.me Frank Leng

    You can connect to the network share powered by FreeNAS from Windows and use that to store your backup.

    I have never used Crashplan myself, but it sounds like another backup scheduler with a cloud option. 

  • Gcsviii

    the secret is use iSCSi for windows so it sees it as an internal drive or hook it to a lunix box. in the FAQ some where @ crashplan they say it will not backup shares in windows but will in linux, so you just need to fine what works best for you. i went the lazy way and used iSCSi.

  • marcopolojetlag

     Your original post was written some time ago, so I imagine there are some newer components out there that might even be better now.  Would the Intel Atom Dual-Core D525 be a good update, or can you recommend something better?  Thanks much.

    http://www.amazon.com/Intel-Dual-Core-Mini-ITX-Motherboard-BOXD525MW/dp/B0041RSC94/ref=pd_sim_e_1

  • http://www.frankleng.me Frank Leng

    I will address this in the next build.  this is more of a limitation from ZFS

  • http://www.frankleng.me Frank Leng

    the D525 is certainly a good candidate if you only want to use the machine as a NAS and you can find a motherboard with more than 4 SATA ports.

    For my next build I actually opted in to use the 2100T i3 chip. It is extremely efficient and powerful. I love it because my NAS is also my HTPC and DVR.
    I will post my new build next week. :)

  • marcopolojetlag

     Wow, well I’m glad I didn’t buy everything yet if you’re about to post some new recommendations.  I did go ahead and order the Antec NSK-2480 case.  Is that still the best one to get?  I guess you are showing how to build a better machine that can do more than just be a NAS machine.  Please be very specific on exactly what to buy, because I don’t know enough about all this stuff to mix and match processors, motherboards, SATA drives, etc.  I just need to know what you recommend–I would never attempt to outguess you.  If you could include links to the items on Amazon, that would be even better!  Then if people want to buy somewhere else, at least they can see exactly what item it is and see the reviews on it.  I started to look at hard drives, but I could see there that the green drives had a high percentage of poor reviews.  What do you think?  Do you still recommend them?  People there had some bad experiences where drives started to go bad slowly and data got corrupted even in a RAID configuration.  It seems like the 7200 RPM drives had a lower failure rate than the green drives, based on reviews.

    Can you also be more explicit about the operating system?  I don’t get all the terminology yet.  Do you recommend Windows or Linux?  If Windows, which version?

    Thank you very, very much for all your work on this.  I, for one, really appreciate this contribution to society.  I’m looking forward to getting this machine built and working.

  • marcopolojetlag

     Thanks very much for your comments.  I have paid for 4 years of unlimited storage on CrashPlan for all the computers in my house, so I really want to get this working.  (I hope CrashPlan servers survive any upcoming apocalypse!!!)

    I looked up iSCSI for Windows and I found this link:

    http://www.microsoft.com/en-us/download/details.aspx?id=18986

    Where it says:

    “The Microsoft iSCSI Software Initiator enables connection of a Windows
    host to an external iSCSI storage array using Ethernet NICs. This
    download can be installed on Windows Server 2003, Windows XP, and
    Windows 2000. For Vista and Windows Server 2008, the iSCSI initiator is
    included inbox.”

    What version of Windows did you use?

    It sounds like you’re saying that I will just have CrashPlan running on my laptop.  But I want CrashPlan to actually run on the server.  Shouldn’t that work?

  • Gcsviii

    well freenas is a freeBSD/solars mutant baby that has been compiled for one job only, now in the newer versions of freenas they have introduced plugins and anyone can make one, (http://doc.freenas.org/index.php/Plugins#Creating_your_own_FreeNAS.C2.AE_PBIs), now if you are lazy like i am, you make your pool, make your data sheet, setup iSCSI on freenas, then from your windows 7 box you will connect to the iSCSI target, then you will need to format the block lvl storage that windows will see, then you just treat it like a drive. 

    now some things to think about, you are probably going to want to hook this to a desktop or a server that is always on, and always at the house, i suggest a normal win7 64bit desktop with ~8gb of ram and a gig nic or 2, if you start getting a lot of data like i have you will need to mod he java engine that runs crash plan, mine eats ~1.5gb of ram and will continue to need more so i told it can use up to 4gb of ram, i am only at /~10.5TB of 35.6TB of data right now so i will be safe for a bit. you will also not want to do this over wifi, you do want to enable crc and digest check sums  etc etc, then from the host share it out from that win7 box over CIFS/samba (standard windows share) and just mount it as a network share or network drive to the other machines. also DO NOT connect more then one machine via iSCSI to the freenas box, iSCSI and NTFS do not like more then one machine writing to the MFT, bad things will happen.

    then we start getting into cache drives, ZIL drives, RAM needs, etc. i can go on for a long time about how ZFS works but it is nothing you can not read for your self and do the research for.

  • marcopolojetlag

     Maybe I don’t want to use FreeNAS after all.  I guess what I want is a server that will also back itself up to CrashPlan.  I have a small photo studio in my house and we have 3 MacBooks, plus a Mac Mini, plus some Windows machines.  I would like any of these machines to be able to connect to the server and dump files for backup.

    So after a photo session (or a wedding), I would like to immediately copy the new files to the server over the network, then I have two copies of each image right away.  Then the server will work on backing these up to the cloud, all day and all night.

    I also noticed that the backup running on my MacBook Pro started going much faster when I took it into the office, where the company has a much faster upload speed.  So I would like to be able to pick up my server and carry it to the office to run over a weekend, and then maybe my backup would finish in a weekend, instead of in 2 weeks.

    So am I correct that FreeNAS is not suitable for the job?

  • Gcsviii

    that all depends on how you use it, my freenas box has 20 drives in it and weighs more then i do. you might just want to get a mac mini server that will do AFP and CIFS/SAMBA, then have crashplain back up from that, and find your self a nice thunderbolt array. you can time machine for the mac,s or setup ccc to do a diff copy to a share. windows is a little harder but not much. there are many ways to do this, just pick one you like.

  • marcopolojetlag

    Is there another case, just as good as the Antec NSK-2480, but also stands up vertical?

  • http://www.frankleng.me Frank Leng

    I am using this case for my next build, which I’ll be writing about this weekend. check it out. it’s great!
    http://www.amazon.com/gp/product/B006L6ZHAU/ref=as_li_ss_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B006L6ZHAU&linkCode=as2&tag=franataglan-20

  • marcopolojetlag

     Frank, I bought the case you’re recommending now–thank you very much.  Will your new post be here, or is there another link?

  • http://www.frankleng.me Frank Leng

    here is the latest build. please check it out: http://frankleng.me/2012/12/15/snapshot-raid-powered-nas-htpc/

  • http://www.frankleng.me Frank Leng

    http://frankleng.me/2012/12/15/snapshot-raid-powered-nas-htpc/
    with this new build. you can add new drives at any time as long as you have pre-allocated empty DRUs at setup. take a read if you want.

  • http://www.frankleng.me Frank Leng

    what I was referring to was the chipset, not the integrated graphics card. the chip is the one that controls the I/O. 
    but in this case, it looks like my add-on SATA controller was the bottleneck. 

    check out the latest build for a better setup http://frankleng.me/2012/12/15/snapshot-raid-powered-nas-htpc/

  • Pingback: High-end NAS roundup

See also: