[Ldsoss] NAS and Firewall Hardware / Software

[Scott Barber]

I'd second the recommendation to split them up. I run IPCop for the
firewall, NAT, etc. and FreeNAS for the network storage.

-Scott

On 5/15/07, Shawn Willden <shawn-ldsoss at willden.org> wrote:
> On Tuesday 15 May 2007 12:26:08 am Kevin Wise wrote:
> > I'd really like one piece of hardware
> > that does both.  In my mind this would save me maintenance (fewer
> > patches to apply) and maybe even cost.  Any comments?
>
> I can see value in separating firewall and other functionality, but I
> personally use one system for both, for just this reason.
>
> > I Is 512 MB of RAM enough?
>
> Plenty.  I have 1 GiB in mine, but that's mostly because I had extra RAM lying
> around from upgrading another box.
>
> > Should I get
> > hardware RAID or software RAID?  In terms of importance to me,
> > reliability is second only to cost.  I don't want my files to disappear
> > because my single RAID controller failed and the drive is unreadable by
> > another controller.
>
> I use software RAID primarily for this reason, but there are other reasons as
> well.  A big one is flexibility.  With Linux MD RAID you can mix and match
> drives of different types and sizes with no problem, and you can use as many
> disks as you can pack into the box.
>
> I also made use of MD RAID's flexibility to make adding new disks easier.  I
> partitioned my drives into small (50GB) pieces and constructed multiple
> arrays (each array element on a different disk, obviously), then combined the
> RAID arrays into a large storage pool with LVM.  That way, when I need to add
> another disk I can add it to the running system by:
>
> 1.  Pick one physical volume (which is a RAID array) and use pvmove to migrate
> all of the data off of it.
> 2.  Remove the array from the volume group
> 3.  Destroy the array and rebuild it, adding another partition from the new
> disk.
> 4.  Add the resulting (larger) physical volume back into the volume group.
> 5.  Go back to step 1, until all arrays have been upgraded.
>
> This approach takes a long time, but it's perfectly safe -- after a power
> outage pvmove picks up right where it left off, yes I have firsthand
> experience -- and the system continues running and serving files the whole
> time.  Last time I did it, I wrote a script to perform the operations.  The
> script took about 30 minutes to write and about four days to run.
>
> Supposedly, someone is looking into giving MD the native ability to add
> another drive into RAID-5 arrays, which would make the partitioning + LVM
> stuff less necessary, but it hasn't happened yet.
>
> One other thing to consider with your RAID configuration is hot spare vs. RAID
> 6.  I use a hot spare, but I'm planning on rebuilding my system with RAID 6
> (one partition array at a time).  The odds of two drives failing at once are
> negligibly small, but I had a scare a few weeks ago when one of the RAID 5
> drives failed and while the system was rebuilding onto the hot spare, another
> drive had some transient error -- I think caused by a SATA controller driver
> bug, but I can't be sure.
>
> The problem with RAID 5 is that the process of rebuilding a degraded RAID 5
> array is very intense, so if you have another drive with any latent problems,
> they'll probably crop up then -- the worst possible time.
>
> I think I did the best possible thing I could do -- I immediately shut the
> machine down (and told the kids the video server was down, possibly for good)
> and thought things over for a full week.  I realized that if I could forcibly
> reconstruct each array with the exact sequence of drives that were running
> when the second failure occurred, I might be able to get it back.  Luckily,
> mdadm had e-mailed me the contents of /proc/mdstat, and that had the
> information I needed.
>
> So I powered the machine back up, forcibly rebuilt an array (still in degraded
> mode) with --assume-clean, then added the spare and crossed my fingers while
> it recalculated parity and changed to non-degraded mode.  When that worked, I
> repeated with each of the other arrays, then held my breath while I
> reactivated LVM and then ran fsck on the file systems.  It worked and I
> didn't lose anything.
>
> After that harrowing experience, I made two decisions:
>
> 1)  I need to be more diligent about backing up my important data.  I had most
> of it, but not all of it.
> 2)  I'm going to move to RAID 6 so that I can take two *simultaneous* disk
> failures and not lose anything.  That's better than RAID 5 with a hot spare,
> and much better than RAID 5 without a spare.
>
> BTW, my system has 4 PATA and 2 SATA drives:
>
> 3 200 GB PATA
> 1 250 GB PATA
> 2 500 GB SATA
>
> I have four PATA controllers (two on the mobo, two on a PCI card), so each
> drive is a master, for better performance.
>
> I use 200 GB of five of the six drives for the main RAID 5 arrays, so I have
> 800 GB of usable storage there.  One of the 200s is the hot spare.  The 500
> GB drives have 300 left over, so I mirrored that, for another 300 GB usable.
> All of that storage is in one big 1.1 TB volume.  The 50 GB left over on the
> 250 GB drive is in a separate volume group, with bits carved out for various
> temp storage uses.  So I'm "wasting" 200 (hot spare) + 200 (parity on RAID-5)
> + 300 (mirrored) = 700.
>
> I'm soon going to add another 500.  When I do I'll add 200 of it to the
> existing RAID 5 (while converting it to RAID 6 and incorporating the current
> hot spare as an active disk), and I guess I'll have to change the mirrored
> 300s to a RAID 5.  That'll get me to 1.65 TB usable of 2.35 TB total. I
> figure I'll go to 8 disks before I start replacing the small 200s, mainly
> because my server case has room for 8.
>
> To support such a large number of drives I had to get a bigger PSU and some
> extra fans to keep everything cool.  BTW, an underpowered PSU causes very
> strange, intermittent drive failures :-)
>
> > Also seems like a waste to buy new ATA drives
> > (are they even available any more?).
>
> Sure, and they're priced basically the same as SATA drives.
>
> > Another option of course is to buy
> > a SATA controller card.  Any idea how much that might cost?
>
> They're cheap.  $20 or so from Newegg or the like.
>
>         Shawn.
> _______________________________________________
> Ldsoss mailing list
> Ldsoss at lists.ldsoss.org
> http://lists.ldsoss.org/mailman/listinfo/ldsoss
>