[Ldsoss] NAS and Firewall Hardware / Software

[Kevin Wise]

Thanks for all the input.  I actually already have a Linksys router for 
a hardware firewall, and I wasn't planning to replace that.  For the two 
of you that recommended splitting the hardware, would it change your 
mind if I limited the new machine to content filtering proxy and NAS 
(and leave the firewall and NAT to the router)?

Kevin

Scott Barber wrote:
> I'd second the recommendation to split them up. I run IPCop for the
> firewall, NAT, etc. and FreeNAS for the network storage.
>
> -Scott
>
> On 5/15/07, Shawn Willden <shawn-ldsoss at willden.org> wrote:
>> On Tuesday 15 May 2007 12:26:08 am Kevin Wise wrote:
>> > I'd really like one piece of hardware
>> > that does both.  In my mind this would save me maintenance (fewer
>> > patches to apply) and maybe even cost.  Any comments?
>>
>> I can see value in separating firewall and other functionality, but I
>> personally use one system for both, for just this reason.
>>
>> > I Is 512 MB of RAM enough?
>>
>> Plenty.  I have 1 GiB in mine, but that's mostly because I had extra 
>> RAM lying
>> around from upgrading another box.
>>
>> > Should I get
>> > hardware RAID or software RAID?  In terms of importance to me,
>> > reliability is second only to cost.  I don't want my files to 
>> disappear
>> > because my single RAID controller failed and the drive is 
>> unreadable by
>> > another controller.
>>
>> I use software RAID primarily for this reason, but there are other 
>> reasons as
>> well.  A big one is flexibility.  With Linux MD RAID you can mix and 
>> match
>> drives of different types and sizes with no problem, and you can use 
>> as many
>> disks as you can pack into the box.
>>
>> I also made use of MD RAID's flexibility to make adding new disks 
>> easier.  I
>> partitioned my drives into small (50GB) pieces and constructed multiple
>> arrays (each array element on a different disk, obviously), then 
>> combined the
>> RAID arrays into a large storage pool with LVM.  That way, when I 
>> need to add
>> another disk I can add it to the running system by:
>>
>> 1.  Pick one physical volume (which is a RAID array) and use pvmove 
>> to migrate
>> all of the data off of it.
>> 2.  Remove the array from the volume group
>> 3.  Destroy the array and rebuild it, adding another partition from 
>> the new
>> disk.
>> 4.  Add the resulting (larger) physical volume back into the volume 
>> group.
>> 5.  Go back to step 1, until all arrays have been upgraded.
>>
>> This approach takes a long time, but it's perfectly safe -- after a 
>> power
>> outage pvmove picks up right where it left off, yes I have firsthand
>> experience -- and the system continues running and serving files the 
>> whole
>> time.  Last time I did it, I wrote a script to perform the 
>> operations.  The
>> script took about 30 minutes to write and about four days to run.
>>
>> Supposedly, someone is looking into giving MD the native ability to add
>> another drive into RAID-5 arrays, which would make the partitioning + 
>> LVM
>> stuff less necessary, but it hasn't happened yet.
>>
>> One other thing to consider with your RAID configuration is hot spare 
>> vs. RAID
>> 6.  I use a hot spare, but I'm planning on rebuilding my system with 
>> RAID 6
>> (one partition array at a time).  The odds of two drives failing at 
>> once are
>> negligibly small, but I had a scare a few weeks ago when one of the 
>> RAID 5
>> drives failed and while the system was rebuilding onto the hot spare, 
>> another
>> drive had some transient error -- I think caused by a SATA controller 
>> driver
>> bug, but I can't be sure.
>>
>> The problem with RAID 5 is that the process of rebuilding a degraded 
>> RAID 5
>> array is very intense, so if you have another drive with any latent 
>> problems,
>> they'll probably crop up then -- the worst possible time.
>>
>> I think I did the best possible thing I could do -- I immediately 
>> shut the
>> machine down (and told the kids the video server was down, possibly 
>> for good)
>> and thought things over for a full week.  I realized that if I could 
>> forcibly
>> reconstruct each array with the exact sequence of drives that were 
>> running
>> when the second failure occurred, I might be able to get it back.  
>> Luckily,
>> mdadm had e-mailed me the contents of /proc/mdstat, and that had the
>> information I needed.
>>
>> So I powered the machine back up, forcibly rebuilt an array (still in 
>> degraded
>> mode) with --assume-clean, then added the spare and crossed my 
>> fingers while
>> it recalculated parity and changed to non-degraded mode.  When that 
>> worked, I
>> repeated with each of the other arrays, then held my breath while I
>> reactivated LVM and then ran fsck on the file systems.  It worked and I
>> didn't lose anything.
>>
>> After that harrowing experience, I made two decisions:
>>
>> 1)  I need to be more diligent about backing up my important data.  I 
>> had most
>> of it, but not all of it.
>> 2)  I'm going to move to RAID 6 so that I can take two *simultaneous* 
>> disk
>> failures and not lose anything.  That's better than RAID 5 with a hot 
>> spare,
>> and much better than RAID 5 without a spare.
>>
>> BTW, my system has 4 PATA and 2 SATA drives:
>>
>> 3 200 GB PATA
>> 1 250 GB PATA
>> 2 500 GB SATA
>>
>> I have four PATA controllers (two on the mobo, two on a PCI card), so 
>> each
>> drive is a master, for better performance.
>>
>> I use 200 GB of five of the six drives for the main RAID 5 arrays, so 
>> I have
>> 800 GB of usable storage there.  One of the 200s is the hot spare.  
>> The 500
>> GB drives have 300 left over, so I mirrored that, for another 300 GB 
>> usable.
>> All of that storage is in one big 1.1 TB volume.  The 50 GB left over 
>> on the
>> 250 GB drive is in a separate volume group, with bits carved out for 
>> various
>> temp storage uses.  So I'm "wasting" 200 (hot spare) + 200 (parity on 
>> RAID-5)
>> + 300 (mirrored) = 700.
>>
>> I'm soon going to add another 500.  When I do I'll add 200 of it to the
>> existing RAID 5 (while converting it to RAID 6 and incorporating the 
>> current
>> hot spare as an active disk), and I guess I'll have to change the 
>> mirrored
>> 300s to a RAID 5.  That'll get me to 1.65 TB usable of 2.35 TB total. I
>> figure I'll go to 8 disks before I start replacing the small 200s, 
>> mainly
>> because my server case has room for 8.
>>
>> To support such a large number of drives I had to get a bigger PSU 
>> and some
>> extra fans to keep everything cool.  BTW, an underpowered PSU causes 
>> very
>> strange, intermittent drive failures :-)
>>
>> > Also seems like a waste to buy new ATA drives
>> > (are they even available any more?).
>>
>> Sure, and they're priced basically the same as SATA drives.
>>
>> > Another option of course is to buy
>> > a SATA controller card.  Any idea how much that might cost?
>>
>> They're cheap.  $20 or so from Newegg or the like.
>>
>>         Shawn.
>> _______________________________________________
>> Ldsoss mailing list
>> Ldsoss at lists.ldsoss.org
>> http://lists.ldsoss.org/mailman/listinfo/ldsoss
>>
> _______________________________________________
> Ldsoss mailing list
> Ldsoss at lists.ldsoss.org
> http://lists.ldsoss.org/mailman/listinfo/ldsoss
>