PGSQL PERFORMANCE 10 RE HARDWARE PURCHASE QUESTION
From: khera@no-spam (Vivek Khera)
Subject: Re: Hardware purchase question
Date: Mon, 13 Dec 2004 11:33:47 -0500


>>>>> "BS" == Bo Stewart <bstewart@no-spam> writes:

BS> The servers listed above are the dell 2650's which have perc 3
BS> controllers. I have seen on this list where they are know for not BS> performing well. So any suggestions for an attached scsi device would BS> be greatly appreciated. Also, any thoughts on fibre channel storage BS> devices?

I have a 2450 and a 2650 both of which are totally sucking IO wise.

The 2650 has a PERC3 card (LSI based) and has one channel holding a mirrored pair for the pg_xlog and OS, and the other channel has 14
U320 disks in a RAID5. If I'm lucky, I'll get 30MB/s out of the disks. Normally it hovers at 5 or 6MB/s on the big RAID.

I'm currently shopping for non-Dell hardware to replace it :-(

However, I keep getting conflicting advice. My choices are along these lines:

Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

I'm sure any of these will whip the bottom off the Dell 2650, but which will be the fastest overall? No way to know without spending lots of money to test. :-(

Dell claims their new 2750 will be faster, but they've lost the battle already, and won't commit to any performance numbers. Won't even give me a ballpark number.

-- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Vivek Khera, Ph.D. Khera Communications, Inc.
Internet: khera@no-spam Rockville, MD +1-301-869-4449 x806
AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@no-spam so that your message can get through to the mailing list cleanly

From: jd@no-spam ("Joshua D. Drake")
Subject: Re: Hardware purchase question
Date: Mon, 13 Dec 2004 09:23:13 -0800

> However, I keep getting conflicting advice. My choices are along > these lines:
> > Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

An Opteron, properly tuned with PostgreSQL will always beat a Xeon in terms of raw cpu.

RAID 10 will typically always outperform RAID 5 with the same HD config.

Fibre channel in general will always beat a normal (especially an LSI) raid.

Dell's suck for PostgreSQL.

Sincerely,

Joshua D. Drake
> > I'm sure any of these will whip the bottom off the Dell 2650, but > which will be the fastest overall? No way to know without spending > lots of money to test. :-(
> > Dell claims their new 2750 will be faster, but they've lost the battle > already, and won't commit to any performance numbers. Won't even give > me a ballpark number.
>
-- Command Prompt, Inc., home of PostgreSQL Replication, and plPHP.
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-667-4564 - jd@no-spam - http://www.commandprompt.com Mammoth PostgreSQL Replicator. Integrated Replication for PostgreSQL
begin:vcard fn:Joshua D. Drake n:Drake;Joshua D.
org:Command Prompt, Inc.
adr:;;PO Box 215;Cascade Locks;Oregon;97014;USA email;internet:jd@no-spam title:Consultant tel;work:503-667-4564
tel;fax:503-210-0334
note:Command Prompt, Inc. is the largest and oldest US based commercial PostgreSQL support provider. We provide the only commercially viable integrated PostgreSQL replication solution, but also custom programming, and support. We authored the book Practical PostgreSQL, the procedural language plPHP, and adding trigger capability to plPerl.

x-mozilla-html:FALSE url:http://www.commandprompt.com/
version:2.1
end:vcard
[plaintext jd.vcf]

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@no-spam

From: josh@no-spam (Josh Berkus)
Subject: Re: Hardware purchase question
Date: Mon, 13 Dec 2004 10:31:40 -0800

Vivek,

> Dual Xeon 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with built-in 6-disk RAID10 or RAID5 (LSI RAID card)
> Dual Opteron 64bit with external RAID via fibre channel (eg, nstor)

Opteron over Xeon, no question. Not only are the Opterons real-world-faster, they are less severely affected by the CS bug.

> I'm sure any of these will whip the bottom off the Dell 2650, but > which will be the fastest overall? No way to know without spending > lots of money to test. :-(

The SAN is going to be faster with a good SAN. That being said, I understand that "a good SAN" is something like a $30,000 NetApp; the less expensive SANs/NASes don't seem to be more than an external drive enclosure with a raid chip (e.g. Apple XRaid). But we saw even a less expensive/slower EMC machine improve performance just moving the pg_xlog off of the local PERC RAID 5 and onto the SAN. So this is probably a good way to go if you can afford it.

-- Josh Berkus Aglio Database Solutions San Francisco
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From: ahood.lists@no-spam (Andrew Hood)
Subject: Re: Hardware purchase question
Date: Tue, 14 Dec 2004 09:33:55 +0000

Joshua D. Drake wrote:
> > An Opteron, properly tuned with PostgreSQL will always beat a Xeon > in terms of raw cpu.
> > RAID 10 will typically always outperform RAID 5 with the same HD config.
> > Fibre channel in general will always beat a normal (especially an LSI) > raid.
> > Dell's suck for PostgreSQL.

Does anyone have any OS recommendations/experiences for PostgreSQL on Opteron?

Thanks,
Andrew
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@no-spam

From: mitch.pirtle@no-spam (Mitch Pirtle)
Subject: Re: Hardware purchase question
Date: Mon, 3 Jan 2005 14:20:41 -0500

On Mon, 13 Dec 2004 09:23:13 -0800, Joshua D. Drake <jd@no-spam> wrote:
> > RAID 10 will typically always outperform RAID 5 with the same HD config.

Isn't RAID10 just RAID5 mirrored? How does that speed up performance?
Or am I missing something?

-- Mitch
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@no-spam

From: linux@no-spam (Madison Kelly)
Subject: Re: Hardware purchase question
Date: Mon, 03 Jan 2005 14:35:26 -0500

Mitch Pirtle wrote:
> On Mon, 13 Dec 2004 09:23:13 -0800, Joshua D. Drake > <jd@no-spam> wrote:
> >>RAID 10 will typically always outperform RAID 5 with the same HD config.
> > > Isn't RAID10 just RAID5 mirrored? How does that speed up performance?
> Or am I missing something?
> > -- Mitch
Hi Mitch,

Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10 you get the benefit of a full mirror which means your system does not need to calculate the XOR parity but you only get 50% disk usage. The mirror causes a slight write hit as the data needs to be split between two disk (or in this case, to striped pairs) but reads can be up to twice as fast (theoretically). By adding the stripe you negate the write hit and actually gain write performance because half the data goes to mirror A, half to mirror B (same with reads, roughly).

Raid 10 is a popular choice for software raid because of the reduced overhead. Raid 5 on the otherhand does require that a parity bit is calculated for every N-1 disks. With r5 you get N-1 disk usage (you get the combined capacity of 3 disks in a 4 disk r5 array) and still get the benefit of striping across the disks so long as you have a dedicated raid asic that can do the XOR calculations. Without it, specially in a failure state, the performance can collapse as the CPU performs all that extra math.

hth
Madison
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

From: linux@no-spam (Madison Kelly)
Subject: Re: Hardware purchase question
Date: Mon, 03 Jan 2005 15:19:04 -0500

Madison Kelly wrote:
> Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10
Woops, that should be "mirror of stripes".

By the way, what you are thinking of is possible, it would be 51 (five one; a raid 5 built on mirrors) or 15 (a mirror of raid 5 arrays). Always be careful, 10 and 01 are also not the same. You want to think carefully about what you want out of your array before building it.

Madison
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

From: gsstark@no-spam (Greg Stark)
Subject: Re: Hardware purchase question
Date: 03 Jan 2005 15:36:07 -0500

Madison Kelly <linux@no-spam> writes:

> Without it, specially in a failure state, the performance can collapse as > the CPU performs all that extra math.

It's really not the math that makes raid 5 hurt. It's that in order to calculate the checksum block the raid controller needs to read in the existing checksum block and write out the new version. So every write causes not just one drive seeking and writing, but a second drive seeking and performing a read and a write.

The usual strategy for dealing with that is stuffing a huge nonvolatile cache in the controller so those reads are mostly cached and the extra writes don't saturate the i/o throughput. But those kinds of controllers are expensive and not an option for software raid.

-- greg
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@no-spam so that your message can get through to the mailing list cleanly

From: mitch.pirtle@no-spam (Mitch Pirtle)
Subject: Re: Hardware purchase question
Date: Mon, 3 Jan 2005 15:44:44 -0500

You are right, I now remember that setup was originally called "RAID 10 plus 1", and I believe is was an incorrect statement from an overzealous salesman ;-)

Thanks for the clarification!

- Mitch
On Mon, 03 Jan 2005 15:19:04 -0500, Madison Kelly <linux@no-spam> wrote:
> Madison Kelly wrote:
> > Nope, Raid 10 (one zero) is a mirror is stripes, no parity. with r10
> > Woops, that should be "mirror of stripes".
> > By the way, what you are thinking of is possible, it would be 51 (five > one; a raid 5 built on mirrors) or 15 (a mirror of raid 5 arrays).
> Always be careful, 10 and 01 are also not the same. You want to think > carefully about what you want out of your array before building it.

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@no-spam so that your message can get through to the mailing list cleanly

From: gregab@no-spam (Grega Bremec)
Subject: Re: Hardware purchase question
Date: Tue, 4 Jan 2005 02:21:10 +0100

=2E..and on Mon, Jan 03, 2005 at 03:44:44PM -0500, Mitch Pirtle used the ke=
yboard:
>
> You are right, I now remember that setup was originally called "RAID > 10 plus 1", and I believe is was an incorrect statement from an > overzealous salesman ;-)
>

Just an afterthought - that could well be the unfortunate consequence of salesmen specializing in sales as an act rather than the goods they were selling - it might be that he/she was referring to the specifics of the concrete configuration they were selling you (or trying to sell you),
which should, in the case you were mentioning, probably be called "a RAID10 array with a hotspare drive" - that is, it would be preconfigured to, upon the failure of one of array members, detect the failed drive and automatically replace it with one that has been sitting there all the time,
doing nothing but waiting for one of its active companions to fail.

But this already falls into the category that has, so far, probably caused the vast majority of misunderstandings, failed investments and grey hair in RAID, namely data safety, and I don't feel particularly qualified for getting into specifics of this at this moment, as it happens to be 2AM, I had a couple of beers (my friend's birthday's due)
and I'm dying to get some sleep. :)

HTH, cheers,
--=20
Grega Bremec gregab at p0f dot net
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFB2e+Gfu4IwuB3+XoRAjS+AJ9YAsLYrbwzItSGHXEJ4ufvj3SYqwCfRx+O QUjcsYMUbSOVfNhygSX7SZc=
=4Cc+
-----END PGP SIGNATURE-----