Description

After months of research, perusing pcpartpicker, dreaming, fretting, saving, hesitating, and dreaming some more, I finally purchased this build that I'm planning on using for GPU data analysis.

I'm a grad student in astrophysics and my research lab has about 14 TB of data. I have access to HPC resources on campus, but it's much more convenient to do things on a dedicated machine if possible. Plus, the analysis I'm working on will definitely benefit from GPU(s).

I frequently consulted Tim Dettmers' help guide for advice about what's important and what's not important when it comes to training machine learning algorithms (e.g. neural networks) on real world data.

CPU: Chose the Intel i7 5930k because this was the ~cheapest intel processor that supports all 40 pcie3 lanes. Added a nice water cooler for overclocking.

GPU: Went with the beast 980 Ti; was torn between 3x960 (4GB) and 1x980 Ti (6GB) [initially], but decided for the latter because of its larger memory bus, beastly core count, and the fact that it should last me for awhile [edited at later date: Added an additional 980 Ti, so now have 2 980 Ti's in SLI]. I know Pascal is set to be released "sometime" this year, but I don't want to fall into the "waiting for the next GPU" hole.

HDD/SDD:

  • Samsung 950 256GB for boot/cache, because at 2-2.5 GB/s read and 1.5 GB/s write, I can justify the $1/GB.

  • 4x2TB Hitachi 7200RPM HDD/s in (software) RAID 6, plus one spare. This gives me 4 TB of usable space, but if I abandon RAID and include my spare I can get 10TB.

OS: Ubuntu 14.04 ; setting up double screens was a nightmare...if you're encountering problems with a similar setup, let me know and I can try to pass on some hard-earned wisdom.

Cable management: is not something I have accomplished here. I went with a cheaper & semi-modular PSU which I regret. I can always upgrade this later. If anyone has any tips and/or tricks, definitely let me know.

Everything else was secondary, so I basically went with the cheapest components I could find that would support the GPU/CPU/storage setup that I wanted (I was sure to get a motherboard with a pcie3x4 M.2 slot and enough PCIe3x16 slots to support 2 GPUs).

Since there motherboard M.2 slot shares PCI bandwidth, I got an external M.2 -> PCIe adapter for ~$30 and just used the third PCI slot to host my M.2 SSD.

Part Reviews

CPU

Zero problems, overclocks extremely easily by about 30% (with the watercooler and a reasonable motherboard).

CPU Cooler

Keeps my i7-5930k at 25/45 degrees C at idle/load, quiet, fairly easy installation

Motherboard

Pros:

  • Easy to overclock once setup

  • lots of bells and whistles, most of which are completely irrelevant.

Cons:

  • I made the n00b mistake of assuming that the "raid" controller was a hardware raid controller (which I naively assumed was better). It isn't, and the entire marketing of "raid" features on most consumer motherboards are just garbage. Either use a real raid controller or setup software raid.

  • may the good lord help you if you aren't running Windows or if you want to deal with getting updates over the internet (a feature they offer but I was never able to get work)

  • Posts twice, and the first "post" (keyboard is unrecognized) can last a long time (like 20-30 seconds? Ain't nobody got time for that). This is apparently a fairly common problem that others have claimed resulted from disconnecting the power supply for any length of time.

Memory

Not much to say here, though I haven't done anything fancy like overclock, etc.

Storage

So worth it. Speeds are as advertised.

Video Card

<3, does anyone anywhere ever have anything bad to say about 980 ti's? I didn't think so.

Case

Lots of room, looks great! Extra back panel to hide cables.

Power Supply

Comes with cables (cool), but semi modular was a baaaaad choice on my part. Cable management has been a bit of a nightmare (though things could be worse, too).

Monitor

Great buy for the price ($90), but a lot of choppiness even with everyday usage (not gaming).

Keyboard

Works fine, but I'm not wild about these switches. I was hoping for something a little more clump-y, but these are extremely click-y, and the restorative force of the keys takes some getting used to. Also quite loud. I think my office neighbor may be planning to murder me.

Log in to rate comments or to post a comment.

Comments

  • 47 months ago
  • 7 points

Let's be real, the only data you're planning to analyze with this machine are your FPS and your K/D.

JK, great build. +1 !

  • 47 months ago
  • 3 points

Awesome! Love the dual 980 Tis.

  • 47 months ago
  • 2 points

Posts twice, and the first "post" (keyboard is unrecognized) can last a long time (like 20-30 seconds? Ain't nobody got time for that).

That reference combined with this build made my day. Thank you, sir, for posting - and best of luck in your studies. Sorry about the mobo tho. Now that I think about it, it makes me a bit sad. Oh well. +1

  • 47 months ago
  • 2 points

Thanks for the +1! Yea, mobo is a bit of a bust :/, but it works (which is a good thing) and I rarely restart my computer anyway, so only have to deal with the double-post shenanigans once and awhile.

  • 11 months ago
  • 1 point

Can I ask a question regarding the PSU? Would it support a gtx 1070 ti? New to the power pin thingies.

  • 47 months ago
  • 2 points

You did really well on this build.

  • 47 months ago
  • 2 points

Thanks a lot!

  • 47 months ago
  • 2 points

Is this rig funded by your professor's grant or 100% by you? If the answer is later than you are the first one I've seen who invest that much into research:). Besides, for HPC, 32 Gb probably is not enough? Well it depends on the situation. Great build anyway.

  • 47 months ago
  • 3 points

Funded by 100% by yours truly. Basically, this thing wasn't completely necessary, and, even if it was, this rig would have to be covered by some governmental funding agency, meaning Uncle Sam would own it. I'm actually planning on leaving academia after my PhD; I like working on more "real-world" problems like Kaggle competitions, etc, and GPU rigs like this one are great for that. And, owning it means I can take it with me when I leave.

Plus, this way I get to do what I want with it! I don't think the National Science Foundation would take too kindly if grad students used their computers to play GTA V.

In terms of RAM -- since most of what I'm doing is on GPU's, the system RAM isn't really too important, as long as [system ram] > [GPU ram]. 32 GB does help with CPU-intensive tasks and for other miscellaneous things.

  • 47 months ago
  • 2 points

Great work on this build. I am a student doing ML too, also thinking about building a system w similar spec +1

What astrophysics data are you running neural net on? (If you don't mind sharing)

  • 47 months ago
  • 1 point

Hey! Yea no problem -- I'm working with time series measurements ("lightcurves" = measurements of a star's brightness over time). The problem is (1) detecting variability, and (2) classifying variable stars. (2) is where neural nets usually come in (though people have tried them for (1), too).

Let me know if you want any more information! What about yourself?

  • 47 months ago
  • 1 point

That sounds really interesting. I'm still an undergrad student, doing mostly Natural Language Processing (classifying medical terminology in diagnosis reports) and thinking about doing more Kaggle stuffs on the side. Some GTA V too since it will be a personal rig :D

I read Tim Dettmer's article as well but there are a few things I'm still not sure about. I have budget for a 2 way SLI but do you think there would be much performance loss if I cheap out on the CPU? (Thinking about the 28 lane 5820k running x16/x8/x4 M.2 instead of your 40 lane 5930k running x16/x16/x4 M.2)

  • 47 months ago
  • 1 point

Hey sorry for the late reply -- I think it completely depends on the application. Worst-case scenario (i.e., if data transfer is the bottleneck for you) you're going to get 75% of the performance (x16/x8 vs x16/x16). I don't think CPU power in and of itself is really a problem (Tim showed the performance loss from underclocking his CPU by a lot and it was only a few percent). But it depends on your budget, etc. For me, data transfer happens to be the bottleneck, and I wanted to know that I had the maximal amount of options in terms of expanding the number of GPU's etc (originally I only had one 980 ti).

Hope that's somewhat useful.

  • 47 months ago
  • 2 points

Since I am still doing decently OK with my current rig, I guess I would wait 3 months until the release of Broadwell-E + Nvidia Pascal and hopefully see some price drop on the "older" tech (5930K + 980ti SLI).

Thanks alot for the detailed answer and good luck with your research.

  • 47 months ago
  • 1 point

See, that's what I was thinking about doing too -- waiting to see what Pascal GPU's really looked like once they came out and/or taking the price break on my current build. I was reading, however, that 980 Ti's/older graphics card prices start going UP after new generations come out, since (1) they're out of production and (2) people want to SLI their older rigs. Don't know how true this is in practice, but food for thought.

That being said, Pascal looks like it's going to be worth the wait, and I'm crossing my fingers (mostly for you people that are waiting) that the rumors about Nvidia lying at CES don't turn out to be true. Or, if they are true, that the actual production schedule isn't hugely affected (see http://techfrag.com/2016/01/12/nvidia-lied-about-the-pascal-gpu-at-ces-that-their-ceo-showed-off/).

  • 47 months ago
  • 2 points

Wow what a build. I'd have to game on this as research for how badass this machine is.

+1

  • 47 months ago
  • 2 points

Oh hey this note came up.

COMPATIBILITY NOTES SLI is disabled when the ASRock X99 Extreme6 ATX LGA2011-3 Motherboard M.2 slot is used.

So did it disable the SLI?

  • 47 months ago
  • 1 point

OK so this is where I'm not 100% sure, but I'm 98% sure that no, it does not disable SLI. I haven't actually used this for any games, and I'm running Ubuntu (not Windows), so I haven't yet found a way to test the actual SLI.

The fix that I used was just to avoid the motherboard's native M.2 slot; I bought a PCIe -> NVME adapter, and used that to host my SSD on the third PCI slot. So no reason for SLI to be disabled, but hey, this motherboard has given me many surprises before, most of them unpleasant.

  • 47 months ago
  • 1 point

Man. Good thing you know what you are doing. Maybe get the GeForce experience program to work and just see if it recognizes the two cards in SLI on the information about your pc area. I've never used Ubuntu on anything. Do you think any of your functionality issues are directly related to compatibility with your operating system?

  • 47 months ago
  • 1 point

Good thing you know what you are doing.

"NNNOOOOOOTTTTT"

  • Borat

Maybe get the GeForce experience program to work and just see if it recognizes the two cards in SLI

I remembered what I did now -- I actually used the CUDA "samples" to test SLI, though this isn't a sure-fire test (gaming would be a surefire test). Running the "bandwidthTest" with --device=all, tests the speed of Device -> Host and Device->Device. Device-> Device has a bandwidth of 493,572.7 MB/s, so I would imagine this has to mean that the cards can at least talk to each other via the SLI bridge (Device->Host is 25,743.6 MB/s). I imagine, though, that there could possibly still be problems with device drivers/etc. that could cause problems in real-world gaming, but everything points to SLI working.

Do you think any of your functionality issues are directly related to compatibility with your operating system?

I've had some hiccups and annoyances; from what I hear, Windows definitely works much better with GPUs than Ubuntu (though I haven't actually tried it yet). So far, for what I want, the problems have been pretty minimal beyond configuring dual screens.

  • 47 months ago
  • 1 point

Your research has done you well....

+1

  • 46 months ago
  • 1 point

Great build man. Looks great.

  • 43 months ago
  • 1 point

Amazing....so much TB....0.8% of budget is on keyboard/mice XD

How has this build served you so far? :)

  • 43 months ago
  • 1 point

Haha, yea if I spent another dollar on this rig my partner would probably smother me with a pillow. The rig has been fantastic. Dual screens are amazing, and everything is just ridiculously fast. I don't do any gaming with it (yet), because if I did I wouldn't stop to eat or sleep, let alone to debug Python scripts for my thesis. Sigh.

[comment deleted by staff]
[comment deleted]
  • 47 months ago
  • 1 point

Why?

  • 47 months ago
  • 7 points

cause if you hold it to your ear you can hear the sea

  • 47 months ago
  • 1 point

Hahaha +1

  • 47 months ago
  • 1 point

Best comment I've seen in months haha.

  • 47 months ago
  • 1 point

Wait, what? When did i comment this? "Why" what? I'm confused..

  • 47 months ago
  • 1 point

Oh, i see now. I asked why SeaSonic. Why did my comment get disliked for that? Sounds like SeaSonics the only good PSU manufacturer. Not true. Evga is excellent too. Great quality control. Great products.

[comment deleted]
  • 47 months ago
  • 1 point

Thanks for the suggestion; I'll keep that in mind if I decide to upgrade at some point. Any particular reason why I should go with SeaSonic instead of some other company?

[comment deleted]
  • 47 months ago
  • 1 point

Thanks for the tip!

  • 47 months ago
  • 2 points

Also, for any psu stuff, check out Jonny Guru's website, they have fantastic reviews and help

  • 47 months ago
  • 1 point

That doesn't sound fanboyish at all.

You could at least tell him what's wrong with it.


From your other comment.

Failure rate is minimal

I'd like to know your sources.

and they use high quality materials.

So does a bunch of other manufacturers.

[comment deleted]
  • 47 months ago
  • 1 point

EVGA (T2) Titanium is made by Super Flower. Platinum is both PS and P2. PS is Seasonic, P2 is Super Flower.

Corsair's Titanium is made by Flextronics. So is the rest of the AXi. The only current Corsair Seasonic is the AX which is only 2 models. The other platinum is the HXi which is CWT.

Cooler Master has no Titanium units, and only 1 current Platinum which is Seasonic. Their older Silent Pro Platinum was Enhance.

XFX uses only Seasonic. Seasonic has no current Titanium platforms.

SeaSonic is an ODM

I assume you mean OEM.

And no, it's not fanboyish, it's a fact.

What is? That their failure rates are minimal? Where's your proof? Not that I'm denying it.

Superflower is also a nice option.

Leadex>X KM3S

And learn how to read "SEASONIC MADE" PSUs

OK... Where does this come in. I never once questioned anything involving "seasonic made"

[comment deleted by staff]
[comment deleted by staff]
[comment deleted by staff]
[comment deleted]
  • 47 months ago
  • 1 point

I assumed it was just a typo.

So are you going to answer my questions?

[comment deleted]
  • 47 months ago
  • 1 point

I know that.

What I'm asking for is a source.

FSP make their own units, but that doesn't mean that their failure rates are low.

[comment deleted]
  • 47 months ago
  • 1 point

Let's start from the beginning.

The tier list has many many flaws. The most noticeable one being the fact that they put power supplies based on the same platform in different tiers. Another one is not splitting up series' that have the same name, but different platforms. (EVGA GS)

So, please tell my why you pointed me to this when my question asked for a source saying they have good quality control.

FSP, CWT, and Chicony makes a lot of units, but that doesn't mean anything. I don't see how an OEM points to good qc.

[comment deleted by staff]
  • 47 months ago
  • 1 point

Please don't copy / paste whole blocks of content from other sites - just the link is fine.

[comment deleted by staff]
  • 47 months ago
  • 3 points

You sound like a grad student yourself!!! Self-loathing is part of the job description

[comment deleted by staff]