building a new work station for processing a vast amount of mobile mapping data

R9mpo

R9mpo

R9mpo

Associate
Joined
25 Sep 2017
Posts
140
hello,

looking for a little help.

I am looking to build a PC with as much power as possible. it will soley be used for processing a very vast amount of mobile mapping data.

My original thought was to go for something like a threadripper 16 cores or a i9 equivelent.
64/128gb of ram
1080ti or titan XP?

But am i best off with something like xeon?
Graphics card wise the company recommend a quadro graphics card which is best/ most cost effective?

i would like to go with a large super fast SSD of 1tb minimum.

and look for a internal drive of 8-10tb.

any help would be appreciated as i need to build this PC / work station ASAP.
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
What software would be used ?
Might help with research to see what is best with your cash
Could be the case of GPU compute power being stronger then any core CPU or the other way around .
Hate to speak solid pieces on both to find out one isn't utilised as much or at all to the other
 

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
Pix4d, leica Pegasus mobile mapping software etc

orbitalwalsh said:
What software would be used ?
Might help with research to see what is best with your cash
Could be the case of GPU compute power being stronger then any core CPU or the other way around .
Hate to speak solid pieces on both to find out one isn't utilised as much or at all to the other
Click to expand...
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
R9mpo said:
Pix4d, leica Pegasus mobile mapping software etc
Click to expand...

did some digging !

Direct from Pix4d site - recommended specs

· Windows 8, 10 64 bits.
· CPU quad-core or hexa-core Intel i7/Xeon.
· GeForce GPU compatible with OpenGL 3.2 and 2 GB RAM.
· Hard disk: SSD.
· Small projects (under 100 images at 14 MP): 8 GB RAM, 15 GB SSD Free Space.
· Medium projects (between 100 and 500 images at 14 MP): 16GB RAM, 30 GB SSD Free Space.
· Large projects (over 500 images at 14 MP): 32 GB RAM, 60 GB SSD Free Space.
Very Large projects (over 2000 images at 14 MP): 64 GB RAM, 120 GB SSD Free Space.
Click to expand...

in relation to GPU or CUDA Support

  • An SSD hard drive can speed up processing.
  • The graphic card may have an improvement on the processing speed for step 1 and step 2 (if the graphic card is compatible with CUDA (NVIDIA Graphic Cards). Processing time of step 3 is not affected by the GPU. The GPU affects considerably the visualization of the rayCloud.
Click to expand...
https://support.pix4d.com/hc/en-us/articles/203405619

Pix4Dmapper leverages the power of Nvidia GPUs for faster processing.

The speed increase depends highly on the project, the number of images, the image size, and the image content. For example, with a GTX 970 a speedup between 10% and 75% on step 1. Initial Processing, and around 10% in step 2. Point Cloud and Mesh was observed. As a rule of thumb, projects with high overlap, high image content and thus with a high number of keypoints benefit more from the speedup.



Minimum requirements for CUDA speedup

Nvidia GPU (GTX, Mobile, Tesla, Titan, Quadro) with at least 2GB RAM, supporting CUDA 6.5 with the latest driver installed (starting at least from 343).
Click to expand...

Testing!

Shows Intel trumps AMD, latest 8700k matches Thread ripper! so guessing core speed and IPC plays a strong part!

https://www.pugetsystems.com/recommended/Recommended-Systems-for-Pix4D-207/Hardware-Recommendations

GPUs! Quadro isn't as fast as Geforce consumer cards- seems GTX 1060 will be fine for this build but if you want the best then 1080ti
wonder if you can SLi them ?

****maybe not
In terms of processing, using SLI technology would not help the processing since it would not increase the available RAM, which is the bottleneck for processing.

SLI would improve the rayCloud visualization for very large projects.

Having two GPUs not connected using SLI would improve the GPU performance for the processing since the state of the GPUs is periodically checked and the less busy one is used.
Click to expand...

https://www.pugetsystems.com/labs/articles/Pix4D-GPU-Comparison-GeForce-Titan-and-Quadro-1085/

and your spec list is a rare one and BENEFITS from NVMe drive!!!

We have not yet performed our own testing to quantify that, but with how little a fast SSD costs these days we strongly advise selecting a M.2 NVMe type drive if at all possible
Click to expand...

Funny, for one program, that company that mentioned Quadro card- it would perform 3x slower then a geforce card!

Also, as mention for XEON , So stick to z370/x299 if you can
a CPU with twice the number of cores but half the clock speed will always come out as a loser.
Click to expand...

Now, for members to help with above info- whats your max budget?
 
Last edited:

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
Th
orbitalwalsh said:
did some digging !

Direct from Pix4d site - recommended specs



in relation to GPU or CUDA Support


https://support.pix4d.com/hc/en-us/articles/203405619



Testing!

Shows Intel trumps AMD, latest 8700k matches Thread ripper! so guessing core speed and IPC plays a strong part!

https://www.pugetsystems.com/recommended/Recommended-Systems-for-Pix4D-207/Hardware-Recommendations

GPUs! Quadro isn't as fast as Geforce consumer cards- seems GTX 1060 will be fine for this build but if you want the best then 1080ti
wonder if you can SLi them ?

****maybe not


https://www.pugetsystems.com/labs/articles/Pix4D-GPU-Comparison-GeForce-Titan-and-Quadro-1085/

and your spec list is a rare one and BENEFITS from NVMe drive!!!



Funny, for one program, that company that mentioned Quadro card- it would perform 3x slower then a geforce card!

Also, as mention for XEON , So stick to z370/x299 if you can


Now, for members to help with above info- whats your max budget?
Click to expand...
Thank you for your help!
Max budget would be 3.5-4k
Would need a Tb of SSD and 4tb HDD
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
R9mpo said:
Th

Thank you for your help!
Max budget would be 3.5-4k
Would need a Tb of SSD and 4tb HDD
Click to expand...

SSD for OS and other apps
1tb NVMe SSD drive for your two key software
4TB HDD
14 cores with cooler to overclock it ! Remember speed counts !!!!
Gtx 1080ti
Gigabyte for UK RMA and support
64GB of ram at the top scale of work
Damn good PSU , 1080ti normally needs 650w but you'll be powering that 14 core chip and will use power when overclocked

My basket at Overclockers UK:
Total: £4,009.54 (includes shipping: £14.70)


Toning some of the parts down and dropping to gtx 1080 lands you 18 core i9 monster

My basket at Overclockers UK:
Total: £4,021.50 (includes shipping: £14.70)​
​
 
Last edited:

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
orbitalwalsh said:
SSD for OS and other apps
1tb NVMe SSD drive for your two key software
4TB HDD
14 cores with cooler to overclock it ! Remember speed counts !!!!
Gtx 1080ti
Gigabyte for UK RMA and support
64GB of ram at the top scale of work
Damn good PSU , 1080ti normally needs 650w but you'll be powering that 14 core chip and will use power when overclocked

My basket at Overclockers UK:
Total: £4,009.54 (includes shipping: £14.70)


Toning some of the parts down and dropping to gtx 1080 lands you 18 core i9 monster

My basket at Overclockers UK:
Total: £4,021.50 (includes shipping: £14.70)
​
Click to expand...
That's amazing .going to try and push for the better i9 and gtx1080ti too I think
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
R9mpo said:
That's amazing .going to try and push for the better i9 and gtx1080ti too I think
Click to expand...

£4100, tried using 750w PSU but seeing how much draw that chip can do with BIG overclocks not worth it ! Plus 10 he warranty

My basket at Overclockers UK:
Total: £4,107.90 (includes shipping: £14.10)
​


Review of that monitor CPU, and why I didn't switch is down to lessor model or other brand
Aorus 7/9 has one of the coolest VRM temps and has Dual 8 pins for the CPU, the 3 and some others just have 8+4 or single 8.
That and with UK RMA and rep support along with OCUK support, you want that expensive Mobo out of action for little as possible time wise if it does go down

https://www.techradar.com/reviews/intel-core-i9-7980xe

Seems it can have 4.4ghz on all 18 cores !! To be honest would just sit at 4Ghz

Dropping to a 16 core does bring it under £4k!
Will say though, ram is currently on deal and long with GPUs, normally mid Wednesday current sales end and new ones take over. That 32GB kit or both of them and 1080ti are on cracking deals !!!!

I'm sure with £4k trolley you can swing free delivery as well !
 
Last edited:

edscdk

edscdk

edscdk

Soldato
Joined
17 Jul 2008
Posts
7,369
Before dropping 1k on a cpu check the software supports it! If the Intel 6 Core can match a 12 or 18 core tr it would suggest to me the software hates amd or cannot use that many cores..
 

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
edscdk said:
Before dropping 1k on a cpu check the software supports it! If the Intel 6 Core can match a 12 or 18 core tr it would suggest to me the software hates amd or cannot use that many cores..
Click to expand...
We currently run Ryzen 7 1700x
64 gb ram
Rx480
500gb SSD
2tb HDD
650w PSU

They out perform our older i7s we have and they are only 4c 4t I believe
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
Why I linked review and software site as reference .
But just core /thread number but its speed and IPC strength

Normally, it's always thread ripper hands down for core count and price...



edscdk said:
Before dropping 1k on a cpu check the software supports it! If the Intel 6 Core can match a 12 or 18 core tr it would suggest to me the software hates amd or cannot use that many cores..
Click to expand...
pic_disp.php
 
Last edited:

JediFragger

JediFragger

JediFragger

Caporegime
Joined
18 Oct 2002
Posts
29,812
Wow, what a monster of a system! Let us know what you go with in the end please @R9mpo

Also it's hard to believe that the 7980XE only gets a 30% performance bump over the 8700k, but I guess in this line of work then every second counts! From a cost £/performance viewpoint then the 7940X comes out on top (in comparison).
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
JediFragger said:
Wow, what a monster of a system! Let us know what you go with in the end please @R9mpo

Also it's hard to believe that the 7980XE only gets a 30% performance bump over the 8700k, but I guess in this line of work then every second counts! From a cost £/performance viewpoint then the 7940X comes out on top (in comparison).
Click to expand...

Been checking through .lot of reviews builds and bench, 14 core is indeed the best, kept the 1080ti.. doesn't make to much difference over 1080 but in more cases of workloads , it's effected performance greater then the 18 core monster ! Also the second program listed , very few articles , seems more heavy on Cuda cores...

So, with further research, dropping to 14 core instead of 18, shouldn't notice to much difference in performance across all stages.
Gtx 1080ti so you can get max 60-70% boost at stage 1. *aorus card added - for heat and sound, and higher power phase count! again speed is key and should hold its OC for long periods.
Stage 3, listed second from top Samsung NVMe, this is retail drive so there are drivers for better support then using Intel's as well as faster times, seems you'll gain max 10% boost and reading everywhere , stage 3 is all about storage speed .
Ram also taken a slight step in speed, with core speed and Cashe speed overclock should gain the benifit, unlike gaming were it doesn't do much .
Silent case ! Silent beefy cooling !
Also, PSU got an upgrade * MORE WATTS! higher quality. 12 year warranty!

***sorry just redid this 2pm- underestimated the power draw of i9s when overclocked! 350w maybe for that chip at 4.4ghz

@R9mpo
My basket at Overclockers UK:
Total: £4,070.54 (includes shipping: £14.70)


Should OC to 4.5ghz nicely without overloading the VRM and producing to much heat its self. board vrm cooling one of the best so should be covered .
Again, to get the best for your system, you'll need to overclock it ! As if not, Thread Ripper looks more attractive for pricing
Wished again could have kept the Gigabyte card for RMA, but that price is a steal !
***added the Gigabyte card- for £4k- you want the best RMA as a just in case​
 
Last edited:

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
JediFragger said:
I have a 240mm Silent Loop and it's excellent, virtually silent. The 7940X will push it harder though.
Click to expand...

done a few silent loops now, silence is bliss. Eisbaer LT handles the 8700k perfectly but as you say this core will push higher as well as cause higher noise. Seems they havent put Pure wing Fans on the LT series, only the Stand Eisbaer and Silent loops have them .

weird not quoting Thread Ripper, but nice for a change. will have to dig deeper into that other program

jesus, been reading up on processing taking days! @R9mpo whats the longest its taken using Pix4d for a given work ? and what was it done with ?

also, do you know how many given photo's and at what resolution do you use for a high workload task?
seems lot of Pix4d users use Agisoft Ram calculator to work out how much ram they my require as its the nearest program to base off, pix4d dont state much .
but given rule is 64GB of ram

http://www.agisoft.com/pdf/tips_and_tricks/PhotoScan_Memory_Requirements.pdf

also changed above spec to allow Gigabyte Aorus card instead of basic gigabyte card , for heat and sound...

seeing some peoples builds with 512gb of ram and triple 1080Ti's even though return is poor for multicard set up - knocks hours of total work time due to the massive scale of the task..

just had to update list, specially on PSU Wattage! insane! this PC might run crysis
 
Last edited:

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
orbitalwalsh said:
done a few silent loops now, silence is bliss. Eisbaer LT handles the 8700k perfectly but as you say this core will push higher as well as cause higher noise. Seems they havent put Pure wing Fans on the LT series, only the Stand Eisbaer and Silent loops have them .

weird not quoting Thread Ripper, but nice for a change. will have to dig deeper into that other program

jesus, been reading up on processing taking days! @R9mpo whats the longest its taken using Pix4d for a given work ? and what was it done with ?

also, do you know how many given photo's and at what resolution do you use for a high workload task?
seems lot of Pix4d users use Agisoft Ram calculator to work out how much ram they my require as its the nearest program to base off, pix4d dont state much .
but given rule is 64GB of ram

http://www.agisoft.com/pdf/tips_and_tricks/PhotoScan_Memory_Requirements.pdf

also changed above spec to allow Gigabyte Aorus card instead of basic gigabyte card , for heat and sound...

seeing some peoples builds with 512gb of ram and triple 1080Ti's even though return is poor for multicard set up - knocks hours of total work time due to the massive scale of the task..

just had to update list, specially on PSU Wattage! insane! this PC might run crysis
Click to expand...

we have had a few projects that have taken days around 4-5000 photos at 12mp. sometimes we use a drone that has a 36mp sensor so it will up at time but the photo count will be much lower.
that was done with the above system r7 1700x 64gb ram rx480 etc.

a normal project is around 1-2000 photos dependent if it split into smaller sections for post processing.
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
R9mpo said:
we have had a few projects that have taken days around 4-5000 photos at 12mp. sometimes we use a drone that has a 36mp sensor so it will up at time but the photo count will be much lower.
that was done with the above system r7 1700x 64gb ram rx480 etc.

a normal project is around 1-2000 photos dependent if it split into smaller sections for post processing.
Click to expand...

***heads up, looks like Pix4d doesn't support AMD cards - only GPUs with Cuda cores - might be worth sticking a gtx 1060 in that and selling the card to a miner :)

The list I created is the best I could come up with using your budget - look like ram could be slightly over .. not sure if your able to use task manager to see max ram you used on the above work but you could think of it this way- you now have a far more powerful machine with more ram so in theory you could 32MP sensor with 5k photo limit!
seems from research intel 6 core 8700k is faster then Ryzen 8 core so this 14 core monster overclocked should fly! again stress key the point of over clocking it! doesn't have to be extreme but getting all cores to at least 4GHz, which should be straight forward and easy to do is key to getting the best out of it and your money back!
on a side note- hope you've also got that 1700x clocked to 4.9ghz all cores with 3200hz ram!


orbitalwalsh said:
My basket at Overclockers UK:
Total: £4,070.54 (includes shipping: £14.70)


​
Click to expand...

deals change roughly 12pm today (every wednesday ) - also worth trying if you do order from OCUK to blag free shipping ;)

hope all the above info has helped - certainly a different animal all together and honestly first machine i've quoted that actually needs i9 core and NVMe drive!

I have 2TB of samsung NVMe just wasted on games haha

was really surprising to see thread ripper beaten by i9 to be honest, normally with mainstream ryzen vs coffeelake its down to programs being optimised for intel like Photoshop but seems your programs are just about blunt power and speed!

just so you have a full spread, thread ripper version - which is cheaper- everything else left the same

My basket at Overclockers UK:
Total: £3,708.57 (includes shipping: £14.70)


least with Ryzen platform - you should have a better upgrade path - x499 is coming Q4 along with Thread ripper refresh which should offer increased speed and ram speeds- then year after or 2020 you'll have Zen 2 Thread Ripper which should be an animal .
ideally you';d want to pair the above with 3400/3600 ram for future CPUs are getting best performance but then costs rise to match intel or higher .

but least you have both versions, and should see good returns over current Ryzen 1700X system

Tried getting i9 7980xe as seems that would have more of an impact then gtx 1080ti over 1080/70ti in terms of performance to price - just overshoots


DE
My basket at Overclockers UK:
Total: £4,139.54 (includes shipping: £14.70)
​
 
Last edited:

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
orbitalwalsh said:
***heads up, looks like Pix4d doesn't support AMD cards - only GPUs with Cuda cores - might be worth sticking a gtx 1060 in that and selling the card to a miner :)

The list I created is the best I could come up with using your budget - look like ram could be slightly over .. not sure if your able to use task manager to see max ram you used on the above work but you could think of it this way- you now have a far more powerful machine with more ram so in theory you could 32MP sensor with 5k photo limit!
seems from research intel 6 core 8700k is faster then Ryzen 8 core so this 14 core monster overclocked should fly! again stress key the point of over clocking it! doesn't have to be extreme but getting all cores to at least 4GHz, which should be straight forward and easy to do is key to getting the best out of it and your money back!
on a side note- hope you've also got that 1700x clocked to 4.9ghz all cores with 3200hz ram!




deals change roughly 12pm today (every wednesday ) - also worth trying if you do order from OCUK to blag free shipping ;)

hope all the above info has helped - certainly a different animal all together and honestly first machine i've quoted that actually needs i9 core and NVMe drive!

I have 2TB of samsung NVMe just wasted on games haha

was really surprising to see thread ripper beaten by i9 to be honest, normally with mainstream ryzen vs coffeelake its down to programs being optimised for intel like Photoshop but seems your programs are just about blunt power and speed!

just so you have a full spread, thread ripper version - which is cheaper- everything else left the same

My basket at Overclockers UK:
Total: £3,708.57 (includes shipping: £14.70)


least with Ryzen platform - you should have a better upgrade path - x499 is coming Q4 along with Thread ripper refresh which should offer increased speed and ram speeds- then year after or 2020 you'll have Zen 2 Thread Ripper which should be an animal .
ideally you';d want to pair the above with 3400/3600 ram for future CPUs are getting best performance but then costs rise to match intel or higher .

but least you have both versions, and should see good returns over current Ryzen 1700X system

Tried getting i9 7980xe as seems that would have more of an impact then gtx 1080ti over 1080/70ti in terms of performance to price - just overshoots


DE
My basket at Overclockers UK:
Total: £4,139.54 (includes shipping: £14.70)
​
Click to expand...

i think we are going to go with the 14core i9 64gb ram with 1080ti. obviously with the recommended hard drives.

We dont currently have a our ryzen overclocked and i dont think we are running high speed Ram either.
 

orbitalwalsh

orbitalwalsh

orbitalwalsh

Soldato
Joined
1 Dec 2015
Posts
18,514
R9mpo said:
i think we are going to go with the 14core i9 64gb ram with 1080ti. obviously with the recommended hard drives.

We dont currently have a our ryzen overclocked and i dont think we are running high speed Ram either.
Click to expand...

Nice ! Well that ram should drop in the Ryzen so you'll be able to see your self just running higher spec - up to ,3200hz can make a big impact with AMD CPUs. Gaming it's 5-10fps from 2133hz default but with workstation loads, it's a lot !
Worth a try in your spare time. I know the 1700x can do 3.8ghz with its auto overclock but it might not hold it when under load, manual overclock should be able to do 3.9ghz across all cores all the time with a decent cooler

But seems that RX card won't help your current system at all :(

Let us know how you get on . Very cool program , different researching something like that

Limits will probably be the PSU, if it was 18 core then you'd be on the limit for 4.5ghz OC , 14 core should pull 50-80w less . But yeah should hit 4.4ghz OC with that core without meltdown and needing a nuclear plant

Not sure if your going through company/vat registered . Should be able to call OCUK and go through business and vat off ?
Is a way to config basket to take VAT off if that's the case .
 
Last edited:

R9mpo

R9mpo

R9mpo

Associate
OP
Joined
25 Sep 2017
Posts
140
orbitalwalsh said:
Nice ! Well that ram should drop in the Ryzen so you'll be able to see your self just running higher spec - up to ,3200hz can make a big impact with AMD CPUs. Gaming it's 5-10fps from 2133hz default but with workstation loads, it's a lot !
Worth a try in your spare time. I know the 1700x can do 3.8ghz with its auto overclock but it might not hold it when under load, manual overclock should be able to do 3.9ghz across all cores all the time with a decent cooler

But seems that RX card won't help your current system at all :(

Let us know how you get on . Very cool program , different researching something like that

Limits will probably be the PSU, if it was 18 core then you'd be on the limit for 4.5ghz OC , 14 core should pull 50-80w less . But yeah should hit 4.4ghz OC with that core without meltdown and needing a nuclear plant

Not sure if your going through company/vat registered . Should be able to call OCUK and go through business and vat off ?
Is a way to config basket to take VAT off if that's the case .
Click to expand...
Yup all through business .
Just wanted to say a huge thank for your help! It honestly has made such a difference and pointed us In the right direction.

We will be ordering after Easter weekend so will see how it goes
 
You must log in or register to reply here.
Back
Top Bottom