Fast connections between PC's

Madmaz · 2025-11-07T11:30:52+0000

Hi - I have a cluster of PC's that I use to collect data from my astronomy kit - and I want to figure out the best/fastest way to transfer data files (usually a few Gbytes) from each PC onto a central store. Current setup has a 1Gb switch that they are all connected to. Would a 10Gb switch help, or fibre links? Any ideas welcome

Hunterlit · 2025-11-07T12:02:38+0000

Even a 2.5gb switch would be an upgrade. Do you have Jumbo frames enabled?

mattyfez · 2025-11-07T12:09:20+0000

'a few gb's' isn't really that much over gigabit, how long are transfers actualy taking and what actual size are the transfers?

Is there a bottle neck on your lan somewhere?

Meddling-Monk · 2025-11-07T12:23:38+0000

How often are these other PCs needing to transfer files to the central store?

What storage medium are you using for the central store? HDD or SSD?

Caged · 2025-11-07T12:41:15+0000

Even if this was 100GB a day I wouldn't be that bothered about upgrading past 1Gb.

Madmaz · 2025-11-07T12:44:28+0000

thanks - so main issue is when I am collecting data from the 4 active PC's and want to store then on either a single PC, on my local home cloud unit or a local NAS drive - each files 300Mb takes over 60 seconds per files and I might have 50+ per PC - so takes ages to get the data from the observatory to a single storage point?

Armageus · 2025-11-07T12:49:13+0000

300Mb 60 seconds? That seems slow even for hard drives

Have the PCs/NAS at each end got SSDs? How are you transferring the files? SMB Shares or some other protocol? Push vs Pull?

Madmaz · 2025-11-07T12:59:20+0000

all the units have SSD - transfer are via simple file explorer transfer (could this be the issue?) - which I assume are "push" actions ffrom one PC to other? All starting to get beyond my understanding hence the questions cheers

aaronyuri · 2025-11-07T13:02:48+0000

Madmaz said:
thanks - so main issue is when I am collecting data from the 4 active PC's and want to store then on either a single PC, on my local home cloud unit or a local NAS drive - each files 300Mb takes over 60 seconds per files and I might have 50+ per PC - so takes ages to get the data from the observatory to a single storage point?

Based on your file size and transfer time, you don't have a throughput bottleneck with 1Gb connectivity. Your bottleneck lies elsewhere. On 1Gb, a 300MB (assuming megabyte) file should take a few seconds (transfer speed should be around 120MB/s).

Outline these things for us:

Frequency of transfers
Number of files / total transfer size - I assume 15GB
Storage on each device that transmits and receives these files - PC 1 - HDD, PC 2 SSD, etc.

Edit: just saw your newer post. If you are using SSDs (even SATA-based) these should be maxing out the 1Gb throughput no problem. You have an issue somewhere.

Edit 2: seen you're in Hereford. PM me and we could look at resolving this issue in person and automating this transfer for you.

Armageus · 2025-11-07T13:06:21+0000

Are all your computers actually connecting at Gigabit? Presumably they are plugged into a Switch (is that showing a gigabit light?)

Presumably no wifi or powerline extenders are involved anywhere, as they are a very likely cause for any transfer speed issues

Caged · 2025-11-07T13:19:43+0000

You need to describe the process. How is this data transferring, you are dragging it to a central shared folder? Where are you currently storing these files, if it's a NAS what is the brand and model? Draw out the network topology.