Downloads stop if I impose a speed limit

Bes · 13 Nov 2010 at 12:43

Hi,

Having an odd problem on my RH box...

Basically if I download a file using curl, but use --limit-rate to keep it under around 30K/sec, the download dies after a minute or so. The problem gets worse (i.e. the download dies in a shorter time) the slower you set --limit rate. For instance, at 7K, the download dies in about 12 seconds.

I have ruled out iptables and (I think) dodgy TCP configuration.

Any ideas? as I have spent ages on it and it's driving me mad.

Thanks

Roland0 · 13 Nov 2010 at 13:08

I don't know about curl, but some programs limit the (average) download rate by just stopping downloads, and then periodically allowing full download speeds In order to average out at the speed limit. So if you set the limit low, and you have a fast connection, it will effectively stop the download, apart from occasional bursts. Quite possibly not what is happening in your case, but a possibility, you may want to look at.

Bes · 13 Nov 2010 at 13:35

Thanks for the suggestion, that is definitely not what is happening here. Once it hits zero, even leaving it for an hr means it never comes back.

The 'trigger' point is 28K/sec- i.e. 28K fail, 29K= ok. K=Kilobytes

Thanks

Fourstar · 13 Nov 2010 at 14:18

Maybe you have an outdated version/library somewhere?

I gave it a test on my Ubuntu 10.10 install at it looks ok.

Have you tried another server? You could try the file below.

Code:

:~$ curl -O --limit-rate 7K http://www.mirrorservice.org/sites/download.openoffice.org/stable/3.1.1/OOo_3.1.1_Win32Intel_install_en-US.exe
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  3  150M    3 5055k    0     0   7158      0  6:06:41  0:12:03  5:54:38  7157
^C
:~$ curl --version
curl 7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18

Bes · 13 Nov 2010 at 14:50

Hard to say if it is a lib problem. Would doubt it as it's just RH EL4 off the CD.

It's definately not the file I am downloading as other (non RH) systems can DL at 7K.. Hmmm

russ0r · 13 Nov 2010 at 17:25

I'm not that great with curl (I usually use wget) but I had a quick look and:

Code:

-Y/--speed-limit   Stop transfer if below speed-limit for 'speed-time' secs
 -y/--speed-time    Time needed to trig speed-limit abort. Defaults to 30

You haven't got these set somewhere have you? In a config file or something?

Bes · 14 Nov 2010 at 09:19

Hi,

Those are part of CURL, right?

No it's not CURL itself, it's something environmental, as if I use that RH box as a proxy (This is what it is for), and download through it with Firefox for example, but set the limit to 7K using the FF speed limiter, I get the same problem.... Download fails after a certain amount of time.

Thanks