Testing FTP Server load with 50 or so FTP Connections/Image transfers

DJMK4

DJMK4

DJMK4

Soldato
Joined
1 Dec 2004
Posts
23,076
Location
S.Wales
Were looking at stress testing our camera image storage server for customer of ours that have IP camera's, they are motion sensored camera's which take images and send them via FTP to our server.

We have set-up 50 test accounts through our web-based GUI (one directory for each camera for the images) the attributes for the test is that we want to send 5 pictures to each directory (so basically 5 x 50) so we can test the load on the server during multiple FTP connections and multiple file transfers.

Is there any app already out that does this? Or would it be best to go down developing a script for this?

I have already asked in the development forum about self developing one of these apps, this thread is more to find out if something like this already exists or if it would be best to use a self developed script.

Thanks all :)
 
Anyone?

Iv starting making a dos batch script this morning however at the moment all iv been able to sort of work out is how to start creating the script which constist of

batch file
Code: 
@echo off
ftp -s:commands.txt
pause
exit

commands.txt
Code: 
open ftp.blah.com
username
password
cd testfolder
binary
testimage.jpg
quit

However what this will do is simulate 1 ftp connection sending an image to a single folder.

I need it to simulate 50 ftp connections with each ftp connection sending 5 images to its directory folder.
 
FTP is very low overhead and uploading a few images isn't going to put any appreciable load on the server.

You'd have to have hundreds of users attempting to upload simultaneously to cause any issues, and the biggest would probably be the limit of the write disk I/O on the server.
 
I understand this, and I have heard this somewhere else, but I think we will still need to run the test to see how much we could upscale. Hence why we have set a base test as 50 concurrent FTP connections uploading 5-10 images at a time to their directories.

As I have been asked to carry out this testing I still feel like I have to do it, even though im thinking the same as your post above.
 
beavis said:
http://jakarta.apache.org/jmeter/ is what I've used for production FTP testing. It's pretty straight forward, let me know if you have any problems.
Click to expand...

I shall have a look in to this assuming I can use it to test the performance of multiple FTP connections transferring the test image 5 times per connection, or 5 test images per connection to each one of the directories concurrently.

Thanks
 
Sorry to sound abit dumb but where does this have to be installed? I have hardly any experience working with open source binary stuff.
 
Nevermind, im getting there :) Im hoping this app works and allows me to send the test files TO the FTP server from my laptop, as by looking at the documentation on their website it tests download from the FTP server which is not really what im looking for

Also, does this simulate 50 seperate FTP connections? as these camera will all be connecting to the ftp server so there will be 50 or so concurrent FTP connections open.

Say if my ftp directory has the following folders:

camera1
camera2
camera3
etc etc

I need it to connect to each one of these and send multiple files
 
Last edited:
Does anyone have any information as to how I can set this FTP request up to send multiple images to multiple directories then?
 
[root@vhc-web5-ftp1 ~]# netstat -an |grep :21 |wc -l
73
[root@vhc-web5-ftp1 ~]# uptime
13:08:52 up 21 days, 5:15, 1 user, load average: 0.11, 0.17, 0.19

I *really* wouldn't worry about load...
 
All you have to watch out for is bandwidth with FTP. FTP is such an old protocol now it will run happily with hundreds of users on old boxes.
 
I agree with you both, but unfortunatly I have been given a task by the operations director, and the MD, it is my duty to provide them with some stats so they can get an idea of how much they could upscale the system.

Me saying to them "Yeah it should be alright" will not be sufficiant although I do agree with it. They need hard evidence of what the load is like based on these 50 concurrent connections sending 5-10 images (per connection) all at the same time.
 
DJMK4 said:
Does anyone have any information as to how I can set this FTP request up to send multiple images to multiple directories then?
Click to expand...

After unpacking double click on bin\jmeter.bat.
Create a new test plan, add a thread group.
Within the thread group and set number of threads (50?) and how many times you want to it loop.
edit->add->sampler->ftp request.
Specify ip, remote file, local file, select 'put(STOR)', username/password. Use binary mode if it's a binary file rather than ascii.
Add various listeners to provide results.

You can use variables in the remote file to make the filenames unique, for example 'Upload\something.${__time}.${__threadNum}' would insert the time and thread number in the filename.

edit - sorry missed you other post. You can also use __threadNum in the path if you want to upload to Camera1 etc. For example
Code: 
Upload\Camera${__threadNum}\Picture${__time}
would give you Camera1 to Camera50 (or however many threads you've set up) with a unique filename for each upload.
 
Last edited:
beavis said:
After unpacking double click on bin\jmeter.bat.
Create a new test plan, add a thread group.
Within the thread group and set number of threads (50?) and how many times you want to it loop.
edit->add->sampler->ftp request.
Specify ip, remote file, local file, select 'put(STOR)', username/password. Use binary mode if it's a binary file rather than ascii.
Add various listeners to provide results.

You can use variables in the remote file to make the filenames unique, for example 'Upload\something.${__time}.${__threadNum}' would insert the time and thread number in the filename.

edit - sorry missed you other post. You can also use __threadNum in the path if you want to upload to Camera1 etc. For example
Code: 
Upload\Camera${__threadNum}
would give you Camera1 to Camera50 (or however many threads you've set up).
Click to expand...

Cool thanks, I will give this a test by setting up an FTP server on my home server before testing on our server in work :)


I have created 10 test directories for my 10 test threads.

I have an image file being stored on my laptop called testimage.jpg which I am going to upload hopefully to each directory.

Where on my laptop would I have to store this testimage.jpg file for the ftp app to pick it up and send it across?
 
Last edited:
You'll need to create the directories first.

I'd also use unique filenames ($__time) as it saves having to allow modify permissions to client accounts (to overwrite the file if you are looping multiple times).

As everyone else has said I don't expect you'll have any throughput problems as long as you have the network bandwidth, but at least JMeter will give you a pretty chart to put in front of a manager.

edit - doh, missed part of your question again - you can specify the full path of the file in the local file box, for example 'C:\jakarta-jmeter-2.3.2\SAT\IS3.6\IS3.6_3\source\something.jpg'
 
Last edited:
Excellent :)

Thanks, I have it working

I think the only issue I have now is the file when it is recieved at the other end, is displaying with the file name as

Picture1274279399252 and is a "file" rather than a jpg file, im assuming this is due to the remote file field in JMeter and the way its been formatted? It looks like the size of the file is the same though so looks like it hasnt done the extension correctly.

Im going to get some bandwidth stats and also server load stats hopefully.

Just done some testing, it seems to be taking a long time with 50 connections repeating the task 10 times (so 10 images per folder). My connection at home is struggling, however the server load is only at 10%

The app has got stuck with 9 connections still active and the test still running however its not doing anything else.

Here are some stats

Server Load Average during connections = 10% on a single core 1GHz server - 512mb ram

stats1.jpg



stats2.jpg
 
Last edited:
Just put .jpg on the end of the remote file field, for example.
Code: 
Upload\Camera${__threadNum}\Picture${__time}.jpg
 
You must log in or register to reply here.
Back
Top Bottom