Fedora 14->15->14 terrible Samba speeds.

Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
Ok, it tested out the "if it ain't broke...." saying and suffered this weekend.

I will try and keep it concise but could be a long post.

I have two machines, a workstation (Win7 Pro) and NAS (Fedora 14). Both machines were working fine but there were a few improvements I wished to make.

The workstation was using 2x60GB Vertex IIs striped on an Adaptec raid controller but they were getting slow. 1x WD Black 1TB for storage.

The NAS is in a 20 bay Norco case but I was only able to run one 8 drive LSI raid card on my current motherboard although I had two cards at home. I was also running out of space and had no more available bays. My stripped media array had 3x1TB WD Blacks.

The mission.....

1. Split the 2 Vertex II drives, one for each machine.
2. Put an i3 motherboard and processor in so I can use both LSI cards and use the i3s GPU for video.
3. Transfer the 1TB black from the workstation and and use it to recreate a new stripped array for my media.
4. Transfer a 1.5TB Seagate 7200.11 to the workstation to replace the WD Black.
5. Reinstall Win7 HomeP on the workstation (look at relocating the users and program files to the Seagate drive)
6. Install Fedora 15 to the NAS.

Getting the hardware up and running on the Nas I set about installing Fedora 15. The install process kept failing with a unspecified error (the dvd had been verified). Turns out one of my 2TB WD Green drives was causing issues when in a specific bay in the case. After loosing all the data I moved it to another bay and it worked fine (data was recoverable from my array which the WD Green was a partial backup for). After finally giving in I tried to reinstall Fedora 14 to get an error message that highlighted what was wrong. Fedora 15 also would not install to the Vertex drive as it was detected as part of an array (the Adaptec ray from the WS) and it would not let me clear or use it. I had to boot a live DVD and wipe the partition before the Fedora 15 installer would allow me to install to it.

Finally got Fedora 15 installed and some of the software I usually use is missing. Most notably is the service management gui. Gnome 3 is ok but it does take longer for me to access what I want. I installed Samba 3 and the config gui. I then configured the Samba shares, rebuilt my Win7 machine (big issues moving the program files directories) and tried the shares. Only getting 2MB/s max. Checked my Nas to media player connection with the same result indicating the issue is with the Nas.

I checked all the settings I could think of (Firewall open for Samba and Samba Client, NIC reporting 1Gb network connected, switch reporting 1Gbit link). I finally reverted back to Fedora 14 with difficulty, my original Fedora 14 install DVD now errored with package dependency clashes and there was no link (that I could find) to Fedora 14 on the Fedora website as everything pointed to Fedora 15.

With a little url editing I managed to download and burn a Fedora 14 live DVD and ran an install off of that. Once installed I tried Samba again with the same results. The Service management gui was also still missing. I tried moving the NIC to another PCI slot which saw the speed jump to 40MB/s for around 15 minutes before dropping back to the original 2MB/s max. I tried the M/B NIC but that didn't seem to be detected at all (forget the make but not realtek).

What I have noticed is that I have Samba log files from my old WS are connections on the SSD Fedora boot drive. The SSD came from the WS and when I rebuilt the WS I gave it a new name. My downloads drive used to be my old Nas Fedora 14 boot drive so I am wondering if Fedora 15 tried to upgrade even though I asked for a custom install and changed the boot and root drive to the SSD.

Ok, two questions;
1. How do I get my Samba speeds back to the 70-80MB/s they used to be (internal Nas drive to drive copies are upto 90MB/s so the drives seem fine).

2. Where is the services management gui as I cannot find anything that looks like it in YUM and I found it a lot easier than having to drop to the command line to set services to run on boot or to restart them.

Thanks
RB
 
Can you post your samba conf? It would be helpful.

Some defaults I've seen have 'SO_RCVBUF=8192 SO_SNDBUF=8192' in the socket options which were used to speed it up. However the kernel now auto-tunes this and so those options can be detrimental (removing them over doubled the transfer speed for me).

I don't know what the gui tool is but on commandline:

chkconfig samba on (it might be chkconfig smb on)

also chkconfig --list shows you all services and if they are on/off. If you are using the desktop version just look to see if they are turned on under runlevel 5.
 
Can you post your samba conf? It would be helpful.

At work but will do when home. All settings should be default apart from the actual shares.

I don't know what the gui tool is but on commandline:

chkconfig samba on (it might be chkconfig smb on)

also chkconfig --list shows you all services and if they are on/off. If you are using the desktop version just look to see if they are turned on under runlevel 5.

Yep I have been doing from the command line.

This is what I had been using. Seems it is part of system-config-services and may just be hidden in the menu.

fedora_14_services_3_thumb.png


TBH I am suspecting an issue with the Network setup as NetworkManager seems to make a right mess of things. I have uninstalled it via Yum but still find various directories it used around.

I have also uninstalled and then reinstalled Samba but the reinstall picked up some of the details from the uninstalled setup so it seems it doesn't uninstall totally.

The old Samba setup before playing around worked fine so not sure why a clean install would suddenly perform so badly.

Rb
 
I had an issue on SuSe where network access was an issue, spent ages trying to figure it out. disabled network manager and all is good :)

not so sure on the gui services thing is in not installable from yum?
 
I had an issue on SuSe where network access was an issue, spent ages trying to figure it out. disabled network manager and all is good :)

not so sure on the gui services thing is in not installable from yum?

I uninstalled network Manager and then had to setup networking to automatically start on boot again. It may have messed some other things up as well. Maybe a clean install from the Fedora 14 install ISO I downloaded last night may help after wiping the SSD first.

The gui is part of the config-system-services package it seems but may be hidden in the menus. I will have a look tonight.

Anyone know how easy it is to install Fedora 15 with Gnome 2.XX rather than v3 ?.

Thanks
RB
 
Right I have cleared the boot drive using a live DVD and reinstalled.

I have disabled network Manager and turned it off.

I have also setup VSFTP.

No change. Can only get 2MB/s copying with Samba or FTP.

File transfers between disks have also dropped to 2.4MB/s although the disks benchmark using the disk utility (read only test) at 60->90 MB/s for single discs and 40->320MB/s for the 4 disc array.

Hope I dont have a bad disc / cable / controller.

Prior to the reinstall the disk to disk transfers were fine.

Trying a reboot now.

RB
 
Ok, after the reboot I got back up to 40MB/s over samba and ftp

Drive to drive was 60MB/s for all single to single drive transfers and 40MB/s to the raid and 20MB/s from the raid.

I have done a mdadm --detail --scan >> /etc/mdadm.conf and rebooted and I am back up to 90MB/s write on the array and 122MB/s write to a single disk from the array.

Single disk to disk has improved in some cases too up to 80MB/s but as low as 60MB/s depending on the drive (singles are all WD Green 1.5TB or 2TB).

Network transfers remain at 40MB/s

The new motherboard is a Asus P8H67-V with a i3 2100 processor, disabled onboard NIC and a PCI Intel Pro 1000 NIC installed along with 2 LSI basic raid cards (same chipset as the Dell Perc 6i).

So now hope it is not a NIC and motherboard resource issue. The nic was working fin in my other Intel MB.

RB
 
Ok, have found something in the logs.....

Have gone through the /var/logs/messages and found the following...

Jun 15 20:45:15 nas kernel: [ 0.439609] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
Jun 15 20:45:15 nas kernel: [ 0.440900] pci 0000:01:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
Jun 15 20:45:15 nas kernel: [ 0.440904] pci 0000:00:01.0: PCI bridge to [bus 01-01]
Jun 15 20:45:15 nas kernel: [ 0.441076] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device. You can enable it with 'pcie_aspm=force'
Jun 15 20:45:15 nas kernel: [ 0.441079] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
Jun 15 20:45:15 nas kernel: [ 0.443111] pci 0000:00:1c.4: PCI bridge to [bus 03-03]
Jun 15 20:45:15 nas kernel: [ 0.445108] pci 0000:00:1c.5: PCI bridge to [bus 04-04]
Jun 15 20:45:15 nas kernel: [ 0.445278] pci 0000:00:1c.7: PCI bridge to [bus 05-06] (subtractive decode)
Jun 15 20:45:15 nas kernel: [ 0.445569] pci 0000:05:00.0: PCI bridge to [bus 06-06] (subtractive decode)
::
Jun 15 20:45:15 nas kernel: [ 0.467163] pci 0000:00:01.0: PCI bridge to [bus 01-01]
Jun 15 20:45:15 nas kernel: [ 0.467165] pci 0000:00:01.0: bridge window [io 0xe000-0xefff]
Jun 15 20:45:15 nas kernel: [ 0.467168] pci 0000:00:01.0: bridge window [mem 0xfb800000-0xfbafffff]
Jun 15 20:45:15 nas kernel: [ 0.467170] pci 0000:00:01.0: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467173] pci 0000:00:1c.0: PCI bridge to [bus 02-02]
Jun 15 20:45:15 nas kernel: [ 0.467175] pci 0000:00:1c.0: bridge window [io 0xd000-0xdfff]
Jun 15 20:45:15 nas kernel: [ 0.467179] pci 0000:00:1c.0: bridge window [mem 0xfb400000-0xfb6fffff]
Jun 15 20:45:15 nas kernel: [ 0.467182] pci 0000:00:1c.0: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467188] pci 0000:00:1c.4: PCI bridge to [bus 03-03]
Jun 15 20:45:15 nas kernel: [ 0.467190] pci 0000:00:1c.4: bridge window [io 0xc000-0xcfff]
Jun 15 20:45:15 nas kernel: [ 0.467194] pci 0000:00:1c.4: bridge window [mem 0xfbe00000-0xfbefffff]
Jun 15 20:45:15 nas kernel: [ 0.467198] pci 0000:00:1c.4: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467203] pci 0000:00:1c.5: PCI bridge to [bus 04-04]
Jun 15 20:45:15 nas kernel: [ 0.467204] pci 0000:00:1c.5: bridge window [io disabled]
Jun 15 20:45:15 nas kernel: [ 0.467208] pci 0000:00:1c.5: bridge window [mem 0xfbd00000-0xfbdfffff]
Jun 15 20:45:15 nas kernel: [ 0.467211] pci 0000:00:1c.5: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467217] pci 0000:05:00.0: PCI bridge to [bus 06-06]
Jun 15 20:45:15 nas kernel: [ 0.467220] pci 0000:05:00.0: bridge window [io 0xb000-0xbfff]
Jun 15 20:45:15 nas kernel: [ 0.467227] pci 0000:05:00.0: bridge window [mem 0xfbc00000-0xfbcfffff]
Jun 15 20:45:15 nas kernel: [ 0.467232] pci 0000:05:00.0: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467241] pci 0000:00:1c.7: PCI bridge to [bus 05-06]
Jun 15 20:45:15 nas kernel: [ 0.467244] pci 0000:00:1c.7: bridge window [io 0xb000-0xbfff]
Jun 15 20:45:15 nas kernel: [ 0.467248] pci 0000:00:1c.7: bridge window [mem 0xfbc00000-0xfbcfffff]
Jun 15 20:45:15 nas kernel: [ 0.467251] pci 0000:00:1c.7: bridge window [mem pref disabled]
Jun 15 20:45:15 nas kernel: [ 0.467268] pci 0000:00:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Jun 15 20:45:15 nas kernel: [ 0.467280] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
Jun 15 20:45:15 nas kernel: [ 0.467290] pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
Jun 15 20:45:15 nas kernel: [ 0.467300] pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
Jun 15 20:45:15 nas kernel: [ 0.467315] pci 0000:00:1c.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19
Jun 15 20:45:15 nas kernel: [ 0.467324] pci 0000:05:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
::
Jun 15 20:45:59 nas kernel: [ 76.862462] irq 16: nobody cared (try booting with the "irqpoll" option)
Jun 15 20:45:59 nas kernel: [ 76.862469] Pid: 0, comm: swapper Not tainted 2.6.35.13-92.fc14.x86_64 #1
Jun 15 20:45:59 nas kernel: [ 76.862472] Call Trace:
Jun 15 20:45:59 nas kernel: [ 76.862474] <IRQ> [<ffffffff810a74f3>] __report_bad_irq.clone.1+0x3d/0x8b
Jun 15 20:45:59 nas kernel: [ 76.862486] [<ffffffff810a765b>] note_interrupt+0x11a/0x17f
Jun 15 20:45:59 nas kernel: [ 76.862490] [<ffffffff810a813b>] handle_fasteoi_irq+0xa8/0xce
Jun 15 20:45:59 nas kernel: [ 76.862495] [<ffffffff8100c2ea>] handle_irq+0x88/0x90
Jun 15 20:45:59 nas kernel: [ 76.862499] [<ffffffff81470b44>] do_IRQ+0x5c/0xb4
Jun 15 20:45:59 nas kernel: [ 76.862504] [<ffffffff8146b093>] ret_from_intr+0x0/0x11
Jun 15 20:45:59 nas kernel: [ 76.862506] <EOI> [<ffffffff8102b7dd>] ? native_safe_halt+0xb/0xd
Jun 15 20:45:59 nas kernel: [ 76.862516] [<ffffffff81290b75>] acpi_safe_halt+0x2a/0x43
Jun 15 20:45:59 nas kernel: [ 76.862521] [<ffffffff81290bae>] acpi_idle_do_entry+0x20/0x30
Jun 15 20:45:59 nas kernel: [ 76.862525] [<ffffffff81290c27>] acpi_idle_enter_c1+0x69/0xb6
Jun 15 20:45:59 nas kernel: [ 76.862529] [<ffffffff8146e01e>] ? notifier_call_chain+0x14/0x63
Jun 15 20:45:59 nas kernel: [ 76.862533] [<ffffffff81395d56>] ? menu_select+0x177/0x28c
Jun 15 20:45:59 nas kernel: [ 76.862537] [<ffffffff81394d6d>] cpuidle_idle_call+0x8b/0xe9
Jun 15 20:45:59 nas kernel: [ 76.862543] [<ffffffff81008325>] cpu_idle+0xaa/0xcc
Jun 15 20:45:59 nas kernel: [ 76.862548] [<ffffffff81452876>] rest_init+0x8a/0x8c
Jun 15 20:45:59 nas kernel: [ 76.862554] [<ffffffff81ba1c49>] start_kernel+0x40b/0x416
Jun 15 20:45:59 nas kernel: [ 76.862559] [<ffffffff81ba12c6>] x86_64_start_reservations+0xb1/0xb5
Jun 15 20:45:59 nas kernel: [ 76.862563] [<ffffffff81ba13c2>] x86_64_start_kernel+0xf8/0x107
Jun 15 20:45:59 nas kernel: [ 76.862566] handlers:
Jun 15 20:45:59 nas kernel: [ 76.862568] [<ffffffff813148a8>] (ata_bmdma_interrupt+0x0/0x1a)
Jun 15 20:45:59 nas kernel: [ 76.862574] [<ffffffffa00d8622>] (mpt_interrupt+0x0/0x890 [mptbase]) <--Sata PCIe controller
Jun 15 20:45:59 nas kernel: [ 76.862585] [<ffffffffa00d8622>] (mpt_interrupt+0x0/0x890 [mptbase]) <--Sata PCIe controller
Jun 15 20:45:59 nas kernel: [ 76.862591] [<ffffffffa0132850>] (e1000_intr+0x0/0xe9 [e1000]) <--- Network Card
Jun 15 20:45:59 nas kernel: [ 76.862599] Disabling IRQ #16
Looks like an interupt clash to me but if so then how to fix ??

RB
 
Ok took the nic out and rebooted and got no IRQ issues in the logs.

Put the nic back in but in a different slot and...

J
un 15 22:10:57 nas kernel: [ 149.680451] irq 19: nobody cared (try booting with the "irqpoll" option)
Jun 15 22:10:57 nas kernel: [ 149.680455] Pid: 0, comm: swapper Not tainted 2.6.35.13-92.fc14.x86_64 #1
Jun 15 22:10:57 nas kernel: [ 149.680456] Call Trace:
Jun 15 22:10:57 nas kernel: [ 149.680458] <IRQ> [<ffffffff810a74f3>] __report_bad_irq.clone.1+0x3d/0x8b
Jun 15 22:10:57 nas kernel: [ 149.680465] [<ffffffff810a765b>] note_interrupt+0x11a/0x17f
Jun 15 22:10:57 nas kernel: [ 149.680468] [<ffffffff810a813b>] handle_fasteoi_irq+0xa8/0xce
Jun 15 22:10:57 nas kernel: [ 149.680471] [<ffffffff8100c2ea>] handle_irq+0x88/0x90
Jun 15 22:10:57 nas kernel: [ 149.680474] [<ffffffff81470b44>] do_IRQ+0x5c/0xb4
Jun 15 22:10:57 nas kernel: [ 149.680477] [<ffffffff8146b093>] ret_from_intr+0x0/0x11
Jun 15 22:10:57 nas kernel: [ 149.680478] <EOI> [<ffffffff81290030>] ? raw_local_irq_enable+0x10/0x12
Jun 15 22:10:57 nas kernel: [ 149.680484] [<ffffffff8106ba54>] ? sched_clock_idle_wakeup_event+0x17/0x1b
Jun 15 22:10:57 nas kernel: [ 149.680487] [<ffffffff81290e9c>] acpi_idle_enter_bm+0x228/0x260
Jun 15 22:10:57 nas kernel: [ 149.680490] [<ffffffff81394d6d>] cpuidle_idle_call+0x8b/0xe9
Jun 15 22:10:57 nas kernel: [ 149.680493] [<ffffffff81008325>] cpu_idle+0xaa/0xcc
Jun 15 22:10:57 nas kernel: [ 149.680497] [<ffffffff81452876>] rest_init+0x8a/0x8c
Jun 15 22:10:57 nas kernel: [ 149.680500] [<ffffffff81ba1c49>] start_kernel+0x40b/0x416
Jun 15 22:10:57 nas kernel: [ 149.680503] [<ffffffff81ba12c6>] x86_64_start_reservations+0xb1/0xb5
Jun 15 22:10:57 nas kernel: [ 149.680506] [<ffffffff81ba13c2>] x86_64_start_kernel+0xf8/0x107
Jun 15 22:10:57 nas kernel: [ 149.680507] handlers:
Jun 15 22:10:57 nas kernel: [ 149.680508] [<ffffffffa0111850>] (e1000_intr+0x0/0xe9 [e1000])
Jun 15 22:10:57 nas kernel: [ 149.680516] Disabling IRQ #19

Seriously, of all the bad luck. The previous MB couldn't handle both SATA HBAs and a PCI video card based on the 8600 chipset (guess which video card I bought) and now the replacement MB cannot handle the Intel NIC and the onboard NIC is not supported in the Linux kernel.....

Any suggestions ?

Thanks
RB.
 
turn off everything you dont need in the bios, eg USB, ESATA internel NIC etc etc. your possible going to flood the pci bus any how if it's just a desktop motherboard.
 
THanks Deno,

Tried that with the previous attempt.

USB3 off, Motherboard NIC off, audio chipset off.

Currently on;

Motherboard SATA controller (required).
Motherboard USB controller (required - keyboard and mouse).
Motherboard Video with i3 processor (required).
2xLSI controllers (PCIex4)
1xGbit NIC

The two PCIe controllers will not saturate the PCIe x16 bus as they can only use a combines 8x bandwidth.

There are also a lot of people using dual controllers on desktop motherboards reported on the unraid forums (not with this motherboard though).

Problem seems to be with Linux support for the Sandy Bridge chipset.

If I cannot find a solution then I will have to, begrudgingly, move to Windows 7 or WHS :(. Obviously not my preferred choice.

RB
 
As listed in post #10.

2.6.35.13-92.fc14.x86_64

Same problem occurred in Fedora 15 install downloaded a couple of weeks ago.

Not likely to reinstall though just to confirm the kernel version. A pretty recent one certainly.

RB
 
Back
Top Bottom