Routing for two NICS, one internal and one external network, no forwarding required.

RimBlock · 20 May 2011 at 09:16

Hi,

I have two network cards in my server and am looking to have one dedicated for internet traffic and one for internal network traffic.

Both NICs are connected to the same switch (HP Procurve 1810-24) and no forwarding is required between the two NICs (not using as a firewall or the like).

I have found the Network manager on Fedora 14 seems to have problems configuring two NICs (or maybe it is me) so the required settings for the NIC config files (eth0 and eth1) and any routing would be great.

Alternatively a decent guide would be great as well.

Many thanks
RB

FatRakoon · 29 May 2011 at 04:19

?
Why the hell are you even bothering to do what you are trying to do?

The fact that you have both NICs connected to the same swicth means .. WHAT?
The data throughput wont all of a sudden double! Just disconnect one and you iwll have exactly the same functions from the remaining one.. What you are considering is a total waste of time, and it really sounds silly, especially whe nyou are connecting them both to the same place anwyay.

Connect one to a Router and the other one to a switch enable ICS, and you are on the right track, but connecting them to the same switch is well... a bit thick and completely pointless.

RimBlock · 31 May 2011 at 02:41

Ok,

You are suggesting that I should connect one NIC to the router and another to the internal switch, which the router is already connected to and then enable ICS on my Fedora 14 Linux box even though, as specified in the text and title of this post, I do not require or even want forwarding between the NICs.

As the two NICs are not the same, one being an Intel 1000/Pro and the other not, disconnecting one will not give me exactly the same results as both NICs have different feature sets.

The HP Procurve 1810-24 is a web managed switch which will allow link aggregation so I can bond the two NICs to provide a pseudo 2Gb connection if I wanted to. Connecting both NICs to the same switch can give me the potential of double bandwidth. I am not doing this at present as I would prefer to have both NICs the same. Thanks for your reply though....

What I would like to do is have one NIC dedicated for internet traffic and one dedicated for internal traffic. I can set either or both up for internet traffic easily enough and bond them but am unsure as to how I would set-up a NIC for internal only traffic if it has access to the Internet router via the switch.

So the question remains.

How to config two NICs, one for only local subnet traffic and the other of only non local subnet traffic via a specific router.

Thanks
RB

FatRakoon · 31 May 2011 at 03:29

RimBlock said:
Ok,

You are suggesting that I should connect one NIC to the router and another to the internal switch, which the router is already connected to and then enable ICS on my Fedora 14 Linux box even though, as specified in the text and title of this post, I do not require or even want forwarding between the NICs.

Erm, not quite no!

Going by how I read your post, and I appologise if I am missing something, you want to have NIC1 talkign to the internet and NIC 2 talking to the switch and yet you have both NICs connected to the switch.

I simply said or meant, that you should have one NIC talking to the MODEM and the other talking to the Switch. Then the one thats talking to the switch, if you enable ICS on that, it will allow any7 other PCs that are connected to the switch, to also communicate with the outside world via ICS on NIC1

RimBlock said:
As the two NICs are not the same, one being an Intel 1000/Pro and the other not, disconnecting one will not give me exactly the same results as both NICs have different feature sets.

Ok, So, its a gigabit NIC, other than, I have to ask... So what?
In order to make full use of that, you also need any other devices plus your switch to also be gigabit, and I will assume they all are, but again, so what? All NICs can go at the lower speeds, so it makes no difference.

Im confused?

RimBlock said:
The HP Procurve 1810-24 is a web managed switch which will allow link aggregation so I can bond the two NICs to provide a pseudo 2Gb connection if I wanted to. Connecting both NICs to the same switch can give me the potential of double bandwidth. I am not doing this at present as I would prefer to have both NICs the same. Thanks for your reply though....

Oh, ok, so now you are getting clearer, but again, why would you want to double the bandwidth and then set each NIC to a different task?

Isnt that pretty much the same as setting up a RAID array and then wanting access to the individual drives instead of the array?

RimBlock said:
What I would like to do is have one NIC dedicated for internet traffic and one dedicated for internal traffic. I can set either or both up for internet traffic easily enough and bond them but am unsure as to how I would set-up a NIC for internal only traffic if it has access to the Internet router via the switch.

So the question remains.

How to config two NICs, one for only local subnet traffic and the other of only non local subnet traffic via a specific router.

Thanks
RB

I see. I think?
But the idea still really does confuse me.

I clearly cannot help you here, but I have to know why you are trying this?

As you said, you want to tie them up to make a 2GB connection, but then set one to be WEB and one to be LAN )

Can you not see how this sounds?

Dont get me wrong, but that to me, really does sound a bit nutty.

unless of course your internet is a 1GB internet package and then why tie them up in the first place only to then split them up again?

Confused???

RimBlock · 31 May 2011 at 04:33

FatRakoon said:
Erm, not quite no!

Going by how I read your post, and I appologise if I am missing something, you want to have NIC1 talkign to the internet and NIC 2 talking to the switch and yet you have both NICs connected to the switch.

That is assuming the router is directly in the same room as the machine but is connected to the switch which also provides connectivity for other machines. I have no wish to force all the other machines to go through this machine in order to access the internet.

FatRakoon said:
I simply said or meant, that you should have one NIC talking to the MODEM and the other talking to the Switch. Then the one thats talking to the switch, if you enable ICS on that, it will allow any7 other PCs that are connected to the switch, to also communicate with the outside world via ICS on NIC1

Yep, I understand what you are suggesting but have no wish to force all internet connectivity through a single machine especially as that machine is streaming HD content internally at the same time to a number of other machines / head units.

FatRakoon said:
Ok, So, its a gigabit NIC, other than, I have to ask... So what?
In order to make full use of that, you also need any other devices plus your switch to also be gigabit, and I will assume they all are, but again, so what? All NICs can go at the lower speeds, so it makes no difference.

Im confused?

The Intel NIC is a far more robust NIC than the other one. It handles the load on the NIC rather than via the processor and can handle link aggregation. The other NIC does and cannot. I would like the full 1Gbit bandwidth to be available to the internal subnet regardless of what information is being sent back and forth to the internet from the same machine or from others (give or take bandwidth usage due to internet traffic on the switch).

FatRakoon said:
Oh, ok, so now you are getting clearer, but again, why would you want to double the bandwidth and then set each NIC to a different task?

Isnt that pretty much the same as setting up a RAID array and then wanting access to the individual drives instead of the array?

Yes it would be which is why I would not aggregate and then use for separate tasks. The idea is to prevent any internet bandwidth affect, at a NIC level on the same machine, from interfering with transfers on the internal subnet coming from this machine.

FatRakoon said:
I see. I think?
But the idea still really does confuse me.

I clearly cannot help you here, but I have to know why you are trying this?

As you said, you want to tie them up to make a 2GB connection, but then set one to be WEB and one to be LAN )

Can you not see how this sounds?

Dont get me wrong, but that to me, really does sound a bit nutty.

unless of course your internet is a 1GB internet package and then why tie them up in the first place only to then split them up again?

Confused???

I can see why you are confused. Nope just looking to route all internet traffic to and from this machine via a dedicated NIC and all internal subnet traffic via another dedicated NIC so bandwidth usage on either NIC does not affect the other. The switch can handle the usage of internet and subnet traffic.

It is partly for segregation of tasks, data . The other option would be to bond the NICs and just have a 2Gb virtual NIC. For this to work I would have to get another Intel 1000/Pro as the second one I had 'got damaged'

...

So in essence. Machine provided HD content to a number of other machines / head units. Machine also provides some internet duties. I don't want the machine as a single point of failure for all internet connectivity. I don't want internet activity to have an effect on transfers to/from the local subnet. I do appreciate that the disks and their attainable IOPS will have a big affect but there is nothing I can do about that at the moment.

Thanks
RB

FatRakoon · 31 May 2011 at 05:31

I would not advise for anyone to use ICS if at all possible, and especially if they are already using a Switch/Router etc... ICS is only really any use if you have a USB Modem I feel.

My own Network is fairly simple.

I am going to re-do it, and I have everythign ready, I just need to get off my bum, but its currently like this :-

MODEM/ROUTER in Living room

Connected to it, is PC1 and a long Ethernet cable also comes from the MODEM/ROUTER and on to the SWITCH in the LAN ROOM.
My NAS is also connected to it too!

Connected to the switch is all my other PCS

My daughters PC however, is currently wireless although I do have a pair of HOMEPLUG adapters that I have used now and then for her PC, but they are slower than Wireless for some reason.

The setup I have is flawless and works 100% perfectly, however the length of the wires I have are too much, and I really want the MODEM/ROUTER, and the NAS all in the kitchen right next to the master socket and only one wire goes to the Living room PC and one wire goes up to the Switch and one wire goes to a HomePlug.

RimBlock · 31 May 2011 at 06:37

My current setup;

Switch / patch panel;

I am trying to maximise upload from the subnet to the NAS which seems to suffer from slowdowns when there is high network activity.

Uploads to individual WD Green drives average around 75MB/s and to the array also average the same.

Copying from one drive to another using a separate PC seems to average around 35MB/s.

First step in trying to speed up the uploads to the box is by taking the internet traffic out of the equation, hence wanting to route all the internet traffic via a dedicated NIC and the subnet traffic also by its own NIC.

Having just renovated our place, I had has big slowdowns with homeplugs which ran fine in our last place. The difference is that our wiring was renewed in our old place and was only added too in the new place. Sometimes new wires were joined to existing cables and I believe this is what caused some issues. 1GB home plugs could not manage HD content speeds without stuttering badly. Now I have wired up the place for cat6 and taken the home plugs out it all runs very nicely. Just need to optimise the NAS.

RB

deano · 31 May 2011 at 07:07

is it not just a case of removing the gateway from the nic you want to use for internal use, so it is unable to get to the internet?

RimBlock · 31 May 2011 at 07:20

deano said:
is it not just a case of removing the gateway from the nic you want to use for internal use, so it is unable to get to the internet?

Not so sure.

I would have thought it would be to do with setting the routing tables but I notice the eth0 config has default=yes set and am not sure if it will override the routing rules.

Having the correct routing rules would be very helpful as well.

The internal network is on the 192.168.1.xxx subnet.

RB

deano · 31 May 2011 at 07:58

whats in ifcfg-eth0, ifcfg-eth1 and what do you get from netstat -rn

RimBlock · 31 May 2011 at 08:00

deano said:
whats in ifcfg-eth0, ifcfg-eth1 and what do you get from netstat -rn

Will post up when I get home as I am in the office at the moment.

Thanks
RB

RimBlock · 31 May 2011 at 14:05

as requested.

[root@nas network-scripts]# more ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
DEFROUTE="no"
HWADDR=xx:xx:xx:xx:xx:xx (replaced with xx)
IPADDR=192.168.1.237
NETMASK=255.255.255.0
GATEWAY=192.168.1.254
DNS1=8.8.8.8
IPV6INIT=no
NAME="System eth0"
ONBOOT=yes
TYPE=Ethernet
UUID="5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03"

[root@nas network-scripts]# more ifcfg-eth1
DEVICE=eth1
DEFROUTE="yes"
ONBOOT=yes
HWADDR=xx:xx:xx:xx:xx:xx (replaced with xx)
TYPE=Ethernet
BOOTPROTO=none
IPADDR=192.168.1.238
NETMASK=255.255.255.0
GATEWAY=192.168.1.254
DNS1=8.8.8.8
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="System eth1"
UUID=9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04

[root@nas network-scripts]# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 eth1

[root@nas network-scripts]# ifconfig
eth0 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx (replaced with xx)
inet6 addr: fe80::227:eff:fe0a:adc7/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:8654726 errors:0 dropped:0 overruns:0 frame:0
TX packets:8538411 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8749511428 (8.1 GiB) TX bytes:8759517453 (8.1 GiB)
Interrupt:20 Memory:e3100000-e3120000

eth1 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx (replaced with xx)
inet addr:192.168.1.238 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe83:95d6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:682765742 errors:16 dropped:901 overruns:0 frame:8
TX packets:655405045 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:631395788944 (588.0 GiB) TX bytes:597340500046 (556.3 GiB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:27457 errors:0 dropped:0 overruns:0 frame:0
TX packets:27457 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:5842699 (5.5 MiB) TX bytes:5842699 (5.5 MiB)

RB

deano · 31 May 2011 at 19:24

sorry for long delay was in work

what happens if you # out the gateway line on eth0 disconnect eth1 and restart networking, do you still have internet access

GaryH-21 · 1 Jun 2011 at 00:09

Can you not set up a routing table for something like

default: eth0
192.168.1.0/24: eth1

So all local traffic goes through eth1 and anything else goes through eth0?

RimBlock · 1 Jun 2011 at 02:04

deano said:
sorry for long delay was in work

what happens if you # out the gateway line on eth0 disconnect eth1 and restart networking, do you still have internet access

Will give it a go tonight Deno. THanks.

GaryH-21 said:
Can you not set up a routing table for something like

default: eth0
192.168.1.0/24: eth1

So all local traffic goes through eth1 and anything else goes through eth0?

This was my first thought but if eth0 can get to 192.168.1.XXX and is set as default, will the routing process even go past the default interface ?. I really don't know and don't have so much time to play around. I can sure give it a try but how will I know which NIC is routing the packets to a specific address ?. Traceroute only seems to report on, presumably, routers and so the first hop is my own internet gateway which would not be used for any internal transfers.

Thanks
RB

GaryH-21 · 1 Jun 2011 at 03:57

Just to test, I guess you could maybe transfer a large file across the network, then do a "ifconfig /intname" and see the Tx packets increase to see what interface it's going through.

I've not read through the entire post so please excuse me if this has already been covered, but would it not be better to maybe vlan it, so the nics are both on different LAN's?

RimBlock · 1 Jun 2011 at 04:06

GaryH-21 said:
Just to test, I guess you could maybe transfer a large file across the network, then do a "ifconfig /intname" and see the Tx packets increase to see what interface it's going through.

I've not read through the entire post so please excuse me if this has already been covered, but would it not be better to maybe vlan it, so the nics are both on different LAN's?

Cheers Gary,

Will give the it a go with the Tx monitoring. I have not shutdown the internet processes but there is really no reason not to.

Yep the VLan is another option and the switch supports it. Just wondering if they is an easier option at a server level rather than infra.

End of the day though, whatever works

.

RB