2 internet connections - static routes? No BGP etc.

[Trifid]

Hello.


Below is a simplified version of our WAN connections, with a default route pointing at the private network, this private network then connects to the internet. We also have our own direct internet connection that is not currently used for outbound web traffic.

The private network uses the 10.0.0.0/8 and the 172.16.0.0/12 ranges but there are also some destinations on public addresses that need to go through the private network to be accessed, not all of these are known to us and we are unlikely to be able to identify them all. It is managed by a 3rd party company.

Every solution we have come up with is messy or unworkable in some way, so I have come here asking what others would do. Maybe a fresh set of eyes would be able to come up with a better solution?

Our ideas were:

1) Create a static route to the 10 and 172.16 networks and anything that is listed in the firewall pointing towards the private network that are on public addresses. Change the default route directly to the internet.

Problems: Addresses will be missed and we would have to identify these as they are reported. This is bad for business continuity but is probably the tidiest solution in the long run?

2) Create a request form for faster internet connectivity to individual sites that are work related. We can then direct requests to these sites straight out the internet.

Problems: If someone wanted to quickly look something up on the internet during their lunch break, they will be stuck using the slow connection. Users, such as purchasing, would have a legitimate business use to access a very wide range of sites are unlikely to put a request in for every site (I wonder if they would bother at all?). Messy, constant admin overhead. IMO it makes us look bad too.

2 can be done two ways, static routes to each IP address (which will result in a huge list of routes) or through the proxy server. If anything is not proxy compliant we would then still have to add static routes, and I don't like things in 2 locations. We could always insist that anything is not proxy compliant than it must go out through the private network but then it makes us look bad?



Can anyone else think of an alternative solution? Both of ours have rather large flaws and in a perfect world we like neither of them. Out of the 2 suggested above, what would you go for?

Thanks.

 

[derfderfley]

Hello.


Below is a simplified version of our WAN connections, with a default route pointing at the private network, this private network then connects to the internet. We also have our own direct internet connection that is not currently used for outbound web traffic.

<SNIP IMAGE>

The private network uses the 10.0.0.0/8 and the 172.16.0.0/12 ranges but there are also some destinations on public addresses that need to go through the private network to be accessed, not all of these are known to us and we are unlikely to be able to identify them all. It is managed by a 3rd party company.

Can anyone else think of an alternative solution? Both of ours have rather large flaws and in a perfect world we like neither of them. Out of the 2 suggested above, what would you go for?

Thanks.

Assuming the remote sites networks are accessed across a VPN, all you really need is for the 3rd party to identify the public (external) IP addresses or network ranges that need to be routed down the VPN (and which leg of the VPN).
I've certainly seen that particular set-up before with car/truck dealers, you just need to add the appropriate external addresses to the VPN encryption domain. The firewall will then identify those external addresses need to be encryped and send it over the VPN link, normal internet traffic can then just flow out of the external interface.
The above is obviously simplified quite a bit, but without seeing your router and firewall configs there is very little detail that any of us will be able to go in to, but I'm sure you get the drift.

 

[Trifid]

It's not a VPN, it's a network of networks similar to the internet but not publicly accessible.

 

[Laser402]

Assuming its mostly standard web traffic that you want to go straight out. Could a rule on the firewall to send any web traffic to a public address out to the internet directly.

Anything else, send through the private network?

This way you get the performance boost for all web traffic. Any services not web based but require private network would get routed the correctly?

Granted, I may have just made a lot of the above up. Its Late!

 

[Trifid]

Assuming its mostly standard web traffic that you want to go straight out. Could a rule on the firewall to send any web traffic to a public address out to the internet directly.

Anything else, send through the private network?

We should be able to identify all not HTTP traffic that is allowed out on the firewall and route that correctly. The problem is there a lot of sites that reside on public addresses that are inaccessible to the public.

 

[Laser402]

Ahh I see!

Well surely if your users are going to these private sites, A link was probably put there by someone. Unless all of your staff are remembering the IP address's of the sites and doing it that way!

Could the server team not give you all of the links that any of the users could see (ones on the start menu for instance) Then you guys work out where its going?

Sounds like a long an tedious task, but unless you know these addresses its never going to be nice.

 

[Trifid]

There is DNS , I think the staff are sent links in emails etc to find these sites and links from other sites.

 

[Laser402]

Id do option #1 then. Should easily and quickly be able to get all of the services back up and running after a few phone calls come in. Call it teething problems ;]


#2 would never end. Staff would either no request the change or assume the site is already on the 'fast connection' and live with it.

 

[Trifid]

I'm in support for option 1, only because I think option 2 is the worse idea long term. I don't like knowingly breaking stuff and there could be a huge fallout from changing the default route.

Are there any alternatives? Anything at all?

 

[dave-lew99]

I think your best solution is #1, however i don't think its brillant - it is however doable from your end without any cooperation.

Personally i'd be asking why these private sites are running on public IPs.

Actually, i've just thought, if you know all of the remote sites/companies involved then you can ask them for their address blocks and route their entire block via their private connection, making connections to their public IPs via the private link.

This way you don't need to discover sites on an ad-hoc basis and can go in armed with a working setup.

 

[Chief Wiggum]

Can the 3rd party network let you in to their routing process? Then you can pickup all the routes you need, then even if you pickup a default route alongside all of the others, setting a static route to the faster Internet connection will override it

 

[bigredshark]

eBGP and a private AS - seriously, dynamic routing is the only sensible way to do this. Not just the best way but the *only* sane way, it's not complex, it's not a security risk, it's a simple elegant solution. If you're providers can't do it, start questioning their competence.

 

[Skidilliplop]

That "large private network" isn't N3 by any chance is it?

Dynamic routing is best here, because it's probably less work in the long run and it'll adapt to changes. Which is quite key if there will be routes to networks outside of your administration. People are prone to changing things and not telling you, routing processes communicate better than people

 

[Trifid]

That "large private network" isn't N3 by any chance is it?

In regards to BGP (bearing in mind I've not used it previously and only know a bit of theory) I don't see it being a setup and forget about solution. There are still destinations on the internet that only accept incoming connections through the private networks public addresses which it won't know about? These will then need to be influenced through path weight on the router? We would want (as per the OP) all other internet bound traffic to go through the direct link. Will this then be a mess of influencing path selection? With the only benefit being redundancy for internet bound traffic?


It's a tough one, because we don't want to block access to destinations that are definitely work related to improve access to ones that may or may not be.

 

[DRZ]

There is a list of subnets that 99.9% of the N3 sites you are talking about will use. The remaining 0.1% of sites seem to have addresses which are on the Internet IP space.

I have set up exactly what you are wanting to do before

My email is in trust

 

[Skidilliplop]

I've set up a routing policy that picks out all the Addresses registered as part of N3 and specific NHS domains, most of which are /16 or /18 networks. So I just do source = our network, destination = N3 address range, set next hop as A.B.C.D.
It's manual and hence annoying, but it's only about 5 statements in a static routing policy file for CFH/CAB stuff. IT doesn't change often and pretty much all the public internet NHS addresses are directly registered with RIPE, which makes finding them out straight forward.

Though I fear is shouldn't tell you because if you're trying to do this you could well be in competition for the same business

 

[Trifid]

Many thanks for your email DRZ.

Though I fear is shouldn't tell you because if you're trying to do this you could well be in competition for the same business

Don't worry, we are doing this in house and not for another organisation. If you have any more information that you don't mind sharing privately my email is in my trust.

How did you manage to search all the .nhs.uk addresses?

 

[Skidilliplop]

Can't remember, I googled it Bit I did do a RIPE search for some of the public addresses and got the full ranges.

I wouldn't worry about the admin overhead for the infrastructure side, if you're not NHS yourself you'll need to apply for access filters based upon destination IP/network and service.
I guarantee you this will be 100x more ballache than adding a line to a routing policy file.

Thinking back for the private IPs I think I emailed CfH support desk. I had the email of one of the senior engineers somehow. The 1st line numpties told me I didn't need to know IP addresses just putting the URL in my browser would get to the pages I wanted... I doubt they even know N3 exists.

 

[DRZ]

Can't remember, I googled it Bit I did do a RIPE search for some of the public addresses and got the full ranges.

I wouldn't worry about the admin overhead for the infrastructure side, if you're not NHS yourself you'll need to apply for access filters based upon destination IP/network and service.
I guarantee you this will be 100x more ballache than adding a line to a routing policy file.

Thinking back for the private IPs I think I emailed CfH support desk. I had the email of one of the senior engineers somehow. The 1st line numpties told me I didn't need to know IP addresses just putting the URL in my browser would get to the pages I wanted... I doubt they even know N3 exists.

I had this experience too and eventually got through to someone that understood my question

How are your IG teams reacting to the LCA diagrams you are showing them? This is on the fringes of the "no common point of connection between N3 and the Internet" rule, you need to be pretty explicit about how your firewalling stops that from happening.


EDIT: If you aren't NHS but do have a really strong IGT submission and a great working relationship with the N3 people you can find some flex in what they will and will not do for you in terms of having to send them tonnes of paperwork each time you want to access a new SIN or have a new SIN access you. Obviously this entirely depends on what line of business you are in

 

[Skidilliplop]

I had this experience too and eventually got through to someone that understood my question

How are your IG teams reacting to the LCA diagrams you are showing them? This is on the fringes of the "no common point of connection between N3 and the Internet" rule, you need to be pretty explicit about how your firewalling stops that from happening.


EDIT: If you aren't NHS but do have a really strong IGT submission and a great working relationship with the N3 people you can find some flex in what they will and will not do for you in terms of having to send them tonnes of paperwork each time you want to access a new SIN or have a new SIN access you. Obviously this entirely depends on what line of business you are in

I didn't deal direct with CfH much so most of those acronyms mean nothing to me. Most of those were pinged between CfH, CIO and our consultant that provides the broker software. Then specs filtered through that and came to me, I built the network accordingly.

All we currently do is connect to CAB. We have a /26 block from the N3 allocation, which we have a PA500 doing NAT with between N3 and our network. All database comms goes through a Broker server which is on a 1:1 NAT with an N3 address. Thus nothing on N3 is aware of anything on our network, and devices on our network are only aware of stuff relevant to us on N3. The PA500 sits between permitting only services that have been authorised into our N3 range, and beyond that the Router BT supplied has further ACLs on it to control what our N3 range can get at.
The way the routing policy is configured on our core network that PA500 only ever receives traffic destined for N3 addresses defined in the routing policy.

That seemed enough to keep them happy. MOST of the really painful stuff got circumvented because we used a Broker solution from a 3rd party that'd already been approved essentiallly