Mikrotik experts in here...

[batfink]

Having read lots of good things about Mikrotik and wanting to upgrade my network to better separate out main network/kids/guests/IoT etc, I thought I'd start with stage 1:

- put my Asus router into AP only mode and replace the router section with a Hex S.

I was aware of what I was getting into before I started. As has been said before, the learning curve is near vertical.

I've e also taken this route because I have more than a passing interest in networking at work (this isn't my main field of work but I've picked up overall ownership for managing our network support team) so I thought it would be a good opportunity to learn more.

The Hex S is a very powerful piece of kit with enterprise levels of config for not a lot of money. Once you get your head around it, RouterOS is brilliant and Winbox, the tool for managing it, is fantastic.

Basic config is easy but I have discovered more than once that it's easy to break things.


My biggest issue so far.....

I'm struggling with routing DNS queries through my PiHole though which is on a static IP of 192.168.1.2 (I use 192.168.1.0/24 for my network). I put the PiHole IP into the DNS section under the DHCP server and untick "use peer DNS" on the interface. At this point, the connection falls over with a constant repeat of connecting....terminating - unable to obtain IP address....etc

The router gets stuck in this loop and will not recover from it. The only way I have found is to 'reset configuration' and start all over again. If I reset then restore a known working backup, the connection issue comes back so it will only work again if I manually rebuild the config/rules again. Slightly frustrating!

My guess is that the interface is getting stuck in some sort of unobtainable loop. The PiHole works fine if I manually configure the DNS on a device (eg mobile phone)

With this in mind, I think it must be a firewall config issue. I've followed the Mikrotik guidance to pretty well lock down my firewall so I'm guessing I might need to either pass through port 53 or add the PiHole IP to one of my firewall rules.

I've found various online guides which suggest this might be the case but I don't want to risk breaking it all again. This YouTube video also suggests I need to do some firewall config:

https://youtu.be/X-wkLYKYaj8

Any Mikrotik experts here that have managed to implement something similar?

 

[WJA96]

What do you have set under IP/DNS? 192.168.1.2?

Don’t put anything in the DHCP DNS entry. It will pull from what is entered into IP/DNS.

 

[batfink]

What do you have set under IP/DNS? 192.168.1.2?

Don’t put anything in the DHCP DNS entry. It will pull from what is entered into IP/DNS.

Nothing at the moment. It's currently pulling the dynamic DNS from my ISP (Plusnet). I did try putting the Pihole IP in there but got the same issue of losing connection.
I have a static IP with Plusnet if that makes a difference ?

 

[WJA96]

IP/DNS is the master location for the DNS server so if you put 192.168.1.2 in there that’s where the system will look for its DNS server.

Your static IP shouldn’t affect anything with regard to DNS. That’s just where the internet knows to find you.

I’m not sure why you felt the need to “lock down” the firewall because it comes locked down by default.

Can I suggest a straightforward hardware reset and use the default settings as much as possible. Change your Hex IP address to 192.168.0.x and the DHCP range to 192.168.0.x. Don’t put anything in the DNS settings for DHCP. In IP/DNS enter your PiHole IP address. Does that work?

[edit]And uncheck “use Peer DNS”[/edit]

 

[batfink]

I’m not sure why you felt the need to “lock down” the firewall because it comes locked down by default.

Can I suggest a straightforward hardware reset and use the default settings as much as possible. Change your Hex IP address to 192.168.0.x and the DHCP range to 192.168.0.x. Don’t put anything in the DNS settings for DHCP. In IP/DNS enter your PiHole IP address. Does that work?

[edit]And uncheck “use Peer DNS”[/edit]

I was following these recommendations to lock things down further than standard. The default firewall is pretty good, I agree.

https://wiki.mikrotik.com/wiki/Manual:Securing_Your_Router

I was preferring to keep my router in the 192.168.1.x range simply because I am still using the Asus for wireless and by default, it expects the network to be in this range. Phase 2 of my project is to get a Ubiquiti AP.

Do you know why the DNS change might be kicking my Plusnet connection out with no way of recovering from it? It gets stuck in some sort of loop. There was nothing in the logs that I could see.

I'll give your DNS suggestion a go when the family are out again later in the week. I don't want to go through a full reset again whilst working from home is still a necessity.

 

[batfink]

The above changes are the only things I've changed on the firewall apart from adding rules for an L2TP/IPSec VPN connection.

 

[WJA96]

I was following these recommendations to lock things down further than standard. The default firewall is pretty good, I agree.

https://wiki.mikrotik.com/wiki/Manual:Securing_Your_Router

I was preferring to keep my router in the 192.168.1.x range simply because I am still using the Asus for wireless and by default, it expects the network to be in this range. Phase 2 of my project is to get a Ubiquiti AP.

Do you know why the DNS change might be kicking my Plusnet connection out with no way of recovering from it? It gets stuck in some sort of loop. There was nothing in the logs that I could see.

I'll give your DNS suggestion a go when the family are out again later in the week. I don't want to go through a full reset again whilst working from home is still a necessity.

My mistake. For some reason I had it in my head you were using 192.168.0.x for your network and had the PiHole on the 182.168.1.x subnet. Leave everything on 192.168.1.x as it’s fine.

When you say the logs are empty I assume you’ve turned logging on?

https://jcutrer.com/howto/networking/mikrotik/persistent-logging-to-disk

If the logs are empty then what are the clients doing when they lose connection? What IP address are they getting and where from? What error do you get when you try to find a web page? Can you ping 8.8.8.8? Are you connecting to the hEX on a wireless or wired client?

If you have nothing in the IP/DNS entry then it uses the ISPs dynamic DNS. If you untick ‘Use Peer DNS’ it ignores whatever you have entered into DHCP/DNS and goes looking for the entry in IP/DNS. So either leave it as in and don’t untick ‘use Peer DNS’ or untick it and put the PiHole IP address in IP/DNS. So it shouldn’t be wigging-out when you uncheck that box.

The ‘securing your router’ was written when Mikrotik were feeling a bit bruised after a gigantic hole was found in the default settings. They closed it, and the default settings are now extremely secure, but they still start off ALL Mikrotik training courses by wiping the router and re-entering the rules from scratch. It’s somewhat pointless unless you’re charging by the hour.

 

[batfink]

My mistake. For some reason I had it in my head you were using 192.168.0.x for your network and had the PiHole on the 182.168.1.x subnet. Leave everything on 192.168.1.x as it’s fine.

When you say the logs are empty I assume you’ve turned logging on?

https://jcutrer.com/howto/networking/mikrotik/persistent-logging-to-disk

I have now. Thanks.

If the logs are empty then what are the clients doing when they lose connection? What IP address are they getting and where from? What error do you get when you try to find a web page? Can you ping 8.8.8.8? Are you connecting to the hEX on a wireless or wired client?

All the clients get a local IP address. The one that it doesn't get is my WAN IP address (even though the error message clearly states it can't get a 'local' IP address). Unable to connect to anything externally or ping anything. The pppoe (plusnet) connection states "disconnected"

If you have nothing in the IP/DNS entry then it uses the ISPs dynamic DNS. If you untick ‘Use Peer DNS’ it ignores whatever you have entered into DHCP/DNS and goes looking for the entry in IP/DNS. So either leave it as in and don’t untick ‘use Peer DNS’ or untick it and put the PiHole IP address in IP/DNS. So it shouldn’t be wigging-out when you uncheck that box.

I'll have to give it another go. The strangest thing was that if I did a reset, manually configured my plusnet connection to a 'connected' state again, then restored a known working backup of the RouterOS config, it immediately fell over again. The only thing I could do was manually recreate my setup (vpn etc)

The ‘securing your router’ was written when Mikrotik were feeling a bit bruised after a gigantic hole was found in the default settings. They closed it, and the default settings are now extremely secure, but they still start off ALL Mikrotik training courses by wiping the router and re-entering the rules from scratch. It’s somewhat pointless unless you’re charging by the hour.

Lots of interesting points there. I'm using this router to really step up my knowledge of networking which it is already doing.

I'll have a go at your DNS recommendations later in the week when I don't risk killing someone's youtube / Tiktok / work from home experience.

I'm going to keep this thread going as I learn more because there is limited information on this forum so far of Mikrotik. RouterOS really is brilliant but it's not for the faint hearted.

Next on my list is some sort of QoS, then when I've networked the house, I want to start putting certain devices (IoT for example) on separate VLANs.

 

[gpuerrilla]

I am using the SXT LTE6, took a while to setup a VLAN and run my cluster on another range (although I got it going without any guides eventually). Not even using most of its potential but its rekindling stuff I learned a long time ago - nice piece of kit and software!

 

[WJA96]

I am using the SXT LTE6, took a while to setup a VLAN and run my cluster on another range (although I got it going without any guides eventually). Not even using most of its potential but its rekindling stuff I learned a long time ago - nice piece of kit and software!

I'm very glad to hear it's behaving itself. What's the throughput like? The 4G Router thread seems to have gone a bit quiet.

 

[gpuerrilla]

I'm very glad to hear it's behaving itself. What's the throughput like? The 4G Router thread seems to have gone a bit quiet.

Yeah the thread is very quiet, the performance has been poor to feedback because the network I am on has been having bad periods. Currently the local mast has been 'repair scheduled for your area' for over a week now, and before that we had 'unscheduled maintenance' as they were upgrading the network all over the place.

I feel I can get some better stats once they stop messing about for a bit with the infrastructure.

 

[batfink]

I am using the SXT LTE6, took a while to setup a VLAN and run my cluster on another range (although I got it going without any guides eventually). Not even using most of its potential but its rekindling stuff I learned a long time ago - nice piece of kit and software!

That looks like a nice piece of kit.

Agree re. the software. Complex but good once you get your head around it. I also like the fact that it's the same RouterOS on all their kit so if you ever change at some point, there is nothing new to learn.

Still waiting for an hour or two when the family are out to make the change re. DNS, just in case it fails and I've got to set everything up again.

I started reading up about VLANs yesterday. There's a great thread on the Mikrotik forum. It's going to take a bit of work to get my head around it. I'm familiar with how they work and understand the concepts but I've never set anything like that up before. VLANs are much further down the 'to do' list though.

Next phase for me is to sort a Ubiquiti WiFi AP out to separate wireless traffic.

 

[gpuerrilla]

To get things working it seems on this piece of kit I have is use Bridges. It can be locked down or adjusted I guess better from Network professionals.

As my LTE is the WAN and interface to the LAN, its not like a standard Router where you can tweak ports specifically. I only plugged one Ethernet in which is PoE. I have since bought a smart switch to the arsenal which means I now have more options but I like it so far.

 

[batfink]

I have since bought a smart switch to the arsenal which means I now have more options but I like it so far.

Mikrotik switch to stay in the same eco-system or a.n.other brand?

 

[gpuerrilla]

It was a netgear as I got it for a steal, it was to replace an old d-link thing I have which must be 10yr+ old.

 

[batfink]

I'm struggling with routing DNS queries through my PiHole though which is on a static IP of 192.168.1.2 (I use 192.168.1.0/24 for my network). I put the PiHole IP into the DNS section under the DHCP server and untick "use peer DNS" on the interface. At this point, the connection falls over with a constant repeat of connecting....terminating - unable to obtain IP address....etc

The router gets stuck in this loop and will not recover from it. The only way I have found is to 'reset configuration' and start all over again. If I reset then restore a known working backup, the connection issue comes back so it will only work again if I manually rebuild the config/rules again. Slightly frustrating!

My guess is that the interface is getting stuck in some sort of unobtainable loop. The PiHole works fine if I manually configure the DNS on a device (eg mobile phone)

It looks like the honeymoon period is over. Got up this morning to no internet. At first I assumed it was the Openreach modem because the Mikrotik has been running for almost 2 weeks with no changes but quickly realised the Mikrotik had fallen over with the above connecting/terminating issue.

It had been running just fine. I haven't changed any settings and I didn't get to do the PiHole config because I haven't had the house to myself at any point for the last two weeks and didn't want to kill anyone's internet whilst they were there. No idea what happened because I didn't have time to look at the logs and had to quickly pull it out of the network before the family got up. The Asus was easily put back into Router mode and slotted straight back into the network. The Asus has never just failed like that in the 5 years it has been running.

I'll try and get the logs up but Windows won't read the USB stick it had been writing them to so I'll have to plug directly into the router I think to do this.

I bought the Mikrotik to learn something new and to give me a more secure and robust network that I can expand upon. I thought they were supposed to be ultra reliable with a steep learning curve. I think it will be going back and I'll look at something else (UDM maybe). I need to have a reliable, stable connection that 'just works'.

 

[gpuerrilla]

What happened does it not light up? If it is still working I would reset to defaults and update to start fresh.

 

[WJA96]

I bought the Mikrotik to learn something new and to give me a more secure and robust network that I can expand upon. I thought they were supposed to be ultra reliable with a steep learning curve. I think it will be going back and I'll look at something else (UDM maybe). I need to have a reliable, stable connection that 'just works’.

There is a very clear dividing line between production systems and labs. On one hand you want a lab system to mess about with and on the other hand you want something utterly stable that ‘just works’. Mikrotik RouterOS is used by some of the biggest companies in the world (Facebook, IBM, Reuters) and it is completely stable and runs fast on modest hardware. It’s utterly solid but very easy to mess up. You have to own the fact that you’re playing about in a production environment (Pihole and non-ISP DNS servers). There is nothing stopping you running the Mikrotik off one of the ports on the ASUS so you can have your own lab environment on 192.168.88.x while everyone else in the house is on 192.168.1.x. You’ll have exactly the same issues with any system if you start messing about with setting in a production environment. By all means spend money on more equipment but the truth is you’ll only learn from your own mistakes so set up a lab net off your ASUS and learn. The problem here isn’t the equipment.

 

[batfink]

Whilst I get your point @WJA96 , don't forget that at this stage I was running a basic setup - default firewall, l2tp vpn, a few static IPs and nothing else. I didn't even get around to sorting the PiHole issue so that wasn't even connected. The system simply fell over with no warning and for no obvious reason - it had been running exactly the same settings for 2 weeks. I hadn't changed anything and awoke to a non-functioning system.

I have read multiple reports of the stability of RouterOS and that was a major pull for me. Maybe it's the Hex S that has the problem? It was certainly odd and not something I could intentionally replicate but it does mean I lost confidence in it. What would I do if this happened when I was out at work and the Mrs was working from home, as she does most days?

 

[batfink]

There is a very clear dividing line between production systems and labs. On one hand you want a lab system to mess about with and on the other hand you want something utterly stable that ‘just works’. Mikrotik RouterOS is used by some of the biggest companies in the world (Facebook, IBM, Reuters) and it is completely stable and runs fast on modest hardware. It’s utterly solid but very easy to mess up. You have to own the fact that you’re playing about in a production environment (Pihole and non-ISP DNS servers). There is nothing stopping you running the Mikrotik off one of the ports on the ASUS so you can have your own lab environment on 192.168.88.x while everyone else in the house is on 192.168.1.x. You’ll have exactly the same issues with any system if you start messing about with setting in a production environment. By all means spend money on more equipment but the truth is you’ll only learn from your own mistakes so set up a lab net off your ASUS and learn. The problem here isn’t the equipment.

I've managed to get access to the logs. Unfortunately they don't tell me why it went down because there are too many of the same thing. Basically from 02:25 am through to the point I discovered it at 7:30am, this is what I've got:

Sep/18/2020 02:25:24 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:25:24 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:25:24 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:25:24 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:25:24 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:25:34 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:25:34 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:25:34 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:25:35 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:25:35 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: connected
Sep/18/2020 02:25:45 interface,info pppoe-out1 detect UNKNOWN
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: terminating...
Sep/18/2020 02:25:45 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:25:55 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:25:55 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:25:56 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:25:56 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:25:56 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:06 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:06 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:06 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:26:06 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:26:06 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:16 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:16 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:16 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:26:17 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:26:17 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:27 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:27 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:27 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:26:27 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:26:27 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:37 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:37 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:38 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:26:38 pppoe,ppp,info pppoe-out1: connected
Sep/18/2020 02:26:38 interface,info pppoe-out1 detect UNKNOWN
Sep/18/2020 02:26:38 pppoe,ppp,info pppoe-out1: terminating...
Sep/18/2020 02:26:38 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:48 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:48 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:49 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:26:49 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:26:49 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:26:59 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:26:59 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:26:59 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:00 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:27:00 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:27:10 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:27:10 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:27:10 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:10 pppoe,ppp,info pppoe-out1: terminating...
Sep/18/2020 02:27:10 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:27:20 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:27:20 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:27:21 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:21 pppoe,ppp,info pppoe-out1: terminating...
Sep/18/2020 02:27:21 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:27:31 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:27:31 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:27:31 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:31 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:27:31 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:27:41 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:27:41 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:27:41 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:42 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address
Sep/18/2020 02:27:42 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: connected
Sep/18/2020 02:27:52 interface,info pppoe-out1 detect UNKNOWN
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: terminating...
Sep/18/2020 02:27:52 pppoe,ppp,info pppoe-out1: disconnected
Sep/18/2020 02:28:02 pppoe,ppp,info pppoe-out1: initializing...
Sep/18/2020 02:28:02 pppoe,ppp,info pppoe-out1: connecting...
Sep/18/2020 02:28:03 pppoe,ppp,info pppoe-out1: authenticated
Sep/18/2020 02:28:03 pppoe,ppp,info pppoe-out1: terminating... - could not determine local IP address

With a 'delete oldest first' policy the logs have obviously overwritten the reason it went down in the first place.