Using Outages.io - help confirming problem network

Ask how a feature works, how agents work, anything related to non specific agent platforms
sam33
Posts: 7
Joined: Tue Apr 14, 2020 5:39 pm

Using Outages.io - help confirming problem network

Post by sam33 »

I've started to use Outages.io over the last week to determine where our internet outages are coming from.

The majority of outages that are being captured state the affected area as : "On your local network" but the Hop down is showing a 10.20.x.x address.

Our internal network is a 192.168.x.x network, and our Public IP, shown in the "At a Glance" section as 203.220.x.x, makes me think that all of these errors on the 10.20.x.x network are coming from our ISP and not internally.

I've read a couple of other forum posts where you've explained there is an algorithm in place to determine what a "local network" is and that its not always accurate, I'm just looking to get some confirmation that my understanding is sound, and that I can continue to pursue this with our ISP?

We are running what's referred to as HFC in Australia. Which is coaxial cable running into the premises which connects to an ISP cable modem, which then plugs into the WAN port via ethernet to our internal Router.

This 10.20.x.x network is the second hop out of our network, so I'm guessing its going to be the ISP cable modem.

Any advice would be greatly appreciated.

Mike
Posts: 18
Joined: Wed Feb 12, 2020 12:54 pm

Re: Using Outages.io - help confirming problem network

Post by Mike »

Hi Sam.

First, can you share what the agent ID is so that I can go look at its reports.

We received an email from someone named Sam earlier, explaining that they were seeing a database error. I looked into this and sure enough, whom ever reported that let us know about a bug we didn't know about. We appreciate bug reports so that we can fix them.

Once you confirm the agent ID, I can check out the data and then respond.

Mike

sam33
Posts: 7
Joined: Tue Apr 14, 2020 5:39 pm

Re: Using Outages.io - help confirming problem network

Post by sam33 »

Hi Mike,

Thanks for the reply.

Yes that was me, but it was for my other network running 4g wireless (agent S 31507)

The Agent related to the issue in this post is S 31508.

Sam

Mike
Posts: 18
Joined: Wed Feb 12, 2020 12:54 pm

Re: Using Outages.io - help confirming problem network

Post by Mike »

Great, thanks for confirming that.

Right now, we're working on finding the DB bug that you've submitted. If you could hold off on contacting the ISP, I can help you with that as soon as we solve this other problem. Just wanted to acknowledge your post.

Mike
Posts: 18
Joined: Wed Feb 12, 2020 12:54 pm

Re: Using Outages.io - help confirming problem network

Post by Mike »

We found the DB error which is now fixed.
Your 31507 reports are back online but the agent continues to be offline.
We looked and it is not communicating with our network at all. Nothing for at least four days.
One odd thing is that both your 31507 and 31508 show no DNS servers.

Maybe check or re-install it as it seems the Windows agent service may not be running.
If you do re-install on the same PC, make sure there is no previous copy already installed and if there is, remove it before re-installing.

Not sure I'll have time to deal with the hops issue tonight but will post if I do otherwise will post in my tomorrow.

sam33
Posts: 7
Joined: Tue Apr 14, 2020 5:39 pm

Re: Using Outages.io - help confirming problem network

Post by sam33 »

Hi Mike,

I've added a new agent to replace 31507, it is now 31546. Are you able to remove 31507?

The agent I'm having all the disconnections with is 31508. I'm not sure why the DNS server's don't show up? The router is set to provide the DNS servers, I've recently updated the agent PC to use google's DNS and restarted the service, but it doesn't seem to have updated in the website.

No trouble with it being tomorrow.

Sam

Mike
Posts: 18
Joined: Wed Feb 12, 2020 12:54 pm

Re: Using Outages.io - help confirming problem network

Post by Mike »

Got it.
The new agent seemed to have the same private IP as the 507 and I wanted to make sure you had removed the old one if that was the same PC.
It's interesting that the DNS servers aren't showing up. There was a bug some time back but it's been fixed.

Now, back to your question.

Upon a first look and based on your input, the second hop seems to be the ISP on the WAN side of your router/modem. I think we can confirm this especially because you say you're using 192.168.x.x and have no 10.x.x.x network/IPs on the LAN side. Meaning, your router is not running multiple networks on the LAN so it's safe to say this is outside of your LAN.

We are constantly trying to tune our classification methods to try and determine what is in and outside the LAN but it's pretty hard to do when both sides are private IPs. We are considering adding a function that would allow members to manually intervene and configure what they know to be fact so that the classification can be better. The concern with that is non technical people might end up messing up their reports and not seeing the right info. It seems better to have members ask us to intervene, at least for now. Maybe with more input, we'll find the right balance.

So, as I see things in terms of hops, it's like this.

1 192.168.1.1 LAN side of your router/modem
2 10.20.x.x WAN side of your router/modem (your reports should be saying 'with your provider)
3 60.x.x.x TPG Telecom Limited

One question for you. Does your provider have two business names, like maybe one company bought the other one out?
We seem to find both TPG Telecom Limited and COMINDICO-AP SOUL Converged Communications Australia, AU.

We'll be talking about and looking at the classification code again today to see what else we can do. We keep tweaking things to get better results. These kinds of questions are how we can better the service and appreciate your taking the time to post.

sam33
Posts: 7
Joined: Tue Apr 14, 2020 5:39 pm

Re: Using Outages.io - help confirming problem network

Post by sam33 »

Hi Mike,

TPG has aquired a vast array of ISP's in Australia, it looks like 'Comindico-ap Soul' is somehow related to TPG. We're actually using Internode, another ISP that has been aquired by TPG.

I guess at the end of the day I'm just trying to rule out that my internal hardware is the problem. When it says "Hop Down" does that mean it has reached that point but can't get any further? Or its having trouble communicating at that point.

If its reaching the ISP cable modem, but then its having trouble commuicating to my router on the 192.168.1.1 network, I'm concerned that its an issue with my internal router. Do the test results indicate this isn't the case, or should i be grabbing another router and testing further?

Mike
Posts: 18
Joined: Wed Feb 12, 2020 12:54 pm

Re: Using Outages.io - help confirming problem network

Post by Mike »

Hi,

Well, without any physical access, here's what I think.

The majority of the issues are definitely on hop 2. Hop 2 seems to be the WAN side of your router.
The WAN side of the router seems to change IP's quite a bit which means it's getting a DHCP IP upstream.
I assume you don't have a fixed (static) IP at that location.

To me, it's either the local router or the interface that your WAN side is connecting to.

>When it says "Hop Down" does that mean it has reached that point but can't get any further?
>Or its having trouble communicating at that point.

The agent keeps track of hops on a very regular basis. When a hop goes away, it retains that information and puts that into the path as the downed hop. Meaning, if it was able to reach hop 5 for example but something happens to hop 5, it knows that it cannot reach hop 5 anymore but uses it's cache as the information for the missing hop.

So to answer your question, if the agent says hop down, it means the hop that 'used' to be there but is suddenly down. Looking at your reports, it looks to me like hop 2 which seems to be the WAN side, the ISP side, keeps going down or is configured in some way that it messes up your routing.

Personally, I would suspect the router since that's what it looks like. If you have a spare, while it's time consuming, it seems worth the effort to try another one. If that changes nothing, I'd have to say there is something that is not configured properly on the ISP side.

sam33
Posts: 7
Joined: Tue Apr 14, 2020 5:39 pm

Re: Using Outages.io - help confirming problem network

Post by sam33 »

Hi Mike,

Thanks for the explanation about the Hops, that makes more sense now.

Unfortunately I don't have a spare router handy but the problem is inconvenient enough to warrant purchasing another one.

I'll let you know if the new router changes things, its probably a week away at this point.

Sam

Post Reply