This is an internet problem - a connectivity issue not between your computer and internet, but with the server and your Internet Provider.
If you perform a TraceRoute to the server IP, it will display the number of hops or servers (one per line) and the miliseconds it took for each hop.
Most likely, when you cannot see PoTM it is because one of the servers along the way drops the packets. You might need to wait until the problem is happening again to run a traceroute, and then you'll give the information to your Internet Provider to look into.