I'm assuming that you can't connect to the server over the internet, even when you try a direct connect. Ignore the following if my assumption is wrong.
Can you connect to the game locally, i.e., can you start a client on the same machine and connect to the server?
If you can, then you know that the server is running. Therefore, it must be a problem with your network.
The easiest way to debug your network is to start from the server, and work your way to your modem. I'm guessing here, but you probably have a setup that looks somewhat like this:
ISP -- modem -- router -- server
So, if you have a computer to spare, set it up as a client and test the following configurations:
1) client -- server
2) client -- router -- server
3) client -- some friend's internet connection -- internet -- isp -- router -- server
Without further information, I'd guess that the most likely place to find an error would be in the firewall settings of your router.