I notice a similar issue related to how far out my camera is zoomed, and how much of the environment is visible. For example, if I am scrolled in close with a top-down camera angle, my fps is great. However, if I turn to look at, say, the horizon, or zoom out, my FPS tanks.
One a side note, where did you get that beautiful skybox?