Hi there Mike,
It sounds as though you may have one of the following issues; you’re using wireless and this is the cause of the fluctuation in performance, the CVC backhaul you’re connected to at your exchange is congested (which is becoming more and more prevalent here in Australia as we all connect to the NBN) or there is another issue (we can address any/most/all of these).
Have you experienced this same issue when running over an Ethernet cable (if you’re using wireless normally)? If you haven’t, try this and give it a shot. I find this to usually be the most common issue.
If you’re still experiencing issues, we can look to see if your CVC is congested. If it is congested, you need to notify your ISP with some specific information (PM me if this is the case and I’ll help you out). What ISP are you with? What router are you using? Let me know and we can take this offline for the sake of preserving some anonymity and getting this fixed for you.