Is anyone else having problems joining and staying in lobbies?

I’ve been having this problem since RAAM came out . It’ll say the opponent has left and take me to the main menu, or it will say that I have disconnected from Xbox Live. I know it’s not my Internet connection because every other game works with no problems or lag. Does anyone know what this could be?

I run on 6mbps down and 1mbps up and do just fine in most games, but if more than 4 or 5 people show up in a lobby, the game auto-kicks me out, exactly in the ways you’ve described.