Party invite on xbox not working

When I was in early game the in-game party invite worked without skipping a beat but now I'm half way through the campaign both me and my partner are struggling to invite each other at all. Even sending invites through xbox friends list is hot and miss.

We found a work around by manually inviting through the party code but the way the bug seems to be intensifying makes me think theres a bigger problem.

Edit: we have both been restarting the game from the dashboard and power cycling the console by holding the button. It helped at first but has stopped working as well. Its almost like the game and the xbox UI arent interacting correctly