You make it about budget
You keep having to deflect when that one couple asks you two to travel with them. You say it’s about budget constraints. You say it’s impossible for you two to get time off work at the same time. But then, you have to hide it when you two do go on a week-long trip with another couple.