Why you ignore it
You say maybe he just doesn’t want to fight because he feels it drives you two apart. But, a guy who plans on being with you for the long haul cares a lot about what you think about him, and how he makes you feel! He would never let you criticize him without defending himself, and he would never watch you cry without wanting to fix it. Meanwhile, a guy who doesn’t plan on sticking around won’t exert any energy into fighting.