You’re the “nag” to his friends
His friends see you as a nag. You know they do. When they invite him to do something, he says he can’t because you won’t let him. You really resent him putting that on you. You’re just trying to keep him from getting a potentially deadly virus, and he’s over there smearing your reputation with his friends.