ANSWER: It doesn't matter how strong the energy is that's coming toward you. What matters is how you react to it. From what you say you are taking it personally and resisting it, which produces the emotions you don't want to have.
The more "transparent" you can become to outside energy carrying unpleasant information, the more effective you can be at dealing with it. Here is one of the best techniques I ever learned for doing that:
You know what the person in question is like and what he does is not a surprise, even though you don't like it. So, give him permission to be the way he is and to do what he does. At the same time, give yourself permission to act on what he does in whatever way you think is appropriate.
He is going to be what he is and do what he does anyway, but by giving him permission to do it you are in charge and your resistance to it rapidly diminishes. With a little practice and some reminders, you'll be able to handle any situation like this with aplomb and ease.