But with me it is a very small thing that I should be judged of you, or of man's judgment: yea I judge not my own self. For I know nothing against myself, yet am I not hereby justified: but He that judges me is the Lord.
Together with all other ills, I know not how, there has come upon man's nature the disease of restless prying and of unseasonable curiosity, which Christ Himself chastised, saying, “Judge not, that you be not judged.” A kind of thing, which has no pleasure as all other sins have, but only punishment and vengeance. For though we are ourselves full of ten thousand evils, and bearing the “beams” in our own eyes, we become exact inquisitors of the offenses of our neighbor which are not at all bigger than “motes.” And so this matter at Corinth was falling out. Religious men and dear to God were ridiculed and cast out for their want of learning; while others, brimful of evils innumerable, were classed highly because of their fluent speech. Then like persons sitting in public to try causes, these were the sort of votes they kept rashly passing: “such an one is worthy: such an one is better than such another; this man is inferior to that; that, better than this.” And, leaving off to mourn for their own bad ways, they had become judges of others; and in this way again were kindling grievous warfare.
Mark then, how wisely Paul corrects them, doing away with this disease. For since he had said, “Moreover, it is required in stewards that a man be found faithful,” and it seemed as if he were giving them an opening to judge and pry into each man's life, and this was aggravating the party feeling; lest such should be the effect on them, he draws them away from that kind of petty disputation, saying, “With me it is a very small thing that I should be judged of you;” again in his own person carrying on the discourse.
Source: Homilies on First Corinthians (New Advent)