23 See, they say, the Prophet says, “Depart ye, go out from thence, and touch no unclean thing;” how then for peace sake should we bear with the wicked, from whom we are commanded to “go out and depart that we touch not the unclean thing”? We understand that “departure” spiritually, they corporally. For I also cry out with the Prophet (for however mean a vessel I am, God makes use of me to minister to you); I also cry out and say to you, “Depart ye, go out from thence, and touch not the unclean thing;” but with the touch of the heart, not of the body.
For what is it to “touch the unclean thing,” but to consent to sin. And what is it to “go out from thence,” but to do what appertains to the rebuking of the wicked, as far as can be done, according to each one's grade and condition, with the maintenance of peace? You are displeased at a man's sin, you have not “touched the unclean thing.” You have reproved, rebuked, admonished him, hast administered, if the case required it, a suitable discipline, and such as does not violate unity; then you have “gone out from thence.”
Now consider the actions of the Saints, lest perhaps this should seem to be an interpretation of my own. As Saints have understood these words, so surely ought they to be understood. “Go ye out from them,” says the Prophet. I will first maintain this meaning of the words from their customary use, and will afterwards show that that meaning is not my own. It often happens that men are accused; and when they are accused they defend themselves, and when the accused defends himself with good reason and justice, the hearers say, “He has got out of this.”
Got out; whither has he gone? He abides still in the place where he was, yet has he “got out of this.” How has he got out of it? By the good account he has rendered, and by his most satisfactory defence. This is what the holy Apostles did when they “shook off the dust from their feet” against those who did not receive the message of peace which was sent to them. That watchman, “got out from thence,” to whom it was I said, “I have made you a watchman unto the house of Israel.” For it was told him “If you warn the wicked, and he turn not from his wickedness, nor from his way, that wicked one shall die in his iniquity, and you shall deliver your soul.” This if he do, he “goes out from him,” not by a bodily separation, but by the defence of his own work.
For he did what it was his duty to do; though the other, whose duty it was to obey, obeyed not. This then is that, “Go ye out from thence.”
Source: Sermons on Selected Lessons of the New Testament (New Advent)