The assertion of the Novatians that they refuse communion only to the lapsed agrees neither with the teaching of holy Scripture nor with their own. And whereas they allege as a pretext their reverence for the divine power, they really are contemning it, inasmuch as it is a sign of low estimation not to use the whole of a power entrusted to one. But the Church rightly claims the power of binding and loosing, which heretics have not, inasmuch as she has received it from the Holy Spirit, against Whom they act presumptuously.
5. But they say that those should not be restored to communion who have fallen into denial of the faith. If they made the crime of sacrilege the only exception to receiving forgiveness, they would be acting harshly indeed, and, as it would seem, would be in opposition to the divine utterances only, while consistent with their own assertions. For when the Lord forgave all sins, He made an exception of none. But since, as it were after the fashion of the Stoics, they think that all sins are equal in gravity, and assert that he who has stolen a common fowl, as they say, no less than he who has smothered his father, should be for ever excluded from the divine mysteries, how can they select those guilty of one special offense, since even they themselves cannot deny that it is most unjust that the penalty of one should extend to many?
6. They affirm that they are showing great reverence for God, to Whom alone they reserve the power of forgiving sins. But in truth none do Him greater injury than they who choose to prune His commandments and reject the office entrusted to them. For inasmuch as the Lord Jesus Himself said in the Gospel: “Receive the Holy Spirit: whosesoever sins you forgive they are forgiven unto them, and whosesoever sins you retain, they are retained,” who is it that honours Him most, he who obeys His bidding or he who rejects it?
7. The Church holds fast its obedience on either side, by both retaining and remitting sin; heresy is on the one side cruel, and on the other disobedient; wishes to bind what it will not loosen, and will not loosen what it has bound, whereby it condemns itself by its own sentence. For the Lord willed that the power of binding and of loosing should be alike, and sanctioned each by a similar condition. So he who has not the power to loose has not the power to bind. For as, according to the Lord's word, he who has the power to bind has also the power to loose, their teaching destroys itself, inasmuch as they who deny that they have the power of loosing ought also to deny that of binding. For how can the one be allowed and the other disallowed? It is plain and evident that either each is allowed or each is disallowed in the case of those to whom each has been given. Each is allowed to the Church, neither to heresy, for this power has been entrusted to priests alone. Rightly, therefore, does the Church claim it, which has true priests; heresy, which has not the priests of God, cannot claim it. And by not claiming this power heresy pronounces its own sentence, that not possessing priests it cannot claim priestly power. And so in their shameless obstinacy a shamefaced acknowledgment meets our view.
8. Consider, too, the point that he who has received the Holy Ghost has also received the power of forgiving and of retaining sin. For thus it is written: “Receive the Holy Spirit: whosesoever sins you forgive, they are forgiven unto them, and whosesoever sins you retain, they are retained.” So, then, he who has not received power to forgive sins has not received the Holy Spirit. The office of the priest is a gift of the Holy Spirit, and His right it is specially to forgive and to retain sins. How, then, can they claim His gift who distrust His power and His right?
9. And what is to be said of their excessive arrogance? For although the Spirit of God is more inclined to mercy than to severity, their will is opposed to that which He wills, and they do that which He wills not; whereas it is the office of a judge to punish, but of mercy to forgive. It would be more endurable, Novatian, that you should forgive than that you should bind. In the one case you would assume the right as one who rarely offended; in the other you would forgive as one who had fellow-feeling with the misery of sin.
Source: On Repentance (New Advent)