15 And there are also they who in excuse of their sins so accuse God, as to say that sins are pleasing to Him. For, if they were displeasing, say they, surely by His most Almighty power He would by no means suffer them to take place. As though indeed God suffered sins to be unpunished, even in the case of those whom by remission of sins He frees from eternal punishment! No one forsooth receives pardon of more grievous punishment due, unless he has suffered some punishment, be it what it may, although far less than what was due: and the fullness of mercy is so conveyed, as that the justice also of discipline is not abandoned.
For also sin, which seems unavenged, has its own attendant punishment, so that there is no one but by reason of what he has done either suffers pain from bitterness, or suffers not through blindness. As therefore you say, Why does He permit those things, if they are displeasing? So I say, Why does He punish them, if they are pleasing? And thus, as I confess that those things would not take place at all, unless they were permitted by the Almighty, so confess that what are punished by the Just One ought not to be done; in order that, by not doing what He punishes, we may deserve to learn of Him, why He permits to exist what He punishes.
For, as it is written, “solid food is for the perfect,” wherein they who have made good progress already understand, that it pertained rather unto the Almighty power of God, to allow the existence of evils coming from the free choice of the will. So great forsooth is His Almighty goodness, as that even of evil He can make good, either by pardoning, or by healing, or by fitting and turning unto the profit of the pious, or even by most justly taking vengeance. For all these are good, and most worthy a good and Almighty God: and yet they are not made save of evils.
What therefore better, what more Almighty, than He, Who, whereas He makes no evil, even of evils makes well? They who have done ill cry unto Him, “Forgive us our debts;” He hears, He pardons. Their own evils have hurt the sinners; He helps and heals their sicknesses. The enemies of His people rage; of their rage He makes martyrs. Lastly, also, He condemns those, whom He judges worthy of condemnation; although they suffer their own evils, yet He does what is good. For what is just cannot but be good, and assuredly as sin is unjust, so the punishment of sin is just.
Source: On Continence (New Advent)