Objection 1. It would seem that idolatry is not the gravest of sins. The worst is opposed to the best (Ethic. viii, 10). But interior worship, which consists of faith, hope and charity, is better than external worship. Therefore unbelief, despair and hatred of God, which are opposed to internal worship, are graver sins than idolatry, which is opposed to external worship.
Objection 2. Further, the more a sin is against God the more grievous it is. Now, seemingly, a man acts more directly against God by blaspheming, or denying the faith, than by giving God's worship to another, which pertains to idolatry. Therefore blasphemy and denial of the faith are more grievous sins than idolatry.
Objection 3. Further, it seems that lesser evils are punished with greater evils. But the sin of idolatry was punished with the sin against nature, as stated in Romans 1:26. Therefore the sin against nature is a graver sin than idolatry.
Objection 4. Further, Augustine says (Contra Faust. xx, 5): "Neither do we say that you," viz. the Manichees, "are pagans, or a sect of pagans, but that you bear a certain likeness to them since you worship many gods: and yet you are much worse than they are, for they worship things that exist, but should not be worshiped as gods, whereas you worship things that exist not at all." Therefore the vice of heretical depravity is more grievous than idolatry.
Objection 5. Further, a gloss of Jerome on Galatians 4:9, "How turn you again to the weak and needy elements?" says: "The observance of the Law, to which they were then addicted, was a sin almost equal to the worship of idols, to which they had been given before their conversion." Therefore idolatry is not the most grievous sin.
On the contrary, A gloss on the saying of Leviticus 15:25, about the uncleanness of a woman suffering from an issue of blood, says: "Every sin is an uncleanness of the soul, but especially idolatry."
I answer that, The gravity of a sin may be considered in two ways. First, on the part of the sin itself, and thus idolatry is the most grievous sin. For just as the most heinous crime in an earthly commonwealth would seem to be for a man to give royal honor to another than the true king, since, so far as he is concerned, he disturbs the whole order of the commonwealth, so, in sins that are committed against God, which indeed are the greater sins, the greatest of all seems to be for a man to give God's honor to a creature, since, so far as he is concerned, he sets up another God in the world, and lessens the divine sovereignty. Secondly, the gravity of a sin may be considered on the part of the sinner. Thus the sin of one that sins knowingly is said to be graver than the sin of one that sins through ignorance: and in this way nothing hinders heretics, if they knowingly corrupt the faith which they have received, from sinning more grievously than idolaters who sin through ignorance. Furthermore other sins may be more grievous on account of greater contempt on the part of the sinner.
Reply to Objection 1. Idolatry presupposes internal unbelief, and to this it adds undue worship. But in a case of external idolatry without internal unbelief, there is an additional sin of falsehood, as stated above (Article 2).
Reply to Objection 2. Idolatry includes a grievous blasphemy, inasmuch as it deprives God of the singleness of His dominion and denies the faith by deeds.
Reply to Objection 3. Since it is essential to punishment that it be against the will, a sin whereby another sin is punished needs to be more manifest, in order that it may make the man more hateful to himself and to others; but it need not be a more grievous sin: and in this way the sin against nature is less grievous than the sin of idolatry. But since it is more manifest, it is assigned as a fitting punishment of the sin of idolatry, in order that, as by idolatry man abuses the order of the divine honor, so by the sin against nature he may suffer confusion from the abuse of his own nature.
Reply to Objection 4. Even as to the genus of the sin, the Manichean heresy is more grievous than the sin of other idolaters, because it is more derogatory to the divine honor, since they set up two gods in opposition to one another, and hold many vain and fabulous fancies about God. It is different with other heretics, who confess their belief in one God and worship Him alone.
Reply to Objection 5. The observance of the Law during the time of grace is not quite equal to idolatry as to the genus of the sin, but almost equal, because both are species of pestiferous superstition.
Source: Summa Theologica (New Advent)