Objection 1. It would seem that all sorrow is contrary to all pleasure. Because, just as whiteness and blackness are contrary species of color, so pleasure and sorrow are contrary species of the soul's passions. But whiteness and blackness are universally contrary to one another. Therefore pleasure and sorrow are so too.
Objection 2. Further, remedies are made of things contrary (to the evil). But every pleasure is a remedy for all manner of sorrow, as the Philosopher declares (Ethic. vii, 14). Therefore every pleasure is contrary to every sorrow.
Objection 3. Further, contraries are hindrances to one another. But every sorrow hinders any kind of pleasure: as is evident from Ethic. x, 5. Therefore every sorrow is contrary to every pleasure.
On the contrary, The same thing is not the cause of contraries. But joy for one thing, and sorrow for the opposite thing, proceed from the same habit: thus from charity it happens that we "rejoice with them that rejoice," and "weep with them that weep" (Romans 12:15). Therefore not every sorrow is contrary to every pleasure.
I answer that, As stated in Metaph. x, 4 contrariety is a difference in respect of a form. Now a form may be generic or specific. Consequently things may be contraries in respect of a generic form, as virtue and vice; or in respect of a specific form, as justice and injustice.
Now we must observe that some things are specified by absolute forms, e.g. substances and qualities; whereas other things are specified in relation to something extrinsic, e.g. passions and movements, which derive their species from their terms or objects. Accordingly in those things that are specified by absolute forms, it happens that species contained under contrary genera are not contrary as to their specific nature: but it does not happen for them to have any affinity or fittingness to one another. For intemperance and justice, which are in the contrary genera of virtue and vice, are not contrary to one another in respect of their specific nature; and yet they have no affinity or fittingness to one another. On the other hand, in those things that are specified in relation to something extrinsic, it happens that species belonging to contrary genera, are not only not contrary to one another, but also that they have a certain mutual affinity or fittingness. The reason of this is that where there is one same relation to two contraries, there is contrariety; e.g. to approach to a white thing, and to approach to a black thing, are contraries; whereas contrary relations to contrary things, implies a certain likeness, e.g. to recede from something white, and to approach to something black. This is most evident in the case of contradiction, which is the principle of opposition: because opposition consists in affirming and denying the same thing, e.g. "white" and "non-white"; while there is fittingness and likeness in the affirmation of one contrary and the denial of the other, as, if I were to say "black" and "not white."
Now sorrow and pleasure, being passions, are specified by their objects. According to their respective genera, they are contrary to one another: since one is a kind of "pursuit," the other a kind of "avoidance," which "are to the appetite, what affirmation and denial are to the intellect" (Ethic. vi, 2). Consequently sorrow and pleasure in respect of the same object, are specifically contrary to one another: whereas sorrow and pleasure in respect of objects that are not contrary but disparate, are not specifically contrary to one another, but are also disparate; for instance, sorrow at the death of a friend, and pleasure in contemplation. If, however, those diverse objects be contrary to one another, then pleasure and sorrow are not only specifically contrary, but they also have a certain mutual fittingness and affinity: for instance to rejoice in good and to sorrow for evil.
Reply to Objection 1. Whiteness and blackness do not take their species from their relationship to something extrinsic, as pleasure and sorrow do: wherefore the comparison does not hold.
Reply to Objection 2. Genus is taken from matter, as is stated in Metaph. viii, 2; and in accidents the subject takes the place of matter. Now it has been said above that pleasure and sorrow are generically contrary to one another. Consequently in every sorrow the subject has a disposition contrary to the disposition of the subject of pleasure: because in every pleasure the appetite is viewed as accepting what it possesses, and in every sorrow, as avoiding it. And therefore on the part of the subject every pleasure is a remedy for any kind of sorrow, and every sorrow is a hindrance of all manner of pleasure: but chiefly when pleasure is opposed to sorrow specifically.
Wherefore the Reply to the Third Objection is evident. Or we may say that, although not every sorrow is specifically contrary to every pleasure, yet they are contrary to one another in regard to their effects: since one has the effect of strengthening the animal nature, while the other results in a kind of discomfort.
Source: Summa Theologica (New Advent)