1 This Psalm also has the title Allelujah prefixed to it: and this twice. But some say, that one Allelujah belongs to the end of the former Psalm, the other to the beginning of this. And they assert, that all the Psalms bearing this title have Allelujah at the end, but not all at the beginning; so that they will not allow any Psalm which has not Allelujah at the end, to have it at the beginning; supposing that what seems to belong to the commencement, really belongs to the end of the former Psalm.
But until they persuade us by some sure proofs that this is true, we will follow the general custom, which, whenever it finds Allelujah, attributes it to the same Psalm, at the head of which it is found. For there are very few copies (and I have found this in none of the Greek copies, which I have been able to inspect) which have Allelujah at the end of the CLth Psalm; after which there is no other which belongs to the same canon. But not even this could outweigh custom, although all the copies had it so.
For it might be that, with some reference to the praise of God, the whole book of Psalms, which is said to consist of five books (for they say that the books severally end where it is written Amen, Amen), might be closed with this last Allelujah, after all that has been sung; nor, on account of the end of the CLth Psalm, do I see that it is necessary that all the Psalms entitled Allelujah, should have Allelujah at the end. But when there is a double Allelujah at the head of a Psalm, why as our Lord sometimes once, sometimes twice over, says Amen, in the same way Allelujah may not sometimes be used once, sometimes twice, I know not: especially, since as in this CVth, both the Allelujahs are placed after the mark by which the number of the Psalm is described, whereas the one, if it belonged to the end of the former Psalm, ought to have been placed before the number; and the Allelujah which belonged to the Psalm of this number, should have been written after the number.
But perhaps even in this an ignorant habit has prevailed, and some reason may be assigned of which we are as yet uninformed, so that the judgment of truth ought rather to be our guide than the prejudice of custom. In the mean time, before we are fully instructed in this matter, whenever we find Allelujah written, whether once or twice, after the number of the Psalm, according to the most usual custom of the Church, we will ascribe it to that Psalm to which the same number is prefixed; confessing that we both believe the mysteries of all the titles in the Psalms, and of the order of the same Psalms, to be important, and that we have not yet been able, as we wish, to penetrate them.
Source: The Enarrations, or Expositions, on the Psalms (New Advent)