Cyprian of Carthage
"The Lord says to Peter: ‘I say to you,’ he says, ‘that you are Peter, and upon this rock I will build my Church… [Matt. 16:18–19]. On him [Peter] he builds the Church, and to him he gives the command to feed the sheep [John 21:17], and although he assigns a like power to all the apostles, yet he founded a single chair [cathedra], and he established by his own authority a source and an intrinsic reason for that unity. Indeed, the others were that also which Peter was [i.e., apostles], but a primacy is given to Peter, whereby it is made clear that there is but one Church and one chair. . . . If someone does not hold fast to this unity of Peter, can he imagine that he still holds the faith? If he [should] desert the chair of Peter upon whom the Church was built, can he still be confident that he is in the Church?" (The Unity of the Catholic Church 4; 1st edition [A.D. 251]).

"There is one God and one Christ, and one Church, and one chair founded on Peter by the word of the Lord. It is not possible to set up another altar or for there to be another priesthood besides that one altar and that one priesthood. Whoever has gathered elsewhere is scattering" (Letters 43[40]:5 [A.D. 253]).

"There [John 6:68–69] speaks Peter, upon whom the Church would be built, teaching in the name of the Church and showing that even if a stubborn and proud multitude withdraws because it does not wish to obey, yet the Church does not withdraw from Christ. The people joined to the priest and the flock clinging to their shepherd are the Church. You ought to know, then, that the bishop is in the Church and the Church in the bishop, and if someone is not with the bishop, he is not in the Church. They vainly flatter themselves who creep up, not having peace with the priests of God, believing that they are secretly [i.e., invisibly] in communion with certain individuals. For the Church, which is one and Catholic, is not split nor divided, but it is indeed united and joined by the cement of priests who adhere one to another" (ibid., 66[69]:8).
 
Augustine
"If the very order of Episcopal succession is to be considered, how much more surely, truly, and safely do we number them [the bishops of Rome] from Peter himself, to whom, as to one representing the whole Church, the Lord said, ‘Upon this rock I will build my Church, and the gates of hell shall not conquer it.’ Peter was succeeded by Linus, Linus by Clement. ... In this order of succession a Donatist bishop is not to be found" (Letters 53:1:2 [A.D. 412]).
     
Jerome
"Simon Peter, the son of John, from the village of Bethsaida in the province of Galilee, brother of Andrew the apostle, and himself chief of the apostles, after having been bishop of the church of Antioch and having preached to the Dispersion . . . pushed on to Rome in the second year of Claudius to overthrow Simon Magus, and held the sacerdotal chair there for twenty-five years until the last, that is the fourteenth, year of Nero. At his hands he received the crown of martyrdom being nailed to the cross with his head towards the ground and his feet raised on high, asserting that he was unworthy to be crucified in the same manner as his Lord" (Lives of Illustrious Men 1 [A.D. 396]).
"[Pope] Stephen . . . was the blessed Peter’s twenty-second successor in the See of Rome" (Against the Luciferians 23 [A.D. 383]).
"I follow no leader but Christ and join in communion with none but your blessedness [Pope Damasus I], that is, with the chair of Peter. I know that this is the rock on which the Church has been built. Whoever eats the Lamb outside this house is profane. Anyone who is not in the ark of Noah will perish when the flood prevails" (ibid., 15:2).

Tertullian
"[T]his is the way in which the apostolic churches transmit their lists: like the church of the Smyrneans, which records that Polycarp was placed there by John, like the church of the Romans, where Clement was ordained by Peter" (Demurrer Against the Heretics 32:2 [A.D. 200]).