Ok while you are in this area (with your back to the arch) the 'Mamertine Prison' (Carcer Tullianum) is 45° to the right about 40 m away but it is above you on the modern ground level and out of sight. If you wish to visit it take the stairs on the right up to the first exit on the right. A building (photo here ) is in front of you (it's a church), walk around the right side of it to the front and there is the Prison entrance (donation expected).
Also to the right of this building about 40 m away is a street like the Via Sacra called the 'Clivus Argentarius' (The Bankers' Rise). It originally entered the Forum near the Comitium. This is almost certainly the street that Julius Caesar took after leaving his home in the Forum and over to the Curia Pompey where he was assassinated. About 3 hours later three slaves carried his bloodied body home in a litter, in all likelihood following this same route. This is also the street the prisoners and slaves took out of the Forum at the end of a Triumphal Parade.
It is very doubtful that St. Peter and Paul were ever held in this prison. But it is an fairly important ancient site. The lower level dates back to at least the third Century BC, some say earlier. It was a 'death row' for *very important* Roman citizens and foreign leaders. If a Leader/General was brought back as a POW in one of Triumphal Parades that is where he ended up (like Jugurtha and Vercingetorix - In HBO's Rome that is who Caesar brought back in his Triumph and had sadly killed). They would either be strangled there or brought down to the Comitium and publicly executed also some might be thrown off the Tarpeian Rock (cliff on the Capitoline Hill).
If they were killed in the prison their body was brought up by rope through a hole and thrown down the stairs that lead to the Forum for public viewing. Sometimes the bodies were left for days, later they would be dragged by meat hooks to the Tiber River and thrown in. The modern stairs basically follow the same path as an ancient staircase that lead from the Forum to the middle of the Capitoline Hill (the Arx). A lower flight of stairs (Forum to prison) was called the 'Scalae Gemoniae' (Gemonian Stairs) or as Pliny called them "The stairs of sighs" (or wailing).
A couple of stories about these stairs, the first is for the dog lovers and very sad. Pliny wrote: "During the reign of Tiberius the dead bodies of Titius Sabinus and his servants were thrown down these stairs after their execution. The corpse of one of these men was watched over day and night by his faithful dog. At one point the dog even went off and stole some bread. When the dog returned it tried to force the bread into his dead master's mouth. Later when their bodies were dragged to the Tiber River and thrown in, the dog jumped in and tried to drag his master's body to shore."
The man who condemned these men was later flung down these same stairs on Tiberius' orders. His name was Sejanus and he was a bad dude. He killed his own wife and was the lover of Tiberius' only son's wife. And together Sejanus and Tiberius' daughter-in-law poisoned Tiberius' son. All part of Sejanus' plan to become Emperor with this evil black widow by his side.
And there was even an Emperor killed on these steps. After Nero's death Rome was in a civil war and had four emperors in about 18 months. Called 'The year of the Four Emperors' (68-69 AD) which ended with Vespasian and beginning the Flavian Era. The third of these four short-term Emperors was named Vitellius. It his youth he was Emperor Tiberius' favorite boy, his nickname was 'Spintria' (sphincter artist :-) . He had the reputation for being a yes-man and a great hedonist. He was a familiar face around the Palace during the reigns of Caligula, Claudius and Nero. Let's see Tiberius was possibly smothered by Caligula, Caligula was killed by his guards, Claudius was poisoned by Nero and Nero was forced to commit suicide, you might think he wouldn't want the damn job :-) . He's declared Emperor in January, finally gets to Rome in mid-July and by mid-December the city is surrounded by the opposing army. Everyone deserts him, his palace is empty. He stuffs some gold coins in a money belt, grabs his dog (yea he really does :-) and flees. But not far, he hides in a gatekeepers house. But he is discovered and captured, a noose is put around his neck and he is lead through the Forum half naked in torn garments. The guards are beating him and the citizens are throwing dung and mud at him and calling him vile names. He is forced to watch as his statues are pulled down. He is finally brought to these Stairs and slowly tortured to death to the delight of the cheering crowd standing in the Forum. He is finally beheaded and his body is dragged by a meathook to the Tiber River. This same crowd had also cheered him when he became Emperor only a few months before:=(. See Emperors Galba and Vitellius' Murder In The Roman Forum for the whole story.
Please see also Mamertine Prison
Next: #11.2: Altar of Saturn