Question about the Wall and the deserter in the first epilogue (Book 3 Spoilers)

I asked this in the Game of Thrones (Book 1) thread but what I wanted to discuss involves issues from Book 3.

In the very first scene we come across 3 rangers led by Royce that are scouting some wildlings north of the Wall, they find the wildlings and 2 of them are killed (including Royce) by a white walker. Next thing, we know is that the brother that made it out alive was caught south the Wall by some soldiers from the North and Ned Stark ends up executing the deserter for his actions.

My questions is how did he get from North of the Wall to South of the Wall?

Some people suggest he came back to the Wall and then fled. This doesnt make too much sense to me, since he would have been questioned about their scouting and the whereabouts of Royce (who is later mentioned as still missing). Also how would he have passed unnoticed and managed to flee Castle Black without being seen at all?

Other people suggest that he as a ranger knew about places where the Wall could be crossed (Similar to Osha and other wildling crossing). This makes the most sense since we know that Mance himself said that he had crossed the Wall and was at the feast when king Robert came to Winterfell. But still if these places exists then why were they not used by Mance himself when he tried to get people over the Wall to attack? They had them climb the Wall and then set ladders to help the other wildlings cross. If such places where the Wall could be crossed easily existed then why not use them to get the troops across?

I have also heard people say that he could have gone to one of the unused forts of the Night's Watch and crossed there, but again I think that those gates were sealed, if not the wildlings could use them?

This is something that bothered me so I thought I would ask here to see what you guys thought.

 

Yeah wasnt it called the Black Gate? Maybe he used that.