However, by the time of the 1841 census, the family were back in Largo where Alexander was now a grocer. A few years later Alexander took over the Steamboat Inn (now Crusoe Hotel). When his son Andrew left school, he initially set out to become a joiner and went to Drumeldrie as an apprentice. Later he worked in Cupar for Robert Douglas, engineer and millwright. When Douglas set up business in Kirkcaldy, Selkirk moved with him. In the 1861 census Andrew was living on Kirkcaldy High Street and is described as an engine fitter.
Not long after this, Andrew Selkirk emigrated to New Zealand with fellow Largo man Walter Guthrie. Accordingly to the Leven Advertiser of 31 March 1909 (reporting on Andrew's death), the pair ended up at the gold mines. They found employment making furniture for the prospectors and built a boat with which to ply up and down the river with their wares. They also tried their hands at house building. However, Selkirk's desire to return home brought him back to Scotland within a few years (unlike Guthrie, who lived out his days in New Zealand). In 1866 he married Largo farmer's daughter Isabella Kinnear in Edinburgh. They settled in Leith where Andrew resumed his occupation as an engine fitter. In 1871, the couple were living in south Leith with their young sons, Alexander and John. However, the following year Isabella died from smallpox aged 35 years.
Andrew relocated soon afterwards to Largo, where he married Jane Henderson in 1874. He changed occupation again - this time becoming a salmon fisherman (a job he had assisted his father with over many summers). The 1881 census finds Andrew, Jane, Alexander and John residing at Harbour Cottage, Lower Largo, along with a servant, a lodger and a visitor. A decade later Andrew and Jane are at Alexandra House (perhaps this was Harbour Cottage with a new name). They are still there in 1901 (when Andrew is described as a 'salmon taxman'). The Selkirk family leased the salmon fishing in Largo estate for over sixty years.
Andrew Selkirk was very proud of these striking new buildings. The Dundee Evening Telegraph of 24 March 1909 noted at the time of his death that "Mr Selkirk owned much property, which was planned and largely built by himself." Andrew was survived by his sons Dr Alexander Selkirk and John Kinnear Selkirk (a marine engineer). The Lower Largo properties were divided between them. In the 1930s, the Fortes bought Alexandra House from Dr Selkirk and created the Rio Cafe.