It is a term traditionally used by people in special roles, such as priests in the Catholic Church, particularly if they are older or in a special position. 

Both the hermit and Ramandu are very old, Lewis is using it to show their special roles. Other terms they could have used are boy, girl, child, laddie, young man, etc. But I prefer Lewis's choices.

I think it could also be used as a form of respect. When the Hermit of the Southern March and Ramandu refer to the children as "my son" or "my daughter", it's likely that they knew the children would treat them with respect.

It's the case often used when a younger person shows a form of respect to an older person. Emeth refers to Rishda as "my father", even though Rishda wasn't really Emeth's father. It was a form of respect for a young officer had for their senior officers.

So it would have been very likely that the children would've treated the Hermit of the Southern March and Ramandu with respect.

Catholic brothers (monks, friars) use those terms of address. Lewis was either copying Catholicism, or copying those Medieval tales that featured monks and brothers (there were a lot.)