I'm English and my wife is Portuguese and we are living in Portugal. I'm bilingual in Portuguese and my wife and I have always spoken Portuguese together. When my son was born, I found it somewhat strange and unnatural to talk to him in English, even though I knew I should have. As a result I only spoke English with him occasionally. Shortly before his 3rd birthday we had a holiday in England, with English friends, where I found it more natural to talk to my son in English. I decided that was my cue to continue, and since then I've managed to talk to him almost exclusively in English, and it now feels completely natural.
My son seems to understand pretty much everything I say in English, but he only ever responds in Portuguese. When he asks me a question in Portuguese, I always reply in English, and he seems to understand, and not find it strange.
After reading everywhere that bilingualism has to start pretty much at birth, I'm worried that I may have left it too late for him to become a native English speaker, especially as I'm the only exposure he has to native spoken English (other than films and TV and a few iPad apps for learning English).
Did I leave it too late? As someone who was raised bilingual myself (English and German), and has a passion for languages, I hope that my early stupidity won't cost my son the chance to be a native English speaker.