I would like to tell: Give importance to the language with more priority for you both. After a few years you speak with your child in the other language as well.

He can identify both easily and can get knowledge of both the languages. Also, your whole family can feel comfortable.