If you'd start right now, you could actually argue that speaking German with your kid will help him learn Dutch, as he'll be able to recognize a lot of words (he will need quite a while to adjust to three languages introduced in a fairly short period of time, don't worry if that takes a lot of time). The most important thing, however, is that it won't suffice if only you speak German with him - he will need other people to speak German with him, both to make sure that hears enough German to learn it, and also to make him understand that he actually needs to learn/speak the language. So absolutely ask your mother and sister to speak German to him, and nothing but German, and stick to German yourself, but also try to find other people who would speak German with him, and other surroundings where he is expected to speak German. If you're moving to Amsterdam, as far as I know there is no German infrastructure other than a German playgroup meeting once a month - not a lot, but it might be enough to become friends with other kids he'd like to meet more often (there is a German school in The Hague).
Regarding your own command of German: It's hard to say whether or not your German is good enough, but if I were you, I'd give it a try, if you are able to make sure that he is also regularly exposed to German spoken by (near-)native speakers. I would also recommend to make an effort to improve your German - if your problem is mostly genders, the probably most efficient way would be to immerse yourself in the language as much as possible. Listening to a talk radio station (not music) like Deutsche Welle whenever you can, even while doing other things, will likely help with that (though don't expect your son to learn German that way, that's not how a kid will pick up a language). Also try to find a few German friends yourself, and speak German with your mother and sister, if you're not doing that already.
I'd recommend reading about bilingualism; I found the work of François Grosjean particularly helpful. You can find a good summary on his website: http://www.francoisgrosjean.ch/for_parents_en.html