My wife and I didn't know we're pregnant and in a weekend fun with friends we smoked a bit of weed - we don't usually smoke weed or cigarettes.
This is not something we're regulary doing as I said, but how bad to the fetus can it be ?
During the first two weeks of pregnancy there is very exchange of substances between the mother and the embrio. And the pregnancy, by definition, actually starts at the moment the woman supposed to have her period. So one may say that the first three weeks are actually quite safe in regards to drinking or other substances use or abuse. After that, however, starts the most dangerous period for drinking, smoking, etc. with differentiation and organogenesis possibly being severely negatively influenced by harmful substances. Therefore - drinking or smoking shortly after conceiving a child is less harmful to the baby than doing it after two or three weeks. So you and your baby should be fine. Don't worry.
However, this does not mean that one should use these two weeks to drink and take drugs because one won't be able to do so for the next 9 months. The risk is always there!
I have checked the pregnancy literature I have at home. There are hints that what I wrote is true, but it is not stated explicitely. Perhaps because there is no conclusive research... Or because authors feared it would make some women use drugs and drink alcohol in that time, which is still somehwhat bad idea.
Still, I have found some references:
: (about alcohol) In the first two weeks following fertilization, excessive alcohol consumption does not generally have a negative effect on the zygote and emerging blastocyst (pre-embryo)
: (critical growth period) In most successful pregnancies, the embryo implants 8 to 10 days after ovulation. [...] (after that) Rapid growth occurs and the embryo's main external features begin to take form. [...] During this critical period (most of the first trimester), the developing embryo is also susceptible to toxic exposures
As a general rule it is probably best to avoid cannabis smoking in pregnancy, but the primary evidence for this is based on the smoking aspect - many of the same compounds that make smoking tobacco in pregnancy potentially unhealthy are also present in weed. We have very little evidence either way as to whether THC itself has any effect on pregnancy - it's a drug that can remain in the body for some weeks post ingestion, but none of the documented biological effects are such as would be likely to have an adverse impact on pregnancy. The raw birth defects data doesn't show any correlation.
In regard to your situation specifically, one-time use in very early pregnancy is extremely unlikely to have any adverse effects whatsoever. It happened and you can't change it, so if you intend to continue the pregnancy then rather than dwell on it I would advise you to concentrate on positive healthy behaviours going forward.
(Source for drug and birth defects info: my job as a data analyst for exactly that.)