At first he would go to sleep fine but then he would only fall asleep with bottle and now he has to be held or on my chest. I have tried laying him down with a pacifier, swaddling him, letting him cry... what is there to do next!?
The first few months of baby's life - but especially the first few weeks - are about teaching baby that you will be there to take care of them when they need you, building trust and a sense of security. If baby feels like you will be there when he needs you, he should be able to sleep fine on his own for brief intervals - maybe up to 4 or 5 hours, but you shouldn't expect that every time. I'd say the fastest way to ruin a baby's sense of security at just a couple of weeks old is to leave it to cry, so if you've been doing that you probably have some repair to do before he will feel safe enough to sleep on his own.
My suggestion would be to take a few days/a week and pay very close attention to what he needs and pick him up whenever he starts fussing (don't wait for him to cry if you can help it). Pretty soon he should start to feel more secure and then when he's fallen asleep he will likely be ok continuing to sleep after being set down.
I think @MAA's answer is good and wise. It is good advice to follow. You can also add a heartbeat simulator (the baby heard this in utero), or a white noise maker at a very low setting. There are certain kinds of "rockers" (they aren't really rockers so much as just moving the baby in a repetitive pattern, like the pricy Momaroo) that soothe a baby, though you are not supposed to let them sleep in them.
I am of a different school of thought, however, and that is that some babies needs a lot of "mommy" (and "daddy") time to be at peace. If your goal is to put him down so that you can get things down, there are a lot of products on the market which allow you to carry the child on your chest while they sleep, and sleeping peacefully, they allow you to get a lot done.
You might not get everything done that you want to do - that extra weight on your chest limits you somewhat - but you'd be pleasantly surprised, I think, at how well your baby sleeps while you go about doing your thing.
Remember that just 3 weeks ago, the baby was moving with you whenever you moved around, He is not used to lying still without movement. :)
The worst cases of this that I've seen are where the mothers were very physically active before giving birth, e.g. were daily runners, or worked out at the gym a lot. These babies needed a lot of jostling to feel "normal", I guess, because they would cry unless really being moved around a lot!