when I asked if that meant he wanted to commit to our relationship, he
told me no.
That, to me, is a huge red flag. Even if it weren't a romantic relationship, he's having kids with you and therefore should make some sort of commitment to you (note -- it doesn't have to be a romantic relationship or marriage). Relationships of every sort are built on a foundation of mutual trust. If you don't have that trust, you don't have a relationship, you have a house of cards.
He wants me to move to where he is and live with him again before our
second son is born in three-ish months, but remains adamantly opposed
to committing to me...He's wavered some in the last three years over
committing - he even gave me a ring at one point. However, this spring
I again became accidentally pregnant and I pushed the issue, with the
result that he came out forcefully opposed to marrying/formally
committing to me
Red flag number two. This is evidence of controlling behavior. He's stringing you along. Moving, especially interstate, is an act of commitment on your part, an act he's not willing to make himself. At the very, very least, this is a one-sided situation.
mainly that I dread the pain of constantly being with someone who
doesn't care for me a tenth part of what I care for him, and I fear
becoming completely financially dependent on someone who could very
easily leave me at any time
Red flags three and four. You can't control what other people do. You need to take care of yourself, first, so that you can take care of your kids. Staying with someone that you know causes you stress, fear, and anxiety isn't good for either you or your kids.
The fact that you have no confidence in him staying and fear becoming dependent on him is a huge indicator that your relationship is built on nothing. Again, there needs to be a foundation of trust or there's nothing.
He certainly treats our son well, and when it comes to treatment of
me... well, I will admit that he has been much better the last few
weeks...He has definitely become disproportionately angry during
disagreements in the past and said cruel things to me, and I'm not
sure whether it has crossed the line into abuse or not
Red flags number five and six. At the very least, he has anger management issues and that affects not only your relationship, but the development of your kids.
I can't tell you whether your situation is abuse, per se, but I will tell you that abuse comes in many forms and that it's insidious, precisely because the abuser doesn't do it all the time (this is known as the cycle of abuse).
Additionally, just because he's currently treating your son well now, that doesn't mean that will continue or that he hasn't already displayed that toxic behavior toward your son. Consider his behavior toward you as foreshadowing of his behavior toward your kids as they get older.
I'd say partly they don't get to see much of me when we're together,
but they are also angry that he has not committed, and they feel that
our participation in the relationship and the duties of childrearing
have always been extremely unequal.
Red flags seven and eight.
Childrearing is a team effort. "It takes a village to raise a child" is not hyperbole. If it's that noticeably one-sided, there are fundamental problems with your relationship that need addressed.
Regarding your friends not getting to see you, you'll have to assess this one for yourself to be sure, but based on what you've said here, I'm marking it as a red flag. Isolation is a control behavior in abusive/toxic relationships. What makes it hard to pinpoint is that reduced socializing with a group of people is very easy to justify (need couple/family time, don't want to deal with people, etc), and total isolation happens slowly. Be mindful of this as a pattern when you assess your path.
But is it worth making those sacrifices for our children to get to
have both parents around?
In short, no.
A multiple-parent household is good for many reasons. However, a toxic environment is a toxic environment and blood relation doesn't overcome poison. A child is far better off in a single parent environment than in a toxic multi-parent environment.
At the very least, it sounds like you two want different things from the relationship, so you need to redefine the relationship between you two. That alone should be reason enough to go your separate ways and work something out for co-parenting or visitation rights (because (assuming the relationship can be healthy), separating from you does not necessarily require separating from the kids). It's not fair to any of you to continue in your current arrangement if it's not mutually beneficial.
However, I highly recommend sitting down -- ideally with a counsellor trained in this -- and taking a hard look at the relationship you two have, so that you can go forward with confidence in whatever route you take.
Additionally, I highly recommend the book More Than Two. While it's geared toward poly relationships, it's one of the best relationship books I've read, because it shines a light on the hidden issues of toxic relationships (many of which may be able to be glossed over and fester for years and even decades in a monogamous relationship, poisoning the relationship from the inside until extensive damage is done) and how to avoid and fix them. It also teaches that it's okay to have boundaries and desires in a relationship, and that it's okay to assert them, and if a romantic relationship doesn't work out, it's okay to leave it before you hate each other.
If I were in your shoes, I'd wash my hands of him at least when it came to my personal relationship with him, and play it by ear when it comes to his relationship with the kids. I don't know who moved where and when, but if he really wants to be involved with the kids, he can put forth some effort for visits instead of expecting you to move to him.
Being a single parent isn't a forever thing unless you choose it to be, and it's not going to irreparably damage your kids to live in a single parent household. By refusing to be strung along, you open yourself up to the opportunity of a better relationship, with someone whom you can trust and that will have no problem committing to you and your kids.
(For some perspective -- a friend of mine had a toxic relationship with her ex, so she left with her youngest, who was only a baby at the time. This opened up the opportunity for her now-husband to come into her life and step up as her son's father in a supportive, equal, healthy relationship. The boy is far, far better off in the blended environment than he would have been in the toxic one with his biological father.)