First: make sure you have the belt sufficiently tight. A tight enough 5-point carseat belt should not allow the child to remove his arms. Tight enough means you can get two fingers under it but not more than that. It's probably a lot tighter than you think it should be - and definitely will be tight for him.
Second: talk to your child. Two is old enough to start understanding why. Keep at it. Model for him. Each time you have to fix it, stop like you say, fix it, and talk about why he needs to keep the belt on properly.