I'm sorry that you went through all of this! Your husband certainly did not have to get physical, that was completely out of line on his part and an utterly inappropriate response to what you were doing.......
However, I do find it curious that you "needed" to vacuum that particular spot while your AH was speaking with his family. Perhaps instead of letting yourself become annoyed with him, you could have left the room or the house and done something else for a while? Then, when his conversation was finished, you could have finished the vacuuming.....
.....or better yet, you could have let him vacuum that spot once he was finished!
I hope that you are feeling a bit better this evening!
HG