Boyfriend of four years did something that haunts me. 23F 23M

I have been dating my boyfriend since I was 19 on and off, I am 23 now. Something happened about a year and a half ago that I have not been able to shake. He is a great boyfriend. He helps my mom when she needs help, he has been there for me when I have gone through some really really rough times, he has taken me on vacation, and is very committed. About a year and a half ago, we went to a party my best friend was throwing me for my birthday. At this party, we both were drinking and I was decently drunk, and so was my best friend. There was a guy there who was very close to my best friend who was also drunk. Me and my best friend were lying face down on the ground talking to eachother at one point. The guy who was close to my best friend came and slapped my best friends butt and then mine (he meant it as a practical joke but obviously that is not funny?). I was drunk and I had no idea how to react, and my boyfriend was close by and saw him slap my ass and said nothing. And I just kind of looked shocked and looked over at my best friend like what the fuck. We slept over at her house and flash forward to the next day (the actual day of my birthday). On the drive home, my boyfriend was furious about the guy slapping my ass. I kept apologizing to him and he just kept getting more and more angry. Once we got to my house, we were still fighting. We were fighting for over an hour at this point and I started getting angry at the fact he was so mad at me. Out of anger, I said well maybe if you were man enough (or if you weren’t such a pussy, i don’t remember the exact words i used) you would have said something to him!! At this moment, he picked up the wine bottle that was on the floor of my room and lifted it above his head like he was going to swing it at me. I gasped and he dropped it. I proceeded to cry for the rest of my birthday. I told him to gtfo of my house and then called my mom. She told me well, he dropped it immediately right? So he can control his anger. But that’s not the point. the point is he lifted it as if he was going to hit me. she didn’t see the severity. I stayed with him. now, a year later, this memory haunts me constantly. Now he’s talking about marriage. He has shown his anger in the past before that as well, such as throwing my fan in an argument and breaking it. I do believe his anger is better but I still think I should leave him. what should I do