Am i a wimp for this?

Of course you're not a wimp, you did the right thing by not retaliating. My kids go to martial arts lessons and their master tells them to always walk away from a fight if possible, you were definitely the bigger man for not hitting back. Don't be down about it, sounds like you avoided what could have been a really nasty situation pretty well to me:)