Do you think that bothers me?
Heck no.
The key part of that sentence is "remember". They REMEMBER me.
Sure, it might only be because of my association with a slimy white food item (even in South Africa) or because of my obsession with numbers or my poor social skills but they REMEMBER me. The next time they see my name, they might be inclined to come to my talk. The next time they need a nerd to translate some technical topic, they just might think of me. That is my goal.
What are you doing to make sure people remember you?