The default click activation threshold of 25 is super sensitive, and I kept accidentally dragging the menu.
I've found a click activation threshold of 95-100 felt more natural.
Alternatively, a grip binding might also work, but I didn't test that.