I personally do use them, but not as much as a lot of other people.
Because of my hotbar layout (1: sword, 2: bow, 3: fishing rod, 9: FnS) it's not perfectly easy for me to switch from my sword to by rod then back, as I have to skip past my bow. Without a gaming mouse with buttons, I often accidentally click on my bow instead of either one, and it completely loses any advantage I may have gotten - and sometimes I even hit people with my bow instead of my sword by accident!
If I'm chasing someone who's taking a long route, I'll often hit them to try to knock them into walls, make their route longer (So I can slowly cut them off) etc., and only if I'm fighting a really good player will I use my fishing rod, as I'm going try-hard mode, to put it simply. But I just find it too intense for an average fight, since I'm almost always good enough to win without it. (Unless I'm going against a super stacked person.)