That's the part I have an issue with. People love to observe other creatures (eg bird-watching, herping, fishing) but most of the time there is no harm caused to the animal. When fishing, there is. Scuba diving seems to me like a better way to look at sea life. If it is more about getting out there with friends and family and doing something together, then why not get out on the boat, minus the fishing?
I've never been fishing and I don't understand the culture of it, so to me sport fishing would be like bird-watching with hooks to snatch them off the branch because they are too far away, then letting them fly off again.
If our waters were clear and we could use binoculars to observe the fish, would people be happy to do that instead? (It's a genuine question, I'm just trying to understand the reasoning behind fishing for sport.)