The fact that a species of whale (or any animal for that matter) is not endangered is no reason to hunt it until it becomes endangered. "Oh look, thats a pretty animal, is it endangered? No, well lets kill them all then." Absolutely ridiculous. The ecological balance can be disturbed by even a slight drop in the numbers of a species.
You are right however about the frequency of Japanese that eat whale, a very small percentage. Many Japanese people oppose whaling it is those in control that have a taste for the meat however and so it stays out of the Japanese media. It would be a lie to say that the issue is not at all discussed in Japan however, I know of a good many Japanese people who openly opposed whaling while living in Japan. Simply because the bigshots keep it out of the news doesn't mean it isn't discussed at all.
Im sure you will label me a racist as seems to be your most prominent argument but I do not beleive the Japanese are the problem but rather certain government and corporate entities (not only and certainly not most prominently those in Japan) that cannot get over the God complex many human beings seem to have. We may be the dominant species but that doesn't give us the right of dictation of how all other species will exist, in fact it leaves us only with a heavy responsibility to protect everything else on the planet from untimely demise.