I'm sure that every other person out there who loves anime, manga, and other various Japanese things wants to learn the native language of the art they love. I always have a good laugh when I hear some nerd saying things like, "I'm learning Japanese by watching anime! I'M STUDYING LOL!"
First off, most Japanese used in anime is very informal and is therefore rude to use in an everyday setting (unless you're going to be talking with very close friends). Second, this does nothing to teach you how to read Japanese, so if you're planning to go there, you'll end up being a very annoying tourist, asking everyone what a simple sign means. Third, if you're just learning on your own through anime, you are relying on your own inferring skills, so simply put, you're going to mess up someway somehow. There might be a grammatical rule that you don't understand, difficulties with pronunciation, amongst other issues.