"Gaijin" (ŠOl, "outside person") is an offensive term; please don't use it. The correct term is "gaikokujin" (ŠO‘l "foreign country person").
And for the most part, we seem to be tolerated. Personally, I've never encountered anything I'd consider as obvert racism or something like that. There are nationalists who want us out, and who want to put the emperor back in power and such, but besides driving around town in garish black buses they don't really do much.