My own view of casual gamers is that it's simply people who play games who don't identify with being "gamers."

I play a lot. Like three hours a weekday, only curbed because I have anime to watch as well. Yet almost all the games I play are that easy to play sort of game, particularly rhythm games (well, I suppose they do eventually get hard at Oni levels, does that count as not being easy to play though?) I don't really consider myself casual despite this.