I also wanted to add that unless you are using "superfine" rice flour - that might account for the grittiness. I find things made with regular grind rice flour are heavy and gritty. Superfine makes a BIG difference.
Just my .02 worth, in case you try again. You might replace the sweet rice flour with tapioca flour (keeping with a lighter pancake...