My experience is exactly the same as yours and I think unlikely to be an allergy. I have not been differentially diagnosed as Celiac vs. IBS - probably IBS based on the spectrum of foods I must avoid. Either way, I wonder about the reason for the lip swelling as well.