I would very much advise you to have the entire family go glutenfree. It will be significantly easier to avoid cross contamination that way.
I've noticed for me personally that amount matters. The reaction seems pretty much the same whether I get in a number of crumbs or an entire sandwich, but there is a clear difference between eating something tha...