If I'm not mistaken most fish farms use fish feed with a mix that can contain wheat derivatives, the problem is that when you eat a simple seafood you're often eating the digestive track of that organism.
Unlike eating a fillet of fish, when you consume a whole sardine, shrimp, or crawfish you're eating more than just it's meat whether you intend to or not...