I used to be ok but recently I started reacting to my regular fish oil supplement. It's an allergy just like many reactions which trigger panic attacks. They just get blown out of proportion into these attacks because our adrenals are all worn down (referring to those with panic/anxiety problems).
It might be helpful to switch the type of fish. Krill might be ok for you if your current one isn't. I'd suggest muscle testing various fish oils in a store and see which ones test ok. If you don't know how to muscle test you can simply start looking at what fish oils have you reacting and slowly eliminate all the ones that do.