Is this an allergy to chemicals or the fruits themselves?

you could see how you react to an organic apple etc, but I would guess it is the fruit you are reacting to