Compatibility of an ion is controlled by two things: its valence and its ionic radius. Both must approximate those of the major element for the trace element to be compatible in the mineral. For instance, olivine (an abundant mineral in the upper mantle) has the chemical formula (Mg,Fe)2SiO4. Nickel, with very similar chemical behaviour to iron and magnesium, substitutes readily for them and hence is very compatible in the mantle. The compatibility of an element in a rock is a weighted average of its compatibility in each of the minerals present. By contrast, an incompatible element is one that is least stable within its crystal structure.
In general, when an element is referred to as being "compatible" without mentioning what rock it is compatible in, the mantle is implied. Thus incompatible elements are those that are enriched in the continental crust and depleted in the mantle. Examples include: rubidium, barium, uranium, and lanthanum. Compatible elements are depleted in the crust and enriched in the mantle, with examples nickel and titanium.
White, William M., 2005. Geochemistry (Online textbook)