The fact that a feature is of a complementary distribution does not seem to be a sufficient reason to discard the feature as irrelevant; especially as they seem phenomenologically relevant.
Features exhibiting complementary distribution or negative point-wise mutual information with the output variable are not necessarily irrelevant. Their lack of co-occurrence might reveal unique or inverse relationships that are valuable for understanding complex phenomena.