Yes, there are certain group of individuals with some specific disease condition who cannot consume some low-calorie sweeteners. For example, people with a genetic disorder namely, Phenylketonuria (PKU) cannot consume aspartame because they cannot metabolize phenylalanine which is breakdown product of aspartame.