If ingrown hair has turned into painful boils with pus, try cleaning the area gently with warm water and soap. Apply warm compresses and avoid picking at the boils. Using over the counter antibiotics and wearing loose clothing can help. Maintain good hygiene and consider pain relief if needed. However, if the situation doesn't improve, worsens, or spreads, consult a dermatologist for proper treatment.