Once the bacteria enter the skin, they set up an infection around the base of hairs, causing painful lumps that swell, fill up with pus and eventually burst. Using ice can help relieve pain but ultimately you may want to burst the boil by lancing to release pus build-up, although you will need to be very hygienic because it is easy to re-infect the area.
Make sure that your hands are clean, use antibacterial soap and cover the boil. If possible, shower last thing at night and early in the morning.
Some people will advise you to burst the boil yourself, but it would be a better idea to have it done by a medical person who will usually drain the pus using a needle and syringe or by making a small cut in the skin. The wound is then cleaned and covered with a dressing until it heals.
The skin usually heals quickly once the pus has been drained but sometimes a course of antibiotics is prescribed to help clear the bacterial infection from the skin.
It would be worthwhile consulting a doctor, especially if the boils keep recurring. Tests can be done to find out the underlying cause and to rule out a condition called Hidradenitis suppurativa that results in excessive boils, ranging in size from small peas to large baseballs. These boils are not caused by bacteria, so antibiotics would not help and you would be wasting your money on them and still have the boils!