It can happen and the only way to find out for sure is to get examined by your doctor who can make an informed diagnosis and prescribe the right treatment.
Can an ovarian cyst rupture and cause a blockage resulting in constipation?
I am not a doctor ,so see a doctor if you are worried. We can't diagnose a treatment like your doctor can through internal examination with a scan or exploring. According to what i just read it is unlikely they will, and most go away on their own or with medical treatment/medicine.this is why your doctor wants to follow up and will probably want other scheduled visits to prevent any further concerns.