Drugs is one major factor which even includes smoking. People consuming drugs always face this problem as drugs directly affects one's brain. I personally hardly know couple of people who are not in to any such activity and they are very active and their memory power sometimes surprises you.
Bad routine is one more factor which gets you sluggish and affects your thinking abilities plus affects memory power up to larger extent.
Same is the case with people having less sleep who obviously then take stimulants like smoking or excessive consumption of tea coffee which really affects your brain power.
Now how to improve your memory power. Making a good routine plus consuming healthy diet specially fruits with vitamins and proteins like oranges,lemon,grapes etc also vegetables having vitamins like carrots etc. Carbohydrates are also very important on daily basis for brain.
These are the primary things to look after but after this one can read more and more good material specifically classic novels which can amuse you as well as help you a lot in improving your memory power.
A friend of mine used to suggest that math, logic etc exercises are also very good in helping in improving memory. Initially I had different views but then with couple of really good experiences now I believe they really do work also.
Hope you find this useful piece of information I gave you. Do work on it but I would suggest these days different drugs are very common so if you do use any even on smaller scale try to quit that first believe me this step can work really good.