WOW! I am sorry to hear. I feel the same way about my boyfriend. We have been together for 4 years and rarely use protection and I wasn't ever really on birth control. But I have never gotten pregnant by him. Have you been pregnant in the past? You yourself could be infertile. But I suggest having sex with him, keeping the condom, and taking it to the doctor to get tested. You just have to research how to keep the sperm alive.
Same here too me and my boyfriend have been trying to get pregnant every possible way for eight month now but fail to get pregnant. Its 50-50 chance me or him. But his big brother and his wife been trying to and fail so I'm guessing sometime its the guy??. Just look for it it might run in the gene?..
I thought my boyfriend was infertile for near enough all of our relationship and he was even tested and told he had an incredibly low sperm count. A year later I was two months late and it turned out we'd conceived. Just remember it's important to keep getting checked because what may be the case one day could be totally different later.
You can easily know if your boyfriend is fertile or not by a lab test. Get his semen analysis done and if sperm count, motility of sperms, Ph are normal , then certainly he is fertile.