I say don't worry. Worrying is too taxing. Just follow a mop up program and be on your marry way towards a better health. You didn't give it a second thought when you got the filling in the first place. Don't let the removal process take power over you and give you fear. Easier said then done but you are the one with the power now. Not the removal process.
I had 21 removed in one week in 2005. Did a mop up job for 53 days and I got my life back.