Gun cleaning question?

I recently bought a Springfield xd9. I go shoot it just about every weekend and clean it after every time i shoot. I start off by running a patch with solvent on it through the barrel. Repeat. Then I use the bore brush and scrub it. I then run another patch with solvent through it and it comes out BLACK. I clean the brush and scrub again. Next patch is black. I repeat this at least 10 times and still, dirty patches. The bore looks smooth and clean and clear of debri but the patches stay black as long as I brush it. Do I really need to keep brushing/swabbing with patch back and forth until the patches come out clean, even if it takes 50 times? And am I doing something wrong here?