There always is even with the best burr grinder.
There is a difference between sludge and grinds.
Grinds slip through the filter when you use coffee that is not course ground.
Sludge is really microparticles that drift to the bottom and then adhere to each other (think - bottom of a glass of Nestle Quik chocolate milk).
If you shake or mix, the microparticles dissipate and you wont get the sludge.