I find that the burrs are usually in one place and leave a groove partially around the shank with a built up knob at it's end. I have used files and even grinding wheels in the past, but presently I prefer to use a small stone (1/4" diameter) in the Dremel. I hold the drill at a right angle to the Dremel axis and try to hit the built up knob only. You can take it out to below the shank's OD but the remaining, untouched area above and below this small notch will allow chucking it properly. I use a strip of fine (150-220 grit) sandpaper like a shoe polishing rag to remove any remaining roughness. Most drill chucks are not that accurate, but if you are anal about it, you can rotate the drill and sand it in that manner from three or four equally spaced directions to take the same amount off all sides. Hold it with aluminum or wood jaws in the shop vise for this sanding.
This goes very quickly and the worst part is taking the Dremel out and putting it away. Save up several damaged drills to do at once and the process is much more efficient.
Make it fit.
You can't win and there is a penalty for trying!