My Machinery's Handbook gives the typical drilling speed and feed for A2 with a HSS drill to be .007" feed/rev and 85 sfm, which works out to very nearly 650 RPM. Optimal speed and feed are given as .015" feed/rev and 45 sfm (~350 RPM).
I have a feeling that any work hardening you are experiencing might be due to insufficient feed. Keep using plenty of cutting fluid. You may also be having problems if you are using cheap/dull drills - I would recommend a cobalt HSS (M42) drill for longer edge life if you have lots of holes, and a 135* split point for the harder material. Name brand drills (Precision Twist Drill, Cleveland, Morse, etc.) will work better for this sort of thing due to better geometry.
Let us know how it turns out, that epoxy granite is some interesting stuff.
In theory, there is no difference between theory and practice... but in practice, there is.