I had a friend (god rest him) who used to preface his comments about how I ought to be doing something by saying, "I don't want to tell you how to f**k your goat, but ..."
So here I am with a goat comment. I have made quite a few ER collet adapters in several ER sizes. It's not hard, and if you've got several lined up to minimize tool setup and changeover it goes pretty fast. Excluding peripheral machining you've got one external thread (metric) and an internal taper. If you're contemplating a series of ER holders it sounds pretty easy. Probably two piece like Doc suggests so you have a plate and short cylinder assembly, welded or Loctited. Then it's up on a face plate to turn the OD and thread it on them all, followed by drill, bore and turn the internal taper.
Nevertheless, I know whatever you do with your goat is no business of mine.
"I am often asked how radio works. Well, you see, wire telegraphy is like a very long cat. You yank his tail in New York and he meows in Los Angeles. Do you understand this? Now, radio is exactly the same, except that there is no cat." : Albert Einstein