unity - Rendering hundreds of animated characters in Unity3D

I am currently doing some research in anticipation to the development of a game where there is going to be hundreds of secondary entities which will interact with the player and among themselves (little soldiers with a very simple AI and not too many faces).

What I've found so far is that there is a way of avoiding the GameObject overhead, at least for the rendering bit of things, by using the Graphics class, with the Drawmesh method more specifically.

I've tested it and it's not too complicated. I could render any of my models using it and then make them move by rendering the graphics around coordinates in some non-unity "Character" object. The fact that the drawmesh call has to be every frame is actually quite convenient in this case !

However, I've ran into a problem : how do I deal with animated objects ? Now, I've also done some research on that, and what I've found is that you can "Bake" the skinned mesh into a mesh usable with Graphics.Drawmesh. Then I thought about rendering animated far away characters by updating their mesh with their current animation at a certain interval. This would make the character look wierd but at long distances and among many others it should render quite well.

But the thing is, since I don't want to create a GameObject for each entity, or more precisely only for the ones up close that need a collider and everything so the players can target them with a raycast. And I sure as hell don't want to create a skinned mesh renderer for each far-away character, as it would defeat the point of using Graphics.Drawmesh.

I've thought about one solution to this problem : I could have one "hidden" GameObject with the proper models and animations get requests from "Optimized", meaning far-away character to set himself to a certain frame of a certain animation, bake his mesh and send it back to whoever requested it. I did not run any tests with that solution (I came up with it while writing this) but I'm kinda afraid that updating the mesh of a single entity potentially hundreds of times per frame might lead to wierd behaviors. Besides, I'd have to put it far away so that it doesn't get found by the players.

So, my exact question is : Do you have any better idea as to how I could optimize rendering all those characters some other way, and if not, is there a way to ask Unity "what would this mesh look like on this frame of this animation", thus allowing me to avoid creating a Gameobject just for that.

Thanks a lot in advance ! Cheers !