c++ – self extending factory: return type and serialization

I’m trying to implement a better management for the objects in my game project.
With input from another thread here on S.O., I decided to implement different collections for different types for better access. Seems to be a common way. So far I #included all the class files and added containers (just maps right now) for each type I needed/wanted, as well as functions to create/add/remove etc., that take care of what is needed. That way no free objects exist (object’s constructors are private) that could be missed somewhere.
But ofc that means for every new type the container/manager/factory needs to be modified and that’s error prone. So I wanted to try and make it self extending with a registration like this:

template<typename ObjectType>
class Registration
{
    public:
        Registration(const std::string& typeID)                                                                                                
        {
            EntMan::getInstance()->registerTypeCreator(typeID, () (Properties* props)->Entity* {return new ObjectType(props); } );
        }
};

saving it in a std::map<std::string, std::function<Entity*(Properties* props)> >. This works, ofc it’s a common pattern, but has especially one flaw: the return type.
I could add as much additional containers, defined by tags for example, as I want (shurely helps with iteration/preselection), but the contained items would always be just base class pointers.
So if I get an item I still would need to dynamic_cast (or static_cast with if(object->getType == "someThing") ) it to the derived type to get full derived access to the object. Seems wasteful and error prone.
Integrating all functionality with virtual functions would solve this, but in the end turns the base in a kind of blob class. Doesn’t feel right either.
Let alone the fact that it would also be also pretty horrible for data locality. Not my main concern right now, but maybe will be in the future.

I’m a bit uneasy with the Properties “parameter amalgamation”-workaround aswell, to fullfill the std::function criteria of having the same signature, as it’s a bit error prone doing it “by hand” for example, but should be less of a problem once it’s abstracted away with parsing those as prefabs from files and
accessing those. Still, it also would mean a lot of wasted space to save those as prefabs…
So I’m at my wits end here.

How could I go on from here, if I need…

  • different collections for different types (for preselection)
  • acessors delivering the proper derived types (for avoiding casts and blobs)
  • self extending, no modification needed just for adding types
  • serializable use (ie: create(“stone”) for use with parsing functions of xml data etc. )

Can this be done? Or is this a dead end? No good idea at all?
Thanks in advance for any help.