The problem with Corel Font Manager is it can really slow down certain operations of CorelDRAW if the folders it watches have thousands or even tens of thousands of font files. It takes more time for the program to launch. CDR files with editable text objects take significantly longer to open than CDR files with all type converted to curves. The difference can be pretty dramatic (and annoying) if the CDR file has a lot of fonts.
Some of the boggy behavior can be alleviated by using the font filters in the CorelDRAW font menu. Most of the time I only have it showing fonts actually installed in the system and then filter down farther than that. My work desktop PC and notebook are starting to show their age. Perhaps a newer, high end PC with a fast CPU and lots more RAM might not suffer such a speed penalty.
Regarding the loading of fonts in Corel Font Manager, you should only have to do that once. It will actively monitor font files or folders you add or remove from the folders it watches. For instance I bought the Avenir Next Pro family at MyFonts a couple days ago (on sale for $79, regularly $499). I put the folder of fonts into my font collection folder. It automatically showed up in the CorelDRAW font menu when all filters were unchecked to show the entire font collection.