Yesterday, I had a similar problem. 100 p.c. system on my 1-core system, terrible, but distributed between Chrome and YTD Downloader (4.3), and after (!) I had downloaded a group of files with YTD Downloader... I had to shut down YTD Downloader, in order to get even Chrome revert to normal state, so I suppose if it's not some "special" site that slows down Chrome in such a way, it's the INTERACTION with something else.
This is weird and should not occur, but I like Chrome too much in order to switch to FF, and in my tries with FF, I also had moments where things didn't went as smoothly as they should have, so there seems to be an "aleatoric factor" in these things.
In my case, that'd be: Switch to another YT downloader, but stay with Chrome; I hope you find a similar cause and solution.