It’s difficult to say without more detail. But I don’t think that’s the right question right now. Plus, it depends on the type of company or projects you’re aiming to work on. And it depends on your own interests and aptitudes.
When you say something like this, I imagine there is more to the story. You are heavily summarizing a complex social interaction. And even if every hiring manager you’ve spoken to said that exact same thing, it doesn’t necessarily mean that’s why they didn’t hire you.
That could be polite short form for, “your reel isn’t good enough, keep trying.” Which means the answer might not be “specialize”. The answer might be to get some trusted feedback on how you can improve your demoreel. (And specializing might be one potential answer! Or maybe you just need a talented friend who can help fill in some parts that aren’t currently working, like animation, rendering, or even video editing.)
Perhaps you have some industry friends you trust. Or a small Discord community. Or you could post it here, if you’re willing. Maybe it isn’t your demoreel! Whatever the reason they are saying that to you, my advice is to seek out trusted feedback. You should not necessarily take feedback you get in an interview at face value.