If you cannot boost for a skill door, you just have to accept it and move on. Not all bonus rooms are able to be opened. I've never used a hoardstalker myself since they seem pretty useless, but going on how invisible boosts work outside of Daemonheim, it would not help you for skill doors or anything like that. It would just speed up how fast you fish, chop wood, etc.