I have noticed the same, Gnome Stronghold is often much faster than the other plots. I always plant Gnomes Stronghold, Tre Gnome Village, and Brimhaven, in that order, I normally don't cut down my palm in Catherby as I used to make a bit of cash on selling coconuts. The Tree in Gnomes is always the first to grow, and since the one in Tre Gnomes Village has always been completed when the one in Brimhaven was, I have assumed it was caused by being planted a tick before the others.