First of all, do the diagram every day if you can. If you get a coffer, easy way for a good jewel. Also, try and farm as many wildlands with archers. I've gotten a few silk yarns for levels 4-6 wildlands, and a sky stone from a level 8. Since you're already ministrel prefect, you can exchange lower jewels for higher ones, so start exchanging the lowers ones you get from bobs and farming. Since you can change silk yarns into azure stones and the like, stop exchanging when you get to silk yarns, and just stockpile on those, since you can exchange for higher jewels anytime. The most important thing is that sky stones take up the exact same amount of level 3 meteors as azure stones, just extra meteor 1 and 2. So try to use meteors for the sky stone, not the azures. Definitely not the silk.