detoid9

OK thanks for that. So my only options for collecting this information is either to a.) crawl a item db like before (I forgot which help site I crawled last time), or b.) team up with a RuneScape Help Site and piggyback off their database. From memory, crawling a site completely saturated my internet connection, so piggybacking is the preferred option of course :)

If you get claws, just make sure you don't rush in PvP, or this might happen:

Hey all! Does Tip.It really alch every single item to get it's high alch values? I feel sorry for the guy who had to alch his devine if that was the case! If not, is there a source for this information from Runescape.com itself? Reason I would like to know, as a while back I worked on this: http://forum.tip.it/topic/244271-web-developer-in-need-of-alch-prices-and-more/ A high performance PHP G.E. crawler and indexer. It turned out better than expected, and I had a bunch of fun with the information I gathered. But I am still wondering if there is a better way of collecting alch prices. It has been some time since I worked on that PHP crawler and I have better technology and knowledge to work with, and would find it fun to make a even better one with http://nodejs.org/ or Ruby on Rails. Any answers or opinions are welcome! P.S. If you want me to develop any custom solutions for this site I am more than happy to, for free as well. I have had past experience in making custom modifications to I.P.B. installations, making web apps etc.

Almost any experienced player would recommend this method. Less likely to burn out combat + slayer xp is really useful.

It works out when to stop, by setting a threshold that after so many bad ID's, it decides that it has reached the end.

A basic example in PHP would be: <?php $ch = curl_init(); curl_setopt( $ch, CURLOPT_HEADER, 0 ); curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 ); curl_setopt( $ch, CURLOPT_URL, "url_to_item_page" ); $text = curl_exec( $ch ); // $text now contains the webpage source from the specified URI, so we can parse it using preg_match() etc. // This just so happens to be code I use in my G.E. Crawler if ( preg_match( '/Market price:<\/b>\\s([,.km\\d]+)/i', $text, $match ) ) $ge_market_price = $match[1]; else $ge_market_price = null; // Will put the market price to the page. To make it a pure integer you still need to convert values like 120.5k to 120500 etc echo $ge_market_price; curl_close( $ch ); ?> If you were to be crawling the entire G.E. database then I would highly recommend using multi-threaded cURL, to greatly reduce latency and speed. You have already contacted me about my PHP Crawler code which uses this method.

OK guys I have finished my PHP crawler class. If you want a copy just PM me. So far it will crawl the official G.E. Database, and a 'unofficial' item database to gather as much information as possible. Once the initial index is complete, a considerable amount of time is taken from following crawls, unless you tell it to do a complete re-crawl. Enough of all this 'crawling', and time to go make some stuff with all this info!

Well my crawler has finished indexing (just a few seconds ago), and with the initial index only 3163 of those 15000+ ID's are actually on the G.E. Database. So I only need information for the 3100+ tradable items. So it shouldn't be too hard.

Tanks ftw? Get 99 Defence 99 range. You shall wreck havoc on all...

Hmm the Tip.It Item Database is somewhat in complete with only 3000+ items in it. See the full list here: http://www.tip.it/runescape/index.php?r ... rch=Search I might look else where for sourcing my Alch and Stats information... Any suggestions?

Definitely get about 85+ strength before pking.

Well I have the G.E. crawler at near completion. Now I have to incorporate the item stats and alch prices etc. I'm tossing up what site to use and how to link items together... EDIT: G.E. crawler done :) It will take a hour or two to make the initial index though -.- EDIT2: Now using multi-threading to speed the process up considerably. It's maxing out my internet connection :P On a 512kbs connection I am getting around 1000 pages every 20 minutes. Should go a lot faster on a server.

A completely automated system is what I'm going to aiming for, so setting a limit isn't really a option. There should be any gaps larger than 100 ID's, so I might set it at that. Also another obstacle to hurdle over is pairing the alch prices with the GE prices. A straight text match might be somewhat un-reliable. But again I'll have to look into that a bit more.

Yeah hopefully I can find some hosting that doesn't have the annoying PHP execution timeouts. Crawling the entire GE database might take a few minutes and most cheap hosting solutions have around 30 second timeouts to save on resources. I'll probably be running this on my local server initially however. I could probably get some free hosting off a friend on a dedicated server. Lucky High Alch prices are static, so a one-time index should do fine. That should cut down a lot of work. Once the database is all initialised things should be all plain sailing. I may even make a Open Source PHP class for others to use ;) A couple of concerns still remain though: - Possible situation where Jagex fills in a bad ID with a new item? Not sure if that will ever happen however. - How to know what is a bad ID and when to end the loop. E.g. How do I know I am at the last item? That is only if bad ID's are treated the same as going over the last ID number. Only way I am currently aware of determining this is if it encounters so many bad ID's in a row, it then declares it has reached the end of the item list. Anyways, a bit of mind spillage. I may release a domain later down the track with some finalized idea's. If I do I will certainly be posting them back here.

Hi to all, especially those you have knowledge in web development. I am wanting to make some great online tools to aid RuneScapers in the area of High Alchemy and RuneScape items. I have made many Web 2.0 apps and other PHP / MySQL / Javascript mash-ups in my career, and have wanted to make some RuneScape tools for some time. Unfortunately I haven't settled on a reliable source of High Alchemy prices that is easily accessible without having to do resource intensive operations like CURL. So was just wondering if anyone out there has something that can fill this hole for me, and in return maybe I can give some coding assistance. I could even give the Tip.It Item Database a make-over if need be... Some projects I had planned: - A PHP on-demand GE price checker with caching. Everytime a user requests item information with prices older than so many hours, it will automatically update GE prices. - High Alchemy adviser. Recommends best items to use with High Alchemy. Languages I am very familiar with (code everyday in them): XHTML, CSS, PHP, Javascript. If Tip.It need any of my expertise, just send me a PM or something :)