Jump to content

Welcome to Rune Tips, the first ever RuneScape help site. We aim to offer skill guides, quest guides, maps, calculators, informative databases, tips, and much more to help you get the most from the Massive Online Adventure Game, RuneScape, by Jagex Ltd © 2009.

Report Ad

Welcome to Forum.Tip.It
Register now to gain access to all of our features. Once registered and logged in, you will be able to create topics, post replies to existing threads, give reputation to your fellow members, get your own private messenger, post status updates, manage your profile and so much more. If you already have an account, login here - otherwise create an account for free today!

Ronan

Ronan

Member Since 28 Oct 2007
Offline Last Active Mar 09 2012 10:04 PM

In Topic: All Tradeable Runescape Item IDs

02 March 2012 - 06:06 PM

In regards to the 200s, I'll just throw in a check for that, and wait some number of seconds before trying again.

Keep in mind it's a HTTP 200 status response, alongside Content-Length being 0 - just thought i'd clarify that!

In Topic: All Tradeable Runescape Item IDs

12 February 2012 - 12:44 PM

Hey, that's great Markup - they could be very useful as a starting point for those not looking to parse the full G.E to populate a database, cheers! (Although they're 403ing at the moment!)

In Topic: All Tradeable Runescape Item IDs

09 February 2012 - 09:20 AM

Cheers for putting that online - there's a few differences to what mine was so that's quite interesting to see how someone else would do it!

I don't have much of a chance to test it at the moment, but I'm interested that a 5 second wait time is enough to not throw up any blocking errors from Jagex. Although, granted; in my testing I was parsing the item's pages after searching for pages - a further 3,400+ requests is going to change things!

I did notice a couple of small things, Line 114 in Program.cs, you set reserror to indicate a WebException:

catch (WebException ex) {
    PageErrors.Add(bs[i].ToString() + " Error:" + ex.Status);
    reserror = true;
}
if (reserror == true) {
    reserror = false;
    continue;
}

As far as I can tell, reserror isn't used anywhere else - it'd make more sense to just continue inside the catch-exception block rather than flag an error?

I also noticed that your PageLoader class uses ArrayList as it's base class. To me that seems a little counter-intuitive, the class is making use of an ArrayList, rather than actually extending it? It seems cleaner to just have a class variable storing the ArrayList and accessors to that - less coupling that way.

Lastly, I see you're not checking for Jagex blocking requests. When they block a request, they return a HTTP 200 OK status, which means no WebException is going to occur - the way to identify it is that the Content-Length response header indicates 0 length. Right now it looks like you're just skipping over this if there is no content - or, possible worst case, passing an empty string to the Json Converter? The biggest trouble I found with this is integrating it into a library such that callers either, 1) Don't have to deal with bottling issues / exceptions or 2) Have to catch the exception and re-send the request. I resorted to providing access to the library API through a gateway, where exceptions had to be caught; but alongside this I constructed an Asynchronous requester which allows callers to request callbacks when the page is retrieved - which would be whenever Jagex stop blocking the requests.

Other than those, I do like it - there's a few points in there that I hadn't thought of and I like the Json parser, wasn't aware of JavascriptSerializer. Nice job! =D>

If you're interested in seeing the API I constructed, I've thrown that up on GitHub: GitHub - GeApi. It's nowhere near perfect but may give you some ideas like yours did mine!

In Topic: I keep drooling on my keypad

09 February 2012 - 08:37 AM

And that chair in picture three doesn't exist.

Hey, my chair does exactly this! Haven't ever used it like that though, it actually is pretty comfy, although I feel like I want to be closer to the screen!

In Topic: All Tradeable Runescape Item IDs

08 February 2012 - 04:54 PM

[...]

I will say... it wasn't an easy task. I actually wrote my own classes to parse the .json. I used the category API with each letter as you said to compile the list of pages with items on them. The challenge that you mentioned about the server requests was a simple but oblivious fix. I used System.threading.thread.sleep(). Its tacky but it works. I also closed the connection after each response. I'll answer the rest of your questions when I add the source. I'm gonna work on creating a library like you mentioned.


Sounds like you went through the same thought process as me then - good stuff! Look forward to seeing the result.