Runescape Highscore Data Grabber

[Leader Rob]

first off i would like to thank TipIt website staff in recently helping me perfect my combat formula on the old combat system,

 

i have managed to make a clan entry calculator that i intend to share with the clan community, i was wondering in terms of design how complicated was it making the username checker tip it use cause i am looking in to making one so the values for my clan entry calculator heres a link for anyone wanting to see

 

www.assentedorder.com/calculator

 

can be automatically acquired?

 

i have seen a few basic PHP data scrappers but i wanted to get the opinion of a runescaper

 

also i will gladly provide a link to tipit for helping me with my code if requested

[Arceus]

I'm not an expert on programming but it should not be too difficult to grab it from here:

 

http://hiscore.runescape.com/index_lite.ws?player=[playername]

 

Of course, since hiscores are not available for free players, it will only work for P2P ones.

[Leader Rob]

thats what i thought but when i used my playername i got this result

 

86644,2131,120747381 178336,90,5759668 174171,89,4870583 164395,96,10482842 180822,94,8222177 183202,85,3535475 122412,85,3553381 198161,90,5682473 93909,99,13237633 33688,99,14855205 207368,82,2442105 21315,99,15306875 140042,86,3939321 99770,80,2009176 101820,80,1991192 67636,87,4094403 155574,72,971940 53710,86,3808801 93640,77,1601220 94589,84,3233030 69750,82,2447800 147642,74,1177700 100311,77,1522917 79053,76,1388744 120936,80,1990377 113162,82,2622343 -1,-1 -1,-1 55325,555415 -1,-1 -1,-1 12844,2550 23278,1530 32122,1356 6436,3582 -1,-1 -1,-1 -1,-1 123019,579 24216,462 20047,617

[Stev]

Didn't they change that, Arceus? My F2P accounts appear on the hiscores.

 

AO-Rob, you have to parse that data.

 

It's

Rank,Level,XP

Rank,Level,XP

...

[Leader Rob]

ive been told to use PHP and Regex but im not too good at those languages, any chance you can help me out developing the code or provide a good guide where i could learn?

[Kaur]

Alright well the basics that you'd need to know to set something like this up is this:

 

1. the website http://hiscore.runescape.com/index_lite.ws?player=name displays the highscores for a given player in a simple string format.

To access this info from your site, you could either use the php function file_get_contents('website'), which would put that hiscore listing into a string. Then you'd use the explode() php function to divide that string up and put it into an array.

 

example:

$contents = file_get_contents(http://hiscore.runescape.com/index_lite.ws?player=Ao_Rob);

^ this puts the source code from that site (your stats) into the variable $contents.

$stats = Array();

$stats = explode('\n', $contents); -- this divides up the hiscores into an array. each one in the array would be an individual stat. $stats[0] would return the player's overall rank, overall skill total and overall xp.

 

You could then do $overall = explode(',' $stats[0]); to put the overall stats into an array. And then if you wanted the overall xp in a variable it'd be $overallxp = $overall[2];

 

If you create loops to go through each of the skills and do this, you can access someone's xp in each of the skills and put them all into variables. And then execute a sql statement to insert the data into a database.

[Ambler]

Hate to thread hijack, but is there a way of doing this in Python? (I use 3.2.3) I've been looking for a way to make one for ages, but can't seem to get it right. :unsure:

[Leader Rob]

but the initial input box would have to store the username in a variable called $username right so would this line be something like

 

$contents = file_get_contents(http://hiscore.runes...s?player=$username);

[Kaida23]

Slightly off topic but...

Didn't they change that, Arceus? My F2P accounts appear on the hiscores.

As far as I know, they didn't. I just looked myself up and I'm not there at all. Are any of your F2P accounts still in, or recently left, trial membership? They aren't always removed right away when it's over.

[Hedgehog]

<?php

$hs = file_get_contents("http://hiscore.runescape.com/index_lite.ws?player=".$_GET["player"]);
$hs = explode("\n",$hs);
$skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction","Summoning","Dungeoneering");
$i = 0;

foreach($skills as $value){
   $hs[$i] = explode(",",$hs[$i]);
   $stats[$value]["rank"] = $hs[$i][0];
   $stats[$value]["level"] = $hs[$i][1];
   $stats[$value]["xp"] = $hs[$i][2];
   $i++;
}
echo $stats["Overall"]["level"];
?>

 

For example, yoursite.com/hiscores.php?player=Meredith would echo "2484".

[Arceus]

Didn't they change that, Arceus? My F2P accounts appear on the hiscores.

 

AO-Rob, you have to parse that data.

 

It's

Rank,Level,XP

Rank,Level,XP

...

As Kaida mentioned, they usually do a big hiscore cleanup, rather than knocking you off the hiscores right away. Perhaps your F2P accounts started with trial membership and haven't been taken off yet. If they haven't ever been P2P, then that would be...interesting.

[Leader Rob]

<?php

$hs = file_get_contents("http://hiscore.runescape.com/index_lite.ws?player=".$_GET["player"]);
$hs = explode("\n",$hs);
$skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction","Summoning","Dungeoneering");
$i = 0;

foreach($skills as $value){
$hs[$i] = explode(",",$hs[$i]);
$stats[$value]["rank"] = $hs[$i][0];
$stats[$value]["level"] = $hs[$i][1];
$stats[$value]["xp"] = $hs[$i][2];
$i++;
}
echo $stats["Overall"]["level"];
?>

 

For example, yoursite.com/hiscores.php?player=Meredith would echo "2484".

 

ok but assuming that gets one set of stats how do i get multiple levels like attack

 

would it be something like this

 

$stats[$value]["attack"] = $hs[$i][5];

[RSBDavid]

<?php

$hs = file_get_contents("http://hiscore.runescape.com/index_lite.ws?player=".$_GET["player"]);
$hs = explode("\n",$hs);
$skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction","Summoning","Dungeoneering");
$i = 0;

foreach($skills as $value){
$hs[$i] = explode(",",$hs[$i]);
$stats[$value]["rank"] = $hs[$i][0];
$stats[$value]["level"] = $hs[$i][1];
$stats[$value]["xp"] = $hs[$i][2];
$i++;
}
echo $stats["Overall"]["level"];
?>

 

For example, yoursite.com/hiscores.php?player=Meredith would echo "2484".

 

ok but assuming that gets one set of stats how do i get multiple levels like attack

 

would it be something like this

 

$stats[$value]["attack"] = $hs[$i][5];

 

 

echo $stats["attack"]["level"];

 

...would echo the attack level. In the foreach-loop it populates the stats array by the skill name as that is the order Jagex uses as well.

[Hedgehog]

Or

 

echo $stats[$_GET["skill"]][$_GET["type"]];

 

Which would look like this in the URL:

 

yoursite.com/hiscores.php?player=Meredith&skill=Overall&type=level, yoursite.com/hiscores.php?player=Meredith&skill=Attack&type=xp, or yoursite.com/hiscores.php?player=Meredith&skill=Dungeoneering&type=rank, etc. depending on what you want.

 

It may be worth converting everything to lowercase so that it is not case sensitive. Also, add something to check if the player is f2p.

[Leader Rob]

It may be worth converting everything to lowercase so that it is not case sensitive. Also, add something to check if the player is f2p.

 

im not too good with PHP as it is, im just about understanding what im getting told

 

would i use an

 

if ($stats != null) {

echo $stats["attack"]["level"];

}

else {

echo 'Sorry your account isn't appearing on highscores, please verify you are a member and your combat levels can appear on the high scores';

}

[Stev]

Hate to thread hijack, but is there a way of doing this in Python? (I use 3.2.3) I've been looking for a way to make one for ages, but can't seem to get it right. :unsure:

Python's split() is perfect for it. :P.

[ozen]

Hi!

Besides players highscores what other data can we get and how?

I'm interested in timers for example tears of guthix and troll invasion etc. So my website or whatever tool will show me when i can go on these minigames.

Items database would also be nice :P

 

edit: ok, i found the GE API and the beastiary API but is there any way to get the time when i could visit skeletal horror again and collect tears of guthix etc??