Copy website content in to excel (Search result)

[greenculture]

I want to Scrape all data from

 

 http://portal.maccem.com.au:8080/cgi-bin/portalnew.sh/portal/namesenq.p?

 

"Names Register List" Search into an Excel spreadsheet.

Will required input of multiple Surname values like aaaa, aaab, aaac... baa bab bac... to gather all data because each search only returns 200 results.

Any one create a vba template to get all data or a Software

[Sy_Accursed]

The bascis for doing tht would be:

=ImportHTML("http://portal.maccem.com.au:8080/cgi-bin/portalnew.sh/portal/namesenq.p?surname=NAME&firstname=&othernames=&knownas=&yearfrom=&yearto=&postfunc=Search&companyid=3&scrollx=0&scrolly=0","table",2)

Just changing 'NAME' to the desired search string in each instance.

 

For slightly easier editting you could do

=ImportHTML(CONCAT(CONCAT("http://portal.maccem.com.au:8080/cgi-bin/portalnew.sh/portal/namesenq.p?surname=", A1), "&firstname=&othernames=&knownas=&yearfrom=&yearto=&postfunc=Search&companyid=3&scrollx=0&scrolly=0"),"table",2)

Where cell A1 contains the search term, meaning you could setup a series referencing different cells for quick changes.

 

The only issue you will run in too is that functions like these are limited to ~20 per workbook; so it'd be quite hard to extract all the data to excel in this way and it runs the risk of everytime you open the work book 1 or more of them goes to error due to the requested url rejecting requests made so close together.

 

What you really need to do is find where it is pulling that data from and see if you cannot tap in to that directly to extract all the data, probably in one long string and then deal with processing that string back into a coherent table. Unfortunately I can't spot where it is pulling the data from in the source code as I'm not familiar enough with the ajax etc stuff they are using.