Compete.com Webstats Scrape Groovy
This is a script for collecting webstats data from compete.com. The scripts takes as input the list of domains that you want to analyze and outputs the compete.com webstats data.
Copy this code and paste it in your HTML
import com.gargoylesoftware.htmlunit.WebClient import com.gargoylesoftware.htmlunit.BrowserVersion
def domainList
= ( new File ( "/root/Desktop/Morningstar/AlexaTop3000.txt" ) ) .
readLines ( ) def outFile
= new File ( "/root/Desktop/Morningstar/CompeteStats3000.csv" ) outFile.delete ( )
def wc
= new WebClient
( BrowserVersion.
FIREFOX_3_6 )
def domainName
= it.
trim ( ) def url
= "http://siteanalytics.compete.com/export_csv/${domainName}/" def page
= wc.
getPage ( url
) def pageLines
= page.
getContent ( ) .
split ( "\n " )
outFile.
append ( "\" ${domainName}\" ,${line}\n " ) }
lineCount++
}
sleep( 400 )
}
Report this snippet