Kay people, it’s Sunday and I’m a bit bored, so I wrote a ruby script to webscrape Starcraft 2 profile pages using the wonderful Nokogiri gem.
Well it’s done now and to be honest I have no idea what to do with it, so I’ll share it with you instead.
#starscrape.rb require 'rubygems' require 'nokogiri' require 'open-uri'def starcraft_profile(url) h = Nokogiri::HTML open(url) { :name=>h.css('#profile-header h2').text, :achievement_points => h.css('#profile-header h3').text, :snapshot => { :favored_race => h.css('#season-snapshot .module-footer a').text, :highest_ranks => Hash[([1,2,3,4].map{|i| ["#{i}on#{i}".to_sym,{ :league => h.xpath("//div[@id=\"best-team-#{i}\"]/div/strong/../@class").text.gsub('badge-',''), :division=> h.xpath("//div[@id=\"best-team-#{i}\"]/div/strong/..").text.match(/Division: (.+)Rank: (\d+)Record: (\d+ - \d+)/)<a href="/posts/1">1</a>.strip, :rank => h.xpath("//div[@id=\"best-team-#{i}\"]/div/strong/..").text.match(/Division: (.+)Rank: (\d+)Record: (\d+ - \d+)/)<a href="/posts/2">2</a>.strip, :record => h.xpath("//div[@id=\"best-team-#{i}\"]/div/strong/..").text.match(/Division: (.+)Rank: (\d+)Record: (\d+ - \d+)/)<a href="/posts/3">3</a>.strip }] })], }, :career_stats => { :league_wins => h.css('#career-stats h2').text, :league_games => h.css('#career-stats ul li span')[0].text, :custom_games => h.css('#career-stats ul li span')<a href="/posts/1">1</a>.text, :coop_vs_ai => h.css('#career-stats ul li span')<a href="/posts/2">2</a>.text, :ffa => h.css('#career-stats ul li span')<a href="/posts/3">3</a>.text, :campaign_status=> h.css('#career-stats h4')<a href="/posts/2">2</a>.text }, :achievements => Hash[h.css('#top-achievements .progress-tile').map{|p| [p.css('a')[0].text.gsub(/[^\w ]/,''),{ :points=> p.css('span').text.strip, :progress => p.css('.progress-bar').attribute('style').value.match(/width: (\S+)%/)<a href="/posts/1">1</a>.to_f }] }], :showcase => h.css('.showcase-tile p a').map{|a| a.text} } end
This code will return a pretty hash stored with all the pretty informaiton: (Yeah I know I’m not hotshot player, but still this is fun ;) )
starcraft_profile http://eu.battle.net/sc2/en/profile/836668/1/telamon/"=>
{:achievement_points=>"2640", :showcase=> ["Team Random 100", "Wings of Liberty: Hard 25", "Hot Shot", "League Qualifier", "Wings of Liberty"], :snapshot=> {:favored_race=>"Random", :highest_ranks=> {:"1on1"=> {:division=>"Roxara Foxtrot", :rank=>"69", :record=>"31 - 41", :league=>"silver"}, :"2on2"=> {:division=>"Immortal Mars", :rank=>"7", :record=>"26 - 21", :league=>"silver"}, :"3on3"=> {:division=>"Sarengo Foxtrot", :rank=>"62", :record=>"11 - 11", :league=>"platinum"}, :"4on4"=> {:division=>"Pridewater Nu", :rank=>"15", :record=>"22 - 28", :league=>"silver"}}}, :career_stats=> {:custom_games=>"50", :coop_vs_ai=>"3", :ffa=>"5", :campaign_status=>"Normal Campaign Ace", :league_wins=>"265", :league_games=>"556"}, :name=>"numri", :achievements=> {"Cooperative"=>{:points=>"0", :progress=>0.0}, "Liberty Campaign"=>{:points=>"1410", :progress=>88.679245283}, "Exploration"=>{:points=>"480", :progress=>100.0}, "Custom Game"=>{:points=>"10", :progress=>1.4705882353}, "Quick Match"=>{:points=>"220", :progress=>25.2873563218}}}