sc2profile, a web scraper library for Battle.net

sc2profile fetches StarCraft II profile information from Battle.net. I wrote it for clan.cx, but it turned out that scraping information off Battle.net is slow. It takes between 300ms and 1500ms to download a page from Battle.net. A profile needs to download at least 2 pages, most profiles need 4 pages or more. Some profiles take more than 10 seconds to be scrapped, that’s too slow for clan.cx.

I ended up using sc2ranks.com’s API, which is fast enough.

sc2profile can still be useful if you don’t want to rely on a service like sc2ranks. It uses lxml to parse & extract information.

If my writing sparked any ideas, questions, or insights, I would love to hear from you.
Email me at henry@precheur.org I read and respond to each email.