Ryan's Journal

I went to gather some programmatically generated data off the web for a personal project and found that most of my toolkit for web scraping had rotted away. I was fairly fond of grabbing data with Solvent, but Solvent and Piggy Bank stopped working after Firefox 3's wide release. I managed to hack on Solvent to a point where it looked the same as it did in Firefox 2, but it's quite dependent on Piggy Bank - and I don't want to touch the pig at all.

There's also Fresno and Crowbar, but having spent time re-learning Firefox add-on development, I didn't feel like going in for more re-education. So I wrote it up in Perl. I was a bit sad. Maybe I'll go back and look at Crowbar again, but Crowbar's not as great without the interactive environment of Solvent.

It's probably time to totally divorce Solvent from dependencies on Piggy Bank, only enabling those portions of Solvent's guts when PB is co-installed, which would be just about never now. That's a project for some other time.

You must login to leave a comment