General Web Scraping Question

Tech-Archive recommends: Repair Windows Errors & Optimize Windows Performance

From: Selden McCabe (seldenm_at_msn.com)
Date: 02/22/04 

Date: Sat, 21 Feb 2004 17:10:12 -0800

I've been working on a web scraping program, and have the basics down.

But I don't understand the parameters.
Normally, you go to a URL (say a reverse yellow pages directory), and enter
some parameters (like area code, phone number, etc.) and POST this back to
the web. Then you parse the response, looking for the data you need.

Ofen I see examples where the data you post contains something like
"AreaCode=503&Number=5551212&x=1&y=2"

Where do the "x=1 and y=2" come from? I have some sites where my post
doesn't work. In one case, you are supposed to enter a contractor's license
number, and then click a button, and the result contains information about
the license. After I post what I think should work, the result coming back
is the same web page, with the contractor's number filled in.

Do the X and Y parameters involve invoking a button? How do you determine
what to use for the parameters?

Thanks in advance for any advice or pointers!
   ---Selden McCabe

Relevant Pages

  • Re: Ann Coulter Comments on VA Tech
    ... I agree that people who are allowed to carry ought to have to ... To get a concealed handgun license that would be good anywhere in the ... You get any of this wrong, no license, the process stops. ... necessary basics. ...
    (rec.music.makers.guitar.acoustic)
  • Re: Another License Idea
    ... > 1) Three classes of license: ... Some bands may be split by mode only. ... Main objective is to keep Basics out of trouble. ... Intermediates can be ...
    (rec.radio.amateur.policy)
  • Re: The Mac is Dead! Long Live the Wintel PC! [was Re: Apple ahead of schedule]
    ... Also, while we're at it, Microsoft also wrote Apple's ... first BASIC language interpreter, as well as their own, and IBM's. ... Later on, Digital Rights didn't want to license it, so they cloned it ... There were better basics out there at the time. ...
    (comp.sys.mac.advocacy)
  • Re: The Mac is Dead! Long Live the Wintel PC! [was Re: Apple ahead of schedule]
    ... Also, while we're at it, Microsoft also wrote Apple's ... first BASIC language interpreter, as well as their own, and IBM's. ... Later on, Digital Rights didn't want to license it, so they cloned it ... There were better basics out there at the time. ...
    (comp.sys.mac.advocacy)
  • Re: someone tell me the basics please
    ... So the principles needed to get your license, ... I'm looking at your question more as "How did the radio community fail ... on the basics, which I initially thought should be part of a basic ... those answers and do a web search on the specific subject to gain a ...
    (rec.radio.cb)