Saturday, October 25, 2014

Retrieving source of web pages and doing a bit more using LWP::UserAgent

The following program shows how to do something more along with retrieval of any web page source, like handling of cookies stored by the web page, specifying a desired browser agent for calling the web page and posting values to a web page.
#!/usr/bin/perl

use LWP::UserAgent; # required module to retrieve the source of any web page

use HTTP::Cookies; # required module for handling cookies

my $ua = new LWP::UserAgent; # create an object of LWP::UserAgent module to access it's methods

$ua->agent('Mozilla/8.0'); # specifying a browser agent, if required

# storing cookies in the file mentioned
$ua->cookie_jar(
 HTTP::Cookies->new(
  file => 'mycookies.txt',
  autosave => 1
 )
);

# posting values to a web page and retrieving it's source
my $response = $ua->post("http://www.hisdates.com/index.html",{name => "James Dcosta",age => "36"});

# on successful retrieval of the web page source, display the same on the screen else display the error message 
if ($response->is_success) {
 print $response->content;
}
else
{
 print $response->status_line;
}


Output:
-------
On running the Perl script from command line, the html source for the webpage "http://www.hisdates.com/index.html" will be displayed on the screen.