Question

HI, im trying to get the host from a url.

sub scrape {
my @m_error_array;
my @m_href_array;
my @href_array;
my ( $self, $DBhost, $DBuser, $DBpass, $DBname ) = @_;
my ($dbh, $query, $result, $array);
my $DNS = "dbi:mysql:$DBname:$DBhost:3306";
$dbh = DBI->connect($DNS, $DBuser, $DBpass ) or die $DBI::errstr;
if( defined( $self->{_process_image} ) && ( -e 'href_w_' . $self->{_process_image} . ".txt" ) ) {
    open  ERROR_W, "error_w_" . $self->{_process_image} . ".txt";
    open  M_HREF_W, "m_href_w_" . $self->{_process_image} . ".txt";
    open  HREF_W, "href_w_" . $self->{_process_image} . ".txt";
    @m_error_array = ( split( '|||', <ERROR_W> ) );
    @m_href_array = ( split( '|||', <M_HREF_W> ) );
    @href_array = ( split( '|||', <HREF_W> ) );
    close ( ERROR_W );
    close ( M_HREF_W );
    close ( HREF_W );
}else{
    @href_array = ( $self->{_url} );
}
my $z = 0;
while( @href_array ){
    if( defined( $self->{_x_more} ) && $z == $self->{_x_more} ) {
        last;
    }
    if( defined( $self->{_process_image} ) ) {
        $self->write( 'm_href_w', @m_href_array );
        $self->write( 'href_w', @href_array );
        $self->write( 'error_w', @m_error_array );
    }
    $self->{_link_count} = scalar @m_href_array;
    my $href = shift( @href_array );
    my $info = URI->new($href);
    my $host = $info->host;
    $host =~ s/^www\.//;
    $result = $dbh->prepare("INSERT INTO `". $host ."` (URL) VALUES ('$href')");
    if( ! $result->execute() ){
        $result = $dbh->prepare("CREATE TABLE `" . $host . "` ( `ID` INT( 255 ) NOT NULL AUTO_INCREMENT , `URL` VARCHAR( 255 ) NOT NULL , PRIMARY KEY ( `ID` )) ENGINE = MYISAM ;");
        $result->execute()
    }
    $self->{_current_page} = $href;
    my $response = $ua->get($href);
    my $responseCode = $response->code;
    print $responseCode;
}

}

Towards te end the line my $host = $info->host; is throwing Can't locate object method "host" via package "URI::_generic"

Can anyone explain this?

Regards,

Phil

Answer 1

URI->new creates instances of a subclass of URI, depending on the scheme of the url you give it. Those subclasses might be URI::http, URI::file, URI::mailto, or something completely different. If URI doesn’t have a specialized subclass for the kind of url you gave it, it’ll create an instance of URI::_generic.

Each of those URI subclasses have different methods. URI::http happens to have a host method, but most others don’t. You’re calling ->host on something that isn’t a URI::http or similar, and therefore doesn’t have a host method.

You probably expected all the strings you pass to URI->new to be http urls. That doesn’t seem to be the case, so you might want to check your data. Otherwise, if you do want to handle non-http urls, you should make sure a method actually exists for that instance before calling it, for example by using ->can or ->isa.