org.apache.manifoldcf.crawler.connectors.rss

Interface IThrottledConnection

All Known Implementing Classes:

ThrottledFetcher.ThrottledConnection

public interface IThrottledConnection

This interface represents an established connection to a URL.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	_rcsid
static int	FETCH_BAD_URI
static int	FETCH_CIRCULAR_REDIRECT
static int	FETCH_IO_ERROR
static int	FETCH_NOT_TRIED
static int	FETCH_SEQUENCE_ERROR
static int	FETCH_UNKNOWN_ERROR
static int	STATUS_NOCHANGE Status code for fetch: No change.
static int	STATUS_OK Status code for fetch: OK
static int	STATUS_PAGEERROR Status code for fetch: Static error; retries won't help, individual page access failed
static int	STATUS_SITEERROR Status code for fetch: Static error; retries won't help, overall access to site in question

Method Summary

Modifier and Type	Method and Description
void	beginFetch(java.lang.String fetchType) Begin the fetch process.
void	close() Close the connection.
void	doneFetch(org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities) Done with the fetch.
int	executeFetch(java.lang.String protocol, int port, java.lang.String urlPath, java.lang.String userAgent, java.lang.String from, java.lang.String lastETag, java.lang.String lastModified) Execute the fetch and get the return code.
java.io.InputStream	getResponseBodyStream() Get the response input stream.
int	getResponseCode() Get the http response code.
java.lang.String	getResponseHeader(java.lang.String headerName) Get a specified response header, if it exists.

Field Detail

_rcsid

static final java.lang.String _rcsid

See Also:

Constant Field Values

STATUS_OK

static final int STATUS_OK

Status code for fetch: OK

See Also:

Constant Field Values

STATUS_SITEERROR

static final int STATUS_SITEERROR

Status code for fetch: Static error; retries won't help, overall access to site in question

See Also:

Constant Field Values

STATUS_PAGEERROR

static final int STATUS_PAGEERROR

Status code for fetch: Static error; retries won't help, individual page access failed

See Also:

Constant Field Values

STATUS_NOCHANGE

static final int STATUS_NOCHANGE

Status code for fetch: No change.

See Also:

Constant Field Values

FETCH_NOT_TRIED

static final int FETCH_NOT_TRIED

See Also:

Constant Field Values

FETCH_CIRCULAR_REDIRECT

static final int FETCH_CIRCULAR_REDIRECT

See Also:

Constant Field Values

FETCH_BAD_URI

static final int FETCH_BAD_URI

See Also:

Constant Field Values

FETCH_SEQUENCE_ERROR

static final int FETCH_SEQUENCE_ERROR

See Also:

Constant Field Values

FETCH_IO_ERROR

static final int FETCH_IO_ERROR

See Also:

Constant Field Values

FETCH_UNKNOWN_ERROR

static final int FETCH_UNKNOWN_ERROR

See Also:

Constant Field Values

Method Detail

beginFetch

void beginFetch(java.lang.String fetchType)
         throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Begin the fetch process.

Parameters:

fetchType - is a short descriptive string describing the kind of fetch being requested. This is used solely for logging purposes.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

executeFetch

int executeFetch(java.lang.String protocol,
                 int port,
                 java.lang.String urlPath,
                 java.lang.String userAgent,
                 java.lang.String from,
                 java.lang.String lastETag,
                 java.lang.String lastModified)
          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                 org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Execute the fetch and get the return code. This method uses the standard logging mechanism to keep track of the fetch attempt. It also signals the following three conditions: ServiceInterruption (if a dynamic error occurs), OK, or a static error code (for a condition where retry is not likely to be helpful). The actual HTTP error code is NOT returned by this method.

Parameters:

protocol - is the protocol to use to perform the access, e.g. "http"

port - is the port to use to perform the access, where -1 means "use the default"

urlPath - is the path part of the url, e.g. "/robots.txt"

userAgent - is the value of the userAgent header to use.

from - is the value of the from header to use.

Returns:

the status code: success, static error, or dynamic error.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

getResponseCode

int getResponseCode()
             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                    org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Get the http response code.

Returns:

the response code. This is either an HTTP response code, or one of the codes above.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

getResponseBodyStream

java.io.InputStream getResponseBodyStream()
                                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                          org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Get the response input stream. It is the responsibility of the caller to close this stream when done.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

getResponseHeader

java.lang.String getResponseHeader(java.lang.String headerName)
                            throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                   org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Get a specified response header, if it exists.

Parameters:

headerName - is the name of the header.

Returns:

the header value, or null if it doesn't exist.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

doneFetch

void doneFetch(org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities)
        throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Done with the fetch. Call this when the fetch has been completed. A log entry will be generated describing what was done.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

close

void close()
    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Close the connection. Call this to end this server connection.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException