org.apache.manifoldcf.agents.output.amazoncloudsearch

Class AmazonCloudSearchConnector

java.lang.Object

org.apache.manifoldcf.core.connector.BaseConnector

org.apache.manifoldcf.agents.output.BaseOutputConnector

org.apache.manifoldcf.agents.output.amazoncloudsearch.AmazonCloudSearchConnector

All Implemented Interfaces:

org.apache.manifoldcf.agents.interfaces.IOutputConnector, org.apache.manifoldcf.agents.interfaces.IPipelineConnector, org.apache.manifoldcf.core.interfaces.IConnector

public class AmazonCloudSearchConnector
extends org.apache.manifoldcf.agents.output.BaseOutputConnector

Field Summary

Fields

Modifier and Type	Field and Description
protected static int	CHUNK_SIZE
static java.lang.String	INGEST_ACTIVITY Ingestion activity
protected org.apache.http.client.methods.HttpPost	poster Local connection
static java.lang.String	REMOVE_ACTIVITY Document removal activity
protected java.lang.String	serverHost
protected java.lang.String	serverPath

Fields inherited from class org.apache.manifoldcf.agents.output.BaseOutputConnector

_rcsid

Fields inherited from class org.apache.manifoldcf.core.connector.BaseConnector

currentContext, params

Fields inherited from interface org.apache.manifoldcf.agents.interfaces.IPipelineConnector

DOCUMENTSTATUS_ACCEPTED, DOCUMENTSTATUS_REJECTED

Constructor Summary

Constructors

Constructor and Description
AmazonCloudSearchConnector() Constructor.

Method Summary

Modifier and Type	Method and Description
int	addOrReplaceDocumentWithException(java.lang.String documentURI, org.apache.manifoldcf.core.interfaces.VersionContext outputDescription, org.apache.manifoldcf.agents.interfaces.RepositoryDocument document, java.lang.String authorityNameString, org.apache.manifoldcf.agents.interfaces.IOutputAddActivity activities) Add (or replace) a document in the output data store using the connector.
java.lang.String	check() Test the connection.
boolean	checkMimeTypeIndexable(org.apache.manifoldcf.core.interfaces.VersionContext outputDescription, java.lang.String mimeType, org.apache.manifoldcf.agents.interfaces.IOutputCheckActivity activities) Detect if a mime type is indexable or not.
void	clearThreadContext() Clear out any state information specific to a given thread.
protected void	conditionallyFlushDocuments(org.apache.manifoldcf.agents.interfaces.IOutputHistoryActivity activities)
void	connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters) Connect.
void	deinstall(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
void	disconnect() Close the connection.
protected void	flushDocuments(org.apache.manifoldcf.agents.interfaces.IOutputHistoryActivity activities)
java.lang.String[]	getActivitiesList() Return the list of activities that this connector supports (i.e.
protected void	getSession() Set up a session
void	install(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
boolean	isConnected() This method is called to assess whether to count this connector instance should actually be counted as being connected.
void	noteJobComplete(org.apache.manifoldcf.agents.interfaces.IOutputNotifyActivity activities)
void	outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.lang.String tabName)
void	outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.util.List<java.lang.String> tabsArray) Output the configuration header section.
java.lang.String	processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) Process a configuration post.
void	removeDocument(java.lang.String documentURI, java.lang.String outputDescription, org.apache.manifoldcf.agents.interfaces.IOutputRemoveActivity activities) Remove a document using the connector.
void	viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) View configuration.

Methods inherited from class org.apache.manifoldcf.agents.output.BaseOutputConnector

checkDateIndexable, checkDocumentIndexable, checkLengthIndexable, checkURLIndexable, getFormCheckJavascriptMethodName, getFormPresaveCheckJavascriptMethodName, getPipelineDescription, noteAllRecordsRemoved, outputSpecificationBody, outputSpecificationHeader, processSpecificationPost, requestInfo, viewSpecification

Methods inherited from class org.apache.manifoldcf.core.connector.BaseConnector

getConfiguration, outputConfigurationBody, outputConfigurationHeader, outputConfigurationHeader, pack, packFixedList, packList, packList, poll, processConfigurationPost, setThreadContext, unpack, unpackFixedList, unpackList, viewConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.manifoldcf.core.interfaces.IConnector

getConfiguration, poll, processConfigurationPost, setThreadContext

Field Detail

INGEST_ACTIVITY

public static final java.lang.String INGEST_ACTIVITY

Ingestion activity

See Also:

Constant Field Values

REMOVE_ACTIVITY

public static final java.lang.String REMOVE_ACTIVITY

Document removal activity

See Also:

Constant Field Values

poster

protected org.apache.http.client.methods.HttpPost poster

Local connection

serverHost

protected java.lang.String serverHost

serverPath

protected java.lang.String serverPath

CHUNK_SIZE

protected static final int CHUNK_SIZE

See Also:

Constant Field Values

Constructor Detail

AmazonCloudSearchConnector

public AmazonCloudSearchConnector()

Constructor.

Method Detail

clearThreadContext

public void clearThreadContext()

Clear out any state information specific to a given thread. This method is called when this object is returned to the connection pool.

Specified by:

clearThreadContext in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

clearThreadContext in class org.apache.manifoldcf.core.connector.BaseConnector

install

public void install(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Specified by:

install in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

install in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

deinstall

public void deinstall(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext)
               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Specified by:

deinstall in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

deinstall in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getActivitiesList

public java.lang.String[] getActivitiesList()

Return the list of activities that this connector supports (i.e. writes into the log).

Specified by:

getActivitiesList in interface org.apache.manifoldcf.agents.interfaces.IOutputConnector

Overrides:

getActivitiesList in class org.apache.manifoldcf.agents.output.BaseOutputConnector

Returns:

the list.

connect

public void connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters)

Connect.

Specified by:

connect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

connect in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

configParameters - is the set of configuration parameters, which in this case describe the target appliance, basic auth configuration, etc. (This formerly came out of the ini file.)

isConnected

public boolean isConnected()

This method is called to assess whether to count this connector instance should actually be counted as being connected.

Specified by:

isConnected in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

isConnected in class org.apache.manifoldcf.core.connector.BaseConnector

Returns:

true if the connector instance is actually connected.

disconnect

public void disconnect()
                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Close the connection. Call this before discarding the connection.

Specified by:

disconnect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

disconnect in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getSession

protected void getSession()
                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Set up a session

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

check

public java.lang.String check()
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Test the connection. Returns a string describing the connection integrity.

Specified by:

check in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

check in class org.apache.manifoldcf.core.connector.BaseConnector

Returns:

the connection's status as a displayable string.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

checkMimeTypeIndexable

public boolean checkMimeTypeIndexable(org.apache.manifoldcf.core.interfaces.VersionContext outputDescription,
                                      java.lang.String mimeType,
                                      org.apache.manifoldcf.agents.interfaces.IOutputCheckActivity activities)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Detect if a mime type is indexable or not. This method is used by participating repository connectors to pre-filter the number of unusable documents that will be passed to this output connector.

Specified by:

checkMimeTypeIndexable in interface org.apache.manifoldcf.agents.interfaces.IPipelineConnector

Overrides:

checkMimeTypeIndexable in class org.apache.manifoldcf.agents.output.BaseOutputConnector

Parameters:

outputDescription - is the document's output version.

mimeType - is the mime type of the document.

Returns:

true if the mime type is indexable by this connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

addOrReplaceDocumentWithException

public int addOrReplaceDocumentWithException(java.lang.String documentURI,
                                             org.apache.manifoldcf.core.interfaces.VersionContext outputDescription,
                                             org.apache.manifoldcf.agents.interfaces.RepositoryDocument document,
                                             java.lang.String authorityNameString,
                                             org.apache.manifoldcf.agents.interfaces.IOutputAddActivity activities)
                                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                             org.apache.manifoldcf.agents.interfaces.ServiceInterruption,
                                             java.io.IOException

Add (or replace) a document in the output data store using the connector. This method presumes that the connector object has been configured, and it is thus able to communicate with the output data store should that be necessary. The OutputSpecification is *not* provided to this method, because the goal is consistency, and if output is done it must be consistent with the output description, since that was what was partly used to determine if output should be taking place. So it may be necessary for this method to decode an output description string in order to determine what should be done.

Specified by:

addOrReplaceDocumentWithException in interface org.apache.manifoldcf.agents.interfaces.IPipelineConnector

Overrides:

addOrReplaceDocumentWithException in class org.apache.manifoldcf.agents.output.BaseOutputConnector

Parameters:

documentURI - is the URI of the document. The URI is presumed to be the unique identifier which the output data store will use to process and serve the document. This URI is constructed by the repository connector which fetches the document, and is thus universal across all output connectors.

outputDescription - is the description string that was constructed for this document by the getOutputDescription() method.

document - is the document data to be processed (handed to the output data store).

authorityNameString - is the name of the authority responsible for authorizing any access tokens passed in with the repository document. May be null.

activities - is the handle to an object that the implementer of an output connector may use to perform operations, such as logging processing activity.

Returns:

the document status (accepted or permanently rejected).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

java.io.IOException

removeDocument

public void removeDocument(java.lang.String documentURI,
                           java.lang.String outputDescription,
                           org.apache.manifoldcf.agents.interfaces.IOutputRemoveActivity activities)
                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                           org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Remove a document using the connector. Note that the last outputDescription is included, since it may be necessary for the connector to use such information to know how to properly remove the document.

Specified by:

removeDocument in interface org.apache.manifoldcf.agents.interfaces.IOutputConnector

Overrides:

removeDocument in class org.apache.manifoldcf.agents.output.BaseOutputConnector

Parameters:

documentURI - is the URI of the document. The URI is presumed to be the unique identifier which the output data store will use to process and serve the document. This URI is constructed by the repository connector which fetches the document, and is thus universal across all output connectors.

outputDescription - is the last description string that was constructed for this document by the getOutputDescription() method above.

activities - is the handle to an object that the implementer of an output connector may use to perform operations, such as logging processing activity.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

noteJobComplete

public void noteJobComplete(org.apache.manifoldcf.agents.interfaces.IOutputNotifyActivity activities)
                     throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                            org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Specified by:

noteJobComplete in interface org.apache.manifoldcf.agents.interfaces.IOutputConnector

Overrides:

noteJobComplete in class org.apache.manifoldcf.agents.output.BaseOutputConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

conditionallyFlushDocuments

protected void conditionallyFlushDocuments(org.apache.manifoldcf.agents.interfaces.IOutputHistoryActivity activities)
                                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                           org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

flushDocuments

protected void flushDocuments(org.apache.manifoldcf.agents.interfaces.IOutputHistoryActivity activities)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

viewConfiguration

public void viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                              org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              java.util.Locale locale,
                              org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException

View configuration. This method is called in the body section of the connector's view configuration page. Its purpose is to present the connection information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate <html> and <body> tags.

Specified by:

viewConfiguration in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

viewConfiguration in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputConfigurationHeader

public void outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                      org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      java.util.Locale locale,
                                      org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                      java.util.List<java.lang.String> tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException

Output the configuration header section. This method is called in the head section of the connector's configuration page. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the configuration editing HTML.

Specified by:

outputConfigurationHeader in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationHeader in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputConfigurationBody

public void outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                    org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    java.util.Locale locale,
                                    org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException

Specified by:

outputConfigurationBody in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationBody in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

processConfigurationPost

public java.lang.String processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                                 org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Process a configuration post. This method is called at the start of the connector's configuration page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the configuration parameters accordingly. The name of the posted form is "editconnection".

Overrides:

processConfigurationPost in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

variableContext - is the set of variables available from the post, including binary file post information.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Returns:

null if all is well, or a string error message if there is an error that should prevent saving of the connection (and cause a redirection to an error page).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException