org.apache.manifoldcf.crawler.connectors.sharepoint

Class SharePointRepository

java.lang.Object

org.apache.manifoldcf.core.connector.BaseConnector

org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

org.apache.manifoldcf.crawler.connectors.sharepoint.SharePointRepository

All Implemented Interfaces:

org.apache.manifoldcf.core.interfaces.IConnector, org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

public class SharePointRepository
extends org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

This is the "repository connector" for Microsoft SharePoint. Document identifiers for this connector come in three forms: (1) An "S" followed by the encoded subsite/library path, which represents the encoded relative path from the root site to a library. [deprecated and no longer supported]; (2) A "D" followed by a subsite/library/folder/file path, which represents the relative path from the root site to a file. [deprecated and no longer supported] (3) Six different kinds of unencoded path, each of which starts with a "/" at the beginning, where the "/" represents the root site of the connection, as follows: /sitepath/ - the relative path to a site. The path MUST both begin and end with a single "/". /sitepath/libraryname// - the relative path to a library. The path MUST begin with a single "/" and end with "//". /sitepath/libraryname//folderfilepath - the relative path to a file. The path MUST begin with a single "/" and MUST include a "//" after the library, and must NOT end with a "/". /sitepath/listname/// - the relative path to a list. The path MUST begin with a single "/" and end with "///". /sitepath/listname///rowid - the relative path to a list item. The path MUST begin with a single "/" and MUST include a "///" after the list name, and must NOT end in a "/". /sitepath/listname///rowid//attachment_filename - the relative path to a list attachment. The path MUST begin with a single "/", MUST include a "///" after the list name, and MUST include a "//" separating the rowid from the filename.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	SharePointRepository.ExecuteMethodThread
protected class	SharePointRepository.FileStream
protected class	SharePointRepository.ListItemStream
protected static class	SharePointRepository.MetadataInformation Metadata information gleaned from document paths and specification.
protected class	SharePointRepository.SystemMetadataDescription Class that tracks paths associated with id's, and the name of the metadata attribute to use for the path.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	_rcsid
static java.lang.String	ACTIVITY_FETCH
protected static java.lang.String[]	attachmentDataNames
protected static java.lang.String[]	fileStreamDataNames
protected static java.lang.String[]	listItemStreamDataNames
protected static long	sessionExpirationInterval
static java.lang.String	wsddPathProperty

Fields inherited from class org.apache.manifoldcf.core.connector.BaseConnector

currentContext, params

Fields inherited from interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

GLOBAL_DENY_TOKEN, JOBMODE_CONTINUOUS, JOBMODE_ONCEONLY, MODEL_ADD, MODEL_ADD_CHANGE, MODEL_ADD_CHANGE_DELETE, MODEL_ALL, MODEL_CHAINED_ADD, MODEL_CHAINED_ADD_CHANGE, MODEL_CHAINED_ADD_CHANGE_DELETE, MODEL_PARTIAL

Constructor Summary

Constructors

Constructor and Description
SharePointRepository() Constructor.

Method Summary

Modifier and Type	Method and Description
java.lang.String	addSeedDocuments(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities, org.apache.manifoldcf.core.interfaces.Specification spec, java.lang.String lastSeedVersion, long seedTime, int jobMode) Queue "seed" documents.
java.lang.String	check() Test the connection.
protected boolean	checkIncludeFile(java.lang.String filePath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a file should be included.
protected boolean	checkIncludeLibrary(java.lang.String libraryPath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a library should be included, given a document specification.
protected boolean	checkIncludeList(java.lang.String listPath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a list should be included, given a document specification.
protected boolean	checkIncludeListItem(java.lang.String itemPath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a list item should be included.
protected boolean	checkIncludeListItemAttachment(java.lang.String attachmentPath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a list item attachment should be included.
protected boolean	checkIncludeSite(java.lang.String sitePath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Check if a site should be included, given a document specification.
protected static boolean	checkMatch(java.lang.String sourceMatch, int sourceIndex, java.lang.String match) Check a match between two strings with wildcards.
protected static boolean	checkPartialPathMatch(java.lang.String sourceMatch, int sourceIndex, java.lang.String match, int requiredExtraPathSections) Check for a partial path match between two strings with wildcards.
void	connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters) Connect.
static java.lang.String	decodePath(java.lang.String relPath) Given a path that is /-separated, and otherwise encoded, decode properly to convert to unencoded form.
void	disconnect() Close the connection.
static java.lang.String	encodePath(java.lang.String relPath) Given a path that is /-separated, and otherwise unencoded, encode properly for an actual URI
protected void	expireSession()
protected void	fetchAndIndexFile(org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities, java.lang.String documentIdentifier, java.lang.String version, java.lang.String fileUrl, java.lang.String fetchUrl, java.lang.String[] accessTokens, java.lang.String[] denyTokens, java.util.Date createdDate, java.util.Date modifiedDate, java.util.Map<java.lang.String,java.lang.String> metadataValues, java.lang.String guid, SharePointRepository.SystemMetadataDescription sDesc) Method that fetches and indexes a file fetched from a SharePoint URL, with appropriate error handling etc.
protected static void	fillInAuthorityTypeTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
protected static void	fillInMetadataTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.Specification ds) Fill in metadata tab
protected static void	fillInPathsTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.Specification ds) Fill in paths tab
protected static void	fillInSecurityTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.Specification ds) Fill in security tab
protected static void	fillInServerTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
protected void	fillInTransientMetadataInfo(java.util.Map<java.lang.String,java.lang.Object> velocityContext, int connectionSequenceNumber) Fill in transient metadata info
protected void	fillInTransientPathsInfo(java.util.Map<java.lang.String,java.lang.Object> velocityContext, int connectionSequenceNumber) Fill in the transient portion of the Paths tab
protected static java.lang.String[]	getAcls(org.apache.manifoldcf.core.interfaces.Specification spec) Grab forced acl out of document specification.
java.lang.String[]	getActivitiesList() Return the list of activities that this connector supports (i.e.
java.lang.String[]	getBinNames(java.lang.String documentIdentifier) Get the bin name string for a document identifier.
java.util.List<NameValue>	getDocLibsBySite(java.lang.String parentSite) Gets a list of document libraries of the given parent site
protected java.lang.String[]	getInterestingFieldSetSorted(SharePointRepository.MetadataInformation metadataInfo, java.lang.String[] allFields)
java.util.Map<java.lang.String,java.lang.String>	getLibFieldList(java.lang.String parentSite, java.lang.String docLibrary) Gets a list of field names of the given document library or list.
java.util.Map<java.lang.String,java.lang.String>	getListFieldList(java.lang.String parentSite, java.lang.String listName) Gets a list of field names of the given document library or list.
java.util.List<NameValue>	getListsBySite(java.lang.String parentSite) Gets a list of lists of the given parent site
int	getMaxDocumentRequest() Get the maximum number of documents to amalgamate together into one batch, for this connector.
protected SharePointRepository.MetadataInformation	getMetadataSpecification(java.lang.String filePath, org.apache.manifoldcf.core.interfaces.Specification documentSpecification) Get a file or item's metadata specification, given a path and a document specification.
protected void	getSession() Set up a session
java.util.List<NameValue>	getSites(java.lang.String parentSite) Gets a list of sites/subsites of the given parent site
protected static void	handleIOException(java.io.IOException e, java.lang.String context)
boolean	isConnected() This method is called to assess whether to count this connector instance should actually be counted as being connected.
protected static java.lang.String	mapExtensionToMimeType(java.lang.String fileName) Map an extension to a mime type
protected static java.lang.String	mapToFileName(java.lang.String fileName) Map document identifier to file name
protected static int	matchSubPath(java.lang.String subPath, java.lang.String fullPath) Match a sub-path.
void	outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.lang.String tabName) Output the configuration body section.
void	outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters, java.util.List<java.lang.String> tabsArray) Output the configuration header section.
void	outputSpecificationBody(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber, int actualSequenceNumber, java.lang.String tabName) Output the specification body section.
void	outputSpecificationHeader(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber, java.util.List<java.lang.String> tabsArray) Output the specification header section.
protected static void	packDate(java.lang.StringBuilder sb, java.util.Date dateValue)
static java.lang.String	pathItemDecode(java.lang.String pathItem) Decode a path item.
static java.lang.String	pathItemEncode(java.lang.String pathItem) Encode a path item.
void	poll() This method is periodically called for all connectors that are connected but not in active use.
protected static boolean	processCheck(boolean caseSensitive, java.lang.String sourceMatch, int sourceIndex, java.lang.String match, int matchIndex) Recursive worker method for checkMatch.
java.lang.String	processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) Process a configuration post.
void	processDocuments(java.lang.String[] documentIdentifiers, org.apache.manifoldcf.crawler.interfaces.IExistingVersions statuses, org.apache.manifoldcf.core.interfaces.Specification spec, org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities, int jobMode, boolean usesDefaultAuthority) Process a set of documents.
protected static boolean	processPartialPathCheck(boolean caseSensitive, java.lang.String sourceMatch, int sourceIndex, java.lang.String match, int matchIndex, int requiredExtraPathSections) Recursive worker method for checkPartialPathMatch.
java.lang.String	processSpecificationPost(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber) Process a specification post.
boolean	requestInfo(org.apache.manifoldcf.core.interfaces.Configuration output, java.lang.String command) Request arbitrary connector information.
protected static void	setDataACLs(org.apache.manifoldcf.agents.interfaces.RepositoryDocument data, java.lang.String[] acls, java.lang.String[] denyAcls)
protected static void	setPathAttribute(org.apache.manifoldcf.agents.interfaces.RepositoryDocument data, SharePointRepository.SystemMetadataDescription sDesc, java.lang.String documentIdentifier)
protected static int	unpackDate(java.lang.String value, int index, java.util.Date theDate)
void	viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext, org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.ConfigParams parameters) View configuration.
void	viewSpecification(org.apache.manifoldcf.core.interfaces.IHTTPOutput out, java.util.Locale locale, org.apache.manifoldcf.core.interfaces.Specification ds, int connectionSequenceNumber) View specification.

Methods inherited from class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

getConnectorModel, getFormCheckJavascriptMethodName, getFormPresaveCheckJavascriptMethodName, getRelationshipTypes

Methods inherited from class org.apache.manifoldcf.core.connector.BaseConnector

clearThreadContext, deinstall, getConfiguration, install, outputConfigurationBody, outputConfigurationHeader, outputConfigurationHeader, pack, packFixedList, packList, packList, processConfigurationPost, setThreadContext, unpack, unpackFixedList, unpackList, viewConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.manifoldcf.core.interfaces.IConnector

clearThreadContext, deinstall, getConfiguration, install, setThreadContext

Field Detail

_rcsid

public static final java.lang.String _rcsid

See Also:

Constant Field Values

wsddPathProperty

public static final java.lang.String wsddPathProperty

See Also:

Constant Field Values

ACTIVITY_FETCH

public static final java.lang.String ACTIVITY_FETCH

See Also:

Constant Field Values

sessionExpirationInterval

protected static final long sessionExpirationInterval

See Also:

Constant Field Values

attachmentDataNames

protected static final java.lang.String[] attachmentDataNames

fileStreamDataNames

protected static final java.lang.String[] fileStreamDataNames

listItemStreamDataNames

protected static final java.lang.String[] listItemStreamDataNames

Constructor Detail

SharePointRepository

public SharePointRepository()

Constructor.

Method Detail

getSession

protected void getSession()
                   throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Set up a session

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

expireSession

protected void expireSession()
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getActivitiesList

public java.lang.String[] getActivitiesList()

Return the list of activities that this connector supports (i.e. writes into the log).

Specified by:

getActivitiesList in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getActivitiesList in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Returns:

the list.

connect

public void connect(org.apache.manifoldcf.core.interfaces.ConfigParams configParameters)

Connect.

Specified by:

connect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

connect in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

configParameters - is the set of configuration parameters, which in this case describe the root directory.

disconnect

public void disconnect()
                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Close the connection. Call this before discarding the repository connector.

Specified by:

disconnect in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

disconnect in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getBinNames

public java.lang.String[] getBinNames(java.lang.String documentIdentifier)

Get the bin name string for a document identifier. The bin name describes the queue to which the document will be assigned for throttling purposes. Throttling controls the rate at which items in a given queue are fetched; it does not say anything about the overall fetch rate, which may operate on multiple queues or bins. For example, if you implement a web crawler, a good choice of bin name would be the server name, since that is likely to correspond to a real resource that will need real throttle protection.

Specified by:

getBinNames in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getBinNames in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

documentIdentifier - is the document identifier.

Returns:

the bin name.

getMaxDocumentRequest

public int getMaxDocumentRequest()

Get the maximum number of documents to amalgamate together into one batch, for this connector.

Specified by:

getMaxDocumentRequest in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

getMaxDocumentRequest in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Returns:

the maximum number. 0 indicates "unlimited".

check

public java.lang.String check()
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Test the connection. Returns a string describing the connection integrity.

Specified by:

check in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

check in class org.apache.manifoldcf.core.connector.BaseConnector

Returns:

the connection's status as a displayable string.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

poll

public void poll()
          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

This method is periodically called for all connectors that are connected but not in active use.

Specified by:

poll in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

poll in class org.apache.manifoldcf.core.connector.BaseConnector

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

isConnected

public boolean isConnected()

This method is called to assess whether to count this connector instance should actually be counted as being connected.

Specified by:

isConnected in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

isConnected in class org.apache.manifoldcf.core.connector.BaseConnector

Returns:

true if the connector instance is actually connected.

requestInfo

public boolean requestInfo(org.apache.manifoldcf.core.interfaces.Configuration output,
                           java.lang.String command)
                    throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Request arbitrary connector information. This method is called directly from the API in order to allow API users to perform any one of several connector-specific queries.

Specified by:

requestInfo in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

requestInfo in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

output - is the response object, to be filled in by this method.

command - is the command, which is taken directly from the API request.

Returns:

true if the resource is found, false if not. In either case, output may be filled in.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

addSeedDocuments

public java.lang.String addSeedDocuments(org.apache.manifoldcf.crawler.interfaces.ISeedingActivity activities,
                                         org.apache.manifoldcf.core.interfaces.Specification spec,
                                         java.lang.String lastSeedVersion,
                                         long seedTime,
                                         int jobMode)
                                  throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                         org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Queue "seed" documents. Seed documents are the starting places for crawling activity. Documents are seeded when this method calls appropriate methods in the passed in ISeedingActivity object. This method can choose to find repository changes that happen only during the specified time interval. The seeds recorded by this method will be viewed by the framework based on what the getConnectorModel() method returns. It is not a big problem if the connector chooses to create more seeds than are strictly necessary; it is merely a question of overall work required. The end time and seeding version string passed to this method may be interpreted for greatest efficiency. For continuous crawling jobs, this method will be called once, when the job starts, and at various periodic intervals as the job executes. When a job's specification is changed, the framework automatically resets the seeding version string to null. The seeding version string may also be set to null on each job run, depending on the connector model returned by getConnectorModel(). Note that it is always ok to send MORE documents rather than less to this method. The connector will be connected before this method can be called.

Specified by:

addSeedDocuments in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

addSeedDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

activities - is the interface this method should use to perform whatever framework actions are desired.

spec - is a document specification (that comes from the job).

seedTime - is the end of the time range of documents to consider, exclusive.

lastSeedVersion - is the last seeding version string for this job, or null if the job has no previous seeding version string.

jobMode - is an integer describing how the job is being run, whether continuous or once-only.

Returns:

an updated seeding version string, to be stored with the job.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

processDocuments

public void processDocuments(java.lang.String[] documentIdentifiers,
                             org.apache.manifoldcf.crawler.interfaces.IExistingVersions statuses,
                             org.apache.manifoldcf.core.interfaces.Specification spec,
                             org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities,
                             int jobMode,
                             boolean usesDefaultAuthority)
                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                             org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Process a set of documents. This is the method that should cause each document to be fetched, processed, and the results either added to the queue of documents for the current job, and/or entered into the incremental ingestion manager. The document specification allows this class to filter what is done based on the job. The connector will be connected before this method can be called.

Specified by:

processDocuments in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

processDocuments in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

documentIdentifiers - is the set of document identifiers to process.

statuses - are the currently-stored document versions for each document in the set of document identifiers passed in above.

activities - is the interface this method should use to queue up new document references and ingest documents.

jobMode - is an integer describing how the job is being run, whether continuous or once-only.

usesDefaultAuthority - will be true only if the authority in use for these documents is the default one.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

packDate

protected static void packDate(java.lang.StringBuilder sb,
                               java.util.Date dateValue)

unpackDate

protected static int unpackDate(java.lang.String value,
                                int index,
                                java.util.Date theDate)

getInterestingFieldSetSorted

protected java.lang.String[] getInterestingFieldSetSorted(SharePointRepository.MetadataInformation metadataInfo,
                                                          java.lang.String[] allFields)

fetchAndIndexFile

protected void fetchAndIndexFile(org.apache.manifoldcf.crawler.interfaces.IProcessActivity activities,
                                 java.lang.String documentIdentifier,
                                 java.lang.String version,
                                 java.lang.String fileUrl,
                                 java.lang.String fetchUrl,
                                 java.lang.String[] accessTokens,
                                 java.lang.String[] denyTokens,
                                 java.util.Date createdDate,
                                 java.util.Date modifiedDate,
                                 java.util.Map<java.lang.String,java.lang.String> metadataValues,
                                 java.lang.String guid,
                                 SharePointRepository.SystemMetadataDescription sDesc)
                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                 org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Method that fetches and indexes a file fetched from a SharePoint URL, with appropriate error handling etc.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

handleIOException

protected static void handleIOException(java.io.IOException e,
                                        java.lang.String context)
                                 throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                        org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

mapExtensionToMimeType

protected static java.lang.String mapExtensionToMimeType(java.lang.String fileName)

Map an extension to a mime type

mapToFileName

protected static java.lang.String mapToFileName(java.lang.String fileName)

Map document identifier to file name

setDataACLs

protected static void setDataACLs(org.apache.manifoldcf.agents.interfaces.RepositoryDocument data,
                                  java.lang.String[] acls,
                                  java.lang.String[] denyAcls)

setPathAttribute

protected static void setPathAttribute(org.apache.manifoldcf.agents.interfaces.RepositoryDocument data,
                                       SharePointRepository.SystemMetadataDescription sDesc,
                                       java.lang.String documentIdentifier)
                                throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

outputConfigurationHeader

public void outputConfigurationHeader(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                      org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      java.util.Locale locale,
                                      org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                      java.util.List<java.lang.String> tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException

Output the configuration header section. This method is called in the head section of the connector's configuration page. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the configuration editing HTML.

Specified by:

outputConfigurationHeader in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationHeader in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputConfigurationBody

public void outputConfigurationBody(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                    org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    java.util.Locale locale,
                                    org.apache.manifoldcf.core.interfaces.ConfigParams parameters,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException

Output the configuration body section. This method is called in the body section of the connector's configuration page. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate <html>, <body>, and <form> tags. The name of the form is "editconnection".

Specified by:

outputConfigurationBody in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

outputConfigurationBody in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

tabName - is the current tab name.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

processConfigurationPost

public java.lang.String processConfigurationPost(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                                                 org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 java.util.Locale locale,
                                                 org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Process a configuration post. This method is called at the start of the connector's configuration page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the configuration parameters accordingly. The name of the posted form is "editconnection".

Specified by:

processConfigurationPost in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

processConfigurationPost in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

variableContext - is the set of variables available from the post, including binary file post information.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Returns:

null if all is well, or a string error message if there is an error that should prevent saving of the connection (and cause a redirection to an error page).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

viewConfiguration

public void viewConfiguration(org.apache.manifoldcf.core.interfaces.IThreadContext threadContext,
                              org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              java.util.Locale locale,
                              org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException

View configuration. This method is called in the body section of the connector's view configuration page. Its purpose is to present the connection information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate <html> and <body>tags.

Specified by:

viewConfiguration in interface org.apache.manifoldcf.core.interfaces.IConnector

Overrides:

viewConfiguration in class org.apache.manifoldcf.core.connector.BaseConnector

Parameters:

threadContext - is the local thread context.

out - is the output to which any HTML should be sent.

parameters - are the configuration parameters, as they currently exist, for this connection being configured.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

fillInAuthorityTypeTab

protected static void fillInAuthorityTypeTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                             org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                             org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                                      throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

fillInServerTab

protected static void fillInServerTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                      org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      org.apache.manifoldcf.core.interfaces.ConfigParams parameters)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

outputSpecificationHeader

public void outputSpecificationHeader(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                      java.util.Locale locale,
                                      org.apache.manifoldcf.core.interfaces.Specification ds,
                                      int connectionSequenceNumber,
                                      java.util.List<java.lang.String> tabsArray)
                               throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                      java.io.IOException

Output the specification header section. This method is called in the head section of a job page which has selected a repository connection of the current type. Its purpose is to add the required tabs to the list, and to output any javascript methods that might be needed by the job editing HTML. The connector will be connected before this method can be called.

Specified by:

outputSpecificationHeader in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

outputSpecificationHeader in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

locale - is the locale the output is preferred to be in.

ds - is the current document specification for this job.

connectionSequenceNumber - is the unique number of this connection within the job.

tabsArray - is an array of tab names. Add to this array any tab names that are specific to the connector.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

outputSpecificationBody

public void outputSpecificationBody(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                    java.util.Locale locale,
                                    org.apache.manifoldcf.core.interfaces.Specification ds,
                                    int connectionSequenceNumber,
                                    int actualSequenceNumber,
                                    java.lang.String tabName)
                             throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                    java.io.IOException

Output the specification body section. This method is called in the body section of a job page which has selected a repository connection of the current type. Its purpose is to present the required form elements for editing. The coder can presume that the HTML that is output from this configuration will be within appropriate <html>, <body>, and <form> tags. The name of the form is always "editjob". The connector will be connected before this method can be called.

Specified by:

outputSpecificationBody in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

outputSpecificationBody in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

locale - is the locale the output is preferred to be in.

ds - is the current document specification for this job.

connectionSequenceNumber - is the unique number of this connection within the job.

actualSequenceNumber - is the connection within the job that has currently been selected.

tabName - is the current tab name. (actualSequenceNumber, tabName) form a unique tuple within the job.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

fillInMetadataTab

protected static void fillInMetadataTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                        org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                        org.apache.manifoldcf.core.interfaces.Specification ds)

Fill in metadata tab

fillInTransientMetadataInfo

protected void fillInTransientMetadataInfo(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                           int connectionSequenceNumber)

Fill in transient metadata info

fillInPathsTab

protected static void fillInPathsTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                     org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                     org.apache.manifoldcf.core.interfaces.Specification ds)

Fill in paths tab

fillInTransientPathsInfo

protected void fillInTransientPathsInfo(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                        int connectionSequenceNumber)

Fill in the transient portion of the Paths tab

fillInSecurityTab

protected static void fillInSecurityTab(java.util.Map<java.lang.String,java.lang.Object> velocityContext,
                                        org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                                        org.apache.manifoldcf.core.interfaces.Specification ds)

Fill in security tab

processSpecificationPost

public java.lang.String processSpecificationPost(org.apache.manifoldcf.core.interfaces.IPostParameters variableContext,
                                                 java.util.Locale locale,
                                                 org.apache.manifoldcf.core.interfaces.Specification ds,
                                                 int connectionSequenceNumber)
                                          throws org.apache.manifoldcf.core.interfaces.ManifoldCFException

Process a specification post. This method is called at the start of job's edit or view page, whenever there is a possibility that form data for a connection has been posted. Its purpose is to gather form information and modify the document specification accordingly. The name of the posted form is always "editjob". The connector will be connected before this method can be called.

Specified by:

processSpecificationPost in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

processSpecificationPost in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

variableContext - contains the post data, including binary file-upload information.

locale - is the locale the output is preferred to be in.

ds - is the current document specification for this job.

connectionSequenceNumber - is the unique number of this connection within the job.

Returns:

null if all is well, or a string error message if there is an error that should prevent saving of the job (and cause a redirection to an error page).

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

viewSpecification

public void viewSpecification(org.apache.manifoldcf.core.interfaces.IHTTPOutput out,
                              java.util.Locale locale,
                              org.apache.manifoldcf.core.interfaces.Specification ds,
                              int connectionSequenceNumber)
                       throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                              java.io.IOException

View specification. This method is called in the body section of a job's view page. Its purpose is to present the document specification information to the user. The coder can presume that the HTML that is output from this configuration will be within appropriate <html> and <body>tags. The connector will be connected before this method can be called.

Specified by:

viewSpecification in interface org.apache.manifoldcf.crawler.interfaces.IRepositoryConnector

Overrides:

viewSpecification in class org.apache.manifoldcf.crawler.connectors.BaseRepositoryConnector

Parameters:

out - is the output to which any HTML should be sent.

locale - is the locale the output is preferred to be in.

ds - is the current document specification for this job.

connectionSequenceNumber - is the unique number of this connection within the job.

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

java.io.IOException

getLibFieldList

public java.util.Map<java.lang.String,java.lang.String> getLibFieldList(java.lang.String parentSite,
                                                                        java.lang.String docLibrary)
                                                                 throws org.apache.manifoldcf.agents.interfaces.ServiceInterruption,
                                                                        org.apache.manifoldcf.core.interfaces.ManifoldCFException

Gets a list of field names of the given document library or list.

Parameters:

parentSite - - parent site path

docLibrary - name

Returns:

list of the fields

Throws:

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getListFieldList

public java.util.Map<java.lang.String,java.lang.String> getListFieldList(java.lang.String parentSite,
                                                                         java.lang.String listName)
                                                                  throws org.apache.manifoldcf.agents.interfaces.ServiceInterruption,
                                                                         org.apache.manifoldcf.core.interfaces.ManifoldCFException

Gets a list of field names of the given document library or list.

Parameters:

parentSite - - parent site path

listName - name

Returns:

list of the fields

Throws:

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getSites

public java.util.List<NameValue> getSites(java.lang.String parentSite)
                                   throws org.apache.manifoldcf.agents.interfaces.ServiceInterruption,
                                          org.apache.manifoldcf.core.interfaces.ManifoldCFException

Gets a list of sites/subsites of the given parent site

Parameters:

parentSite - the unencoded parent site path to search for subsites, empty for root.

Returns:

list of the sites

Throws:

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

org.apache.manifoldcf.core.interfaces.ManifoldCFException

getDocLibsBySite

public java.util.List<NameValue> getDocLibsBySite(java.lang.String parentSite)
                                           throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                                  org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Gets a list of document libraries of the given parent site

Parameters:

parentSite - the unencoded parent site to search for libraries, empty for root.

Returns:

list of the libraries

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

getListsBySite

public java.util.List<NameValue> getListsBySite(java.lang.String parentSite)
                                         throws org.apache.manifoldcf.core.interfaces.ManifoldCFException,
                                                org.apache.manifoldcf.agents.interfaces.ServiceInterruption

Gets a list of lists of the given parent site

Parameters:

parentSite - the unencoded parent site to search for lists, empty for root.

Returns:

list of the lists

Throws:

org.apache.manifoldcf.core.interfaces.ManifoldCFException

org.apache.manifoldcf.agents.interfaces.ServiceInterruption

checkIncludeLibrary

protected boolean checkIncludeLibrary(java.lang.String libraryPath,
                                      org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a library should be included, given a document specification.

Parameters:

libraryPath - is the unencoded canonical library name (including site path from root site), without any starting slash.

documentSpecification - is the specification.

Returns:

true if it should be included.

checkIncludeList

protected boolean checkIncludeList(java.lang.String listPath,
                                   org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a list should be included, given a document specification.

Parameters:

listPath - is the unencoded canonical list name (including site path from root site), without any starting slash.

documentSpecification - is the specification.

Returns:

true if it should be included.

checkIncludeSite

protected boolean checkIncludeSite(java.lang.String sitePath,
                                   org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a site should be included, given a document specification.

Parameters:

sitePath - is the unencoded canonical site path name from the root site level, without any starting slash.

documentSpecification - is the specification.

Returns:

true if it should be included.

getMetadataSpecification

protected SharePointRepository.MetadataInformation getMetadataSpecification(java.lang.String filePath,
                                                                            org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Get a file or item's metadata specification, given a path and a document specification.

Parameters:

filePath - is the unencoded path to a file or item, including sites and library/list, beneath the root site.

documentSpecification - is the document specification.

Returns:

the metadata description appropriate to the file.

checkIncludeFile

protected boolean checkIncludeFile(java.lang.String filePath,
                                   org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a file should be included.

Parameters:

filePath - is the path to the file, including sites and library, beneath the root site.

documentSpecification - is the document specification.

Returns:

true if file should be included.

checkIncludeListItemAttachment

protected boolean checkIncludeListItemAttachment(java.lang.String attachmentPath,
                                                 org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a list item attachment should be included.

Parameters:

attachmentPath - is the path to the attachment, including sites and list name, beneath the root site.

documentSpecification - is the document specification.

Returns:

true if file should be included.

checkIncludeListItem

protected boolean checkIncludeListItem(java.lang.String itemPath,
                                       org.apache.manifoldcf.core.interfaces.Specification documentSpecification)

Check if a list item should be included.

Parameters:

itemPath - is the path to the item, including sites and list name, beneath the root site.

documentSpecification - is the document specification.

Returns:

true if file should be included.

matchSubPath

protected static int matchSubPath(java.lang.String subPath,
                                  java.lang.String fullPath)

Match a sub-path. The sub-path must match the complete starting part of the full path, in a path sense. The returned value should point into the file name beyond the end of the matched path, or be -1 if there is no match.

Parameters:

subPath - is the sub path.

fullPath - is the full path.

Returns:

the index of the start of the remaining part of the full path, or -1.

checkPartialPathMatch

protected static boolean checkPartialPathMatch(java.lang.String sourceMatch,
                                               int sourceIndex,
                                               java.lang.String match,
                                               int requiredExtraPathSections)

Check for a partial path match between two strings with wildcards. Match allowance also must be made for the minimum path components in the rest of the path.

processPartialPathCheck

protected static boolean processPartialPathCheck(boolean caseSensitive,
                                                 java.lang.String sourceMatch,
                                                 int sourceIndex,
                                                 java.lang.String match,
                                                 int matchIndex,
                                                 int requiredExtraPathSections)

Recursive worker method for checkPartialPathMatch. Returns 'true' if there is a path that consumes the source string entirely, and leaves the remainder of the match string able to match the required followup.

Parameters:

caseSensitive - is true if file names are case sensitive.

sourceMatch - is the source string (w/o wildcards)

sourceIndex - is the current point in the source string.

match - is the match string (w/wildcards)

matchIndex - is the current point in the match string.

Returns:

true if there is a match.

checkMatch

protected static boolean checkMatch(java.lang.String sourceMatch,
                                    int sourceIndex,
                                    java.lang.String match)

Check a match between two strings with wildcards.

Parameters:

sourceMatch - is the expanded string (no wildcards)

sourceIndex - is the starting point in the expanded string.

match - is the wildcard-based string.

Returns:

true if there is a match.

processCheck

protected static boolean processCheck(boolean caseSensitive,
                                      java.lang.String sourceMatch,
                                      int sourceIndex,
                                      java.lang.String match,
                                      int matchIndex)

Recursive worker method for checkMatch. Returns 'true' if there is a path that consumes both strings in their entirety in a matched way.

Parameters:

caseSensitive - is true if file names are case sensitive.

sourceMatch - is the source string (w/o wildcards)

sourceIndex - is the current point in the source string.

match - is the match string (w/wildcards)

matchIndex - is the current point in the match string.

Returns:

true if there is a match.

getAcls

protected static java.lang.String[] getAcls(org.apache.manifoldcf.core.interfaces.Specification spec)

Grab forced acl out of document specification.

Parameters:

spec - is the document specification.

Returns:

the acls.

pathItemDecode

public static java.lang.String pathItemDecode(java.lang.String pathItem)

Decode a path item.

pathItemEncode

public static java.lang.String pathItemEncode(java.lang.String pathItem)

Encode a path item.

decodePath

public static java.lang.String decodePath(java.lang.String relPath)

Given a path that is /-separated, and otherwise encoded, decode properly to convert to unencoded form.

encodePath

public static java.lang.String encodePath(java.lang.String relPath)

Given a path that is /-separated, and otherwise unencoded, encode properly for an actual URI