TagParseState

java.lang.Object
- org.apache.manifoldcf.connectorcommon.fuzzyml.CharacterReceiver
- - org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
  - - org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState

Direct Known Subclasses:

HTMLParseState, XMLFuzzyParseState, XMLParseState
```
public class TagParseState
extends SingleCharacterReceiver
```
This class represents a basic xml/html tag parser. It is capable of recognizing the following xml and html constructs:
```
 '<' <token> <attrs> '>' ... '</' <token> '>'
 '<' <token> <attrs> '/>'
 '<?' <token> <attrs>  '?>'
 '<![' [<token>] '[' ... ']]>'
 '<!' <token> ... '>'
 ''
 
```
Each of these, save the comment, has supporting protected methods that will be called by the parsing engine. Overriding these methods will allow an extending class to perform higher-level data extraction and parsing. Of these, the messiest is the <! ... > construct, since there can be multiple nested btags, cdata-like escapes, and qtags inside. Ideally the parser should produce a sequence of preparsed tokens from these tags. Since they can be nested, keeping track of the depth is also essential, so we do that with a btag depth counter. Thus, in this case, it is not the state that matters, but the btag depth, to determine if the parser is operating inside a btag.

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.lang.StringBuilder`	`accumBuffer` This is the only buffer we actually accumulate stuff in.
`protected java.lang.StringBuilder`	`ampBuffer` Buffer of characters seen after ampersand.
`protected int`	`bTagDepth` The btag depth, which indicates btag behavior when > 0.
`protected java.util.List<AttrNameValue>`	`currentAttrList`
`protected java.lang.String`	`currentAttrName`
`protected java.lang.StringBuilder`	`currentAttrNameBuffer`
`protected int`	`currentState`
`protected java.lang.String`	`currentTagName`
`protected java.lang.StringBuilder`	`currentTagNameBuffer`
`protected java.lang.StringBuilder`	`currentValueBuffer`
`protected boolean`	`inAmpersand` Whether we've seen an ampersand
`protected static java.util.Map<java.lang.String,java.lang.String>`	`mapLookup`
`protected static int`	`TAGPARSESTATE_IN_ATTR_LOOKING_FOR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_ATTR_NAME`
`protected static int`	`TAGPARSESTATE_IN_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_BANG_TOKEN`
`protected static int`	`TAGPARSESTATE_IN_BRACKET_TOKEN`
`protected static int`	`TAGPARSESTATE_IN_CDATA_BODY`
`protected static int`	`TAGPARSESTATE_IN_COMMENT`
`protected static int`	`TAGPARSESTATE_IN_DOUBLE_QUOTES_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_END_TAG_NAME`
`protected static int`	`TAGPARSESTATE_IN_QTAG_ATTR_LOOKING_FOR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_QTAG_ATTR_NAME`
`protected static int`	`TAGPARSESTATE_IN_QTAG_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_QTAG_DOUBLE_QUOTES_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_QTAG_NAME`
`protected static int`	`TAGPARSESTATE_IN_QTAG_SAW_QUESTION`
`protected static int`	`TAGPARSESTATE_IN_QTAG_SINGLE_QUOTES_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_QTAG_UNQUOTED_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_SINGLE_QUOTES_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_TAG_NAME`
`protected static int`	`TAGPARSESTATE_IN_TAG_SAW_SLASH`
`protected static int`	`TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE`
`protected static int`	`TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE_SAW_SLASH`
`protected static int`	`TAGPARSESTATE_NEED_FINAL_BRACKET`
`protected static int`	`TAGPARSESTATE_NORMAL`
`protected static int`	`TAGPARSESTATE_SAWCOMMENTDASH`
`protected static int`	`TAGPARSESTATE_SAWDASH`
`protected static int`	`TAGPARSESTATE_SAWEXCLAMATION`
`protected static int`	`TAGPARSESTATE_SAWLEFTANGLE`
`protected static int`	`TAGPARSESTATE_SAWRIGHTBRACKET`
`protected static int`	`TAGPARSESTATE_SAWSECONDCOMMENTDASH`
`protected static int`	`TAGPARSESTATE_SAWSECONDRIGHTBRACKET`

Fields inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
charBuffer

Constructor Summary

Constructors
Constructor and Description

TagParseState()

Constructors
Constructor and Description
`TagParseState()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected boolean`	`acceptNewTag()` Allow parsing within tag.
`protected static java.lang.String`	`attributeDecode(java.lang.String input)` Decode an html attribute
`boolean`	`dealWithCharacter(char thisChar)` Deal with a character.
`protected boolean`	`dumpValues(java.lang.String value)`
`protected static boolean`	`isPunctuation(char x)` Is a character markup language punctuation?
`protected static boolean`	`isWhitespace(char x)` Is a character markup language whitespace?
`protected static java.lang.String`	`mapChunk(java.lang.String input)` Map an entity reference back to a character
`protected java.lang.StringBuilder`	`newBuffer()` Allocate the buffer.
`protected boolean`	`noteBTag(java.lang.String tagName)` This method is called for every <! <token> ...
`protected boolean`	`noteBTagToken(java.lang.String token)` This method gets called for every token inside a btag.
`protected boolean`	`noteEndBTag()` This method is called for the end of every btag, or any time there's a naked '>' in the document.
`protected boolean`	`noteEndEscaped()` Called for the end of every cdata-like tag.
`protected boolean`	`noteEndTag(java.lang.String tagName)` This method gets called for every end tag.
`protected boolean`	`noteEscaped(java.lang.String token)` Called for the start of every cdata-like tag, e.g.
`protected boolean`	`noteEscapedCharacter(char thisChar)` This method gets called for every character that is found within an escape block, e.g.
`protected boolean`	`noteNormalCharacter(char thisChar)` This method gets called for every character that is not part of a tag etc.
`protected boolean`	`noteQTag(java.lang.String tagName, java.util.List<AttrNameValue> attributes)` This method is called for every <? ...
`protected boolean`	`noteTag(java.lang.String tagName, java.util.List<AttrNameValue> attributes)` This method gets called for every tag.
`protected boolean`	`outputAmpBuffer()` Interpret ampersand buffer.

Methods inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
dealWithCharacters, dealWithRemainder

Methods inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.CharacterReceiver
finishUp

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - TAGPARSESTATE_NORMAL
```
protected static final int TAGPARSESTATE_NORMAL
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWLEFTANGLE
```
protected static final int TAGPARSESTATE_SAWLEFTANGLE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWEXCLAMATION
```
protected static final int TAGPARSESTATE_SAWEXCLAMATION
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWDASH
```
protected static final int TAGPARSESTATE_SAWDASH
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_COMMENT
```
protected static final int TAGPARSESTATE_IN_COMMENT
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWCOMMENTDASH
```
protected static final int TAGPARSESTATE_SAWCOMMENTDASH
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWSECONDCOMMENTDASH
```
protected static final int TAGPARSESTATE_SAWSECONDCOMMENTDASH
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_TAG_NAME
```
protected static final int TAGPARSESTATE_IN_TAG_NAME
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_ATTR_NAME
```
protected static final int TAGPARSESTATE_IN_ATTR_NAME
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_TAG_SAW_SLASH
```
protected static final int TAGPARSESTATE_IN_TAG_SAW_SLASH
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_END_TAG_NAME
```
protected static final int TAGPARSESTATE_IN_END_TAG_NAME
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_ATTR_LOOKING_FOR_VALUE
```
protected static final int TAGPARSESTATE_IN_ATTR_LOOKING_FOR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_SINGLE_QUOTES_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_SINGLE_QUOTES_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_DOUBLE_QUOTES_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_DOUBLE_QUOTES_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_NAME
```
protected static final int TAGPARSESTATE_IN_QTAG_NAME
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_ATTR_NAME
```
protected static final int TAGPARSESTATE_IN_QTAG_ATTR_NAME
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_SAW_QUESTION
```
protected static final int TAGPARSESTATE_IN_QTAG_SAW_QUESTION
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_QTAG_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_ATTR_LOOKING_FOR_VALUE
```
protected static final int TAGPARSESTATE_IN_QTAG_ATTR_LOOKING_FOR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_SINGLE_QUOTES_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_QTAG_SINGLE_QUOTES_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_DOUBLE_QUOTES_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_QTAG_DOUBLE_QUOTES_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_QTAG_UNQUOTED_ATTR_VALUE
```
protected static final int TAGPARSESTATE_IN_QTAG_UNQUOTED_ATTR_VALUE
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_BRACKET_TOKEN
```
protected static final int TAGPARSESTATE_IN_BRACKET_TOKEN
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_NEED_FINAL_BRACKET
```
protected static final int TAGPARSESTATE_NEED_FINAL_BRACKET
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_BANG_TOKEN
```
protected static final int TAGPARSESTATE_IN_BANG_TOKEN
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_CDATA_BODY
```
protected static final int TAGPARSESTATE_IN_CDATA_BODY
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWRIGHTBRACKET
```
protected static final int TAGPARSESTATE_SAWRIGHTBRACKET
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_SAWSECONDRIGHTBRACKET
```
protected static final int TAGPARSESTATE_SAWSECONDRIGHTBRACKET
```
    See Also:
    
    Constant Field Values
  - TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE_SAW_SLASH
```
protected static final int TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE_SAW_SLASH
```
    See Also:
    
    Constant Field Values
  - currentState
```
protected int currentState
```
  - bTagDepth
```
protected int bTagDepth
```
    The btag depth, which indicates btag behavior when > 0.
  - accumBuffer
```
protected java.lang.StringBuilder accumBuffer
```
    This is the only buffer we actually accumulate stuff in.
  - currentTagNameBuffer
```
protected java.lang.StringBuilder currentTagNameBuffer
```
  - currentAttrNameBuffer
```
protected java.lang.StringBuilder currentAttrNameBuffer
```
  - currentValueBuffer
```
protected java.lang.StringBuilder currentValueBuffer
```
  - currentTagName
```
protected java.lang.String currentTagName
```
  - currentAttrName
```
protected java.lang.String currentAttrName
```
  - currentAttrList
```
protected java.util.List<AttrNameValue> currentAttrList
```
  - inAmpersand
```
protected boolean inAmpersand
```
    Whether we've seen an ampersand
  - ampBuffer
```
protected java.lang.StringBuilder ampBuffer
```
    Buffer of characters seen after ampersand.
  - mapLookup
```
protected static final java.util.Map<java.lang.String,java.lang.String> mapLookup
```
- Constructor Detail
  - TagParseState
```
public TagParseState()
```
- Method Detail
  - dealWithCharacter
```
public boolean dealWithCharacter(char thisChar)
                          throws ManifoldCFException
```
    Deal with a character. No exceptions are allowed, since those would represent syntax errors, and we don't want those to cause difficulty.
    
    Specified by:
    
    dealWithCharacter in class SingleCharacterReceiver
    
    Returns:
    
    true if done.
    
    Throws:
    
    ManifoldCFException
  - acceptNewTag
```
protected boolean acceptNewTag()
```
    Allow parsing within tag.
  - newBuffer
```
protected java.lang.StringBuilder newBuffer()
```
    Allocate the buffer.
  - outputAmpBuffer
```
protected boolean outputAmpBuffer()
                           throws ManifoldCFException
```
    Interpret ampersand buffer.
    
    Throws:
    
    ManifoldCFException
  - dumpValues
```
protected boolean dumpValues(java.lang.String value)
                      throws ManifoldCFException
```
    Throws:
    
    ManifoldCFException
  - noteTag
```
protected boolean noteTag(java.lang.String tagName,
                          java.util.List<AttrNameValue> attributes)
                   throws ManifoldCFException
```
    This method gets called for every tag. Override this method to intercept tag begins.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteEndTag
```
protected boolean noteEndTag(java.lang.String tagName)
                      throws ManifoldCFException
```
    This method gets called for every end tag. Override this method to intercept tag ends.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteQTag
```
protected boolean noteQTag(java.lang.String tagName,
                           java.util.List<AttrNameValue> attributes)
                    throws ManifoldCFException
```
    This method is called for every <? ... ?> construct, or 'qtag'. Override it to intercept such constructs.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteBTag
```
protected boolean noteBTag(java.lang.String tagName)
                    throws ManifoldCFException
```
    This method is called for every <! <token> ... > construct, or 'btag'. Override it to intercept these.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteEndBTag
```
protected boolean noteEndBTag()
                       throws ManifoldCFException
```
    This method is called for the end of every btag, or any time there's a naked '>' in the document. Override it if you want to intercept these.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteEscaped
```
protected boolean noteEscaped(java.lang.String token)
                       throws ManifoldCFException
```
    Called for the start of every cdata-like tag, e.g. <![ <token> [ ... ]]>
    
    Parameters:
    
    token - may be empty!!!
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteEndEscaped
```
protected boolean noteEndEscaped()
                          throws ManifoldCFException
```
    Called for the end of every cdata-like tag.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteBTagToken
```
protected boolean noteBTagToken(java.lang.String token)
                         throws ManifoldCFException
```
    This method gets called for every token inside a btag.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteNormalCharacter
```
protected boolean noteNormalCharacter(char thisChar)
                               throws ManifoldCFException
```
    This method gets called for every character that is not part of a tag etc. Override this method to intercept such characters.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - noteEscapedCharacter
```
protected boolean noteEscapedCharacter(char thisChar)
                                throws ManifoldCFException
```
    This method gets called for every character that is found within an escape block, e.g. CDATA. Override this method to intercept such characters.
    
    Returns:
    
    true to halt further processing.
    
    Throws:
    
    ManifoldCFException
  - attributeDecode
```
protected static java.lang.String attributeDecode(java.lang.String input)
```
    Decode an html attribute
  - mapChunk
```
protected static java.lang.String mapChunk(java.lang.String input)
```
    Map an entity reference back to a character
  - isWhitespace
```
protected static boolean isWhitespace(char x)
```
    Is a character markup language whitespace?
  - isPunctuation
```
protected static boolean isPunctuation(char x)
```
    Is a character markup language punctuation?

Class TagParseState

Field Summary

Fields inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver

Constructor Summary

Method Summary

Methods inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver

Methods inherited from class org.apache.manifoldcf.connectorcommon.fuzzyml.CharacterReceiver

Methods inherited from class java.lang.Object

Field Detail

TAGPARSESTATE_NORMAL

TAGPARSESTATE_SAWLEFTANGLE

TAGPARSESTATE_SAWEXCLAMATION

TAGPARSESTATE_SAWDASH

TAGPARSESTATE_IN_COMMENT

TAGPARSESTATE_SAWCOMMENTDASH

TAGPARSESTATE_SAWSECONDCOMMENTDASH

TAGPARSESTATE_IN_TAG_NAME

TAGPARSESTATE_IN_ATTR_NAME

TAGPARSESTATE_IN_ATTR_VALUE

TAGPARSESTATE_IN_TAG_SAW_SLASH

TAGPARSESTATE_IN_END_TAG_NAME

TAGPARSESTATE_IN_ATTR_LOOKING_FOR_VALUE

TAGPARSESTATE_IN_SINGLE_QUOTES_ATTR_VALUE

TAGPARSESTATE_IN_DOUBLE_QUOTES_ATTR_VALUE

TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE

TAGPARSESTATE_IN_QTAG_NAME

TAGPARSESTATE_IN_QTAG_ATTR_NAME

TAGPARSESTATE_IN_QTAG_SAW_QUESTION

TAGPARSESTATE_IN_QTAG_ATTR_VALUE

TAGPARSESTATE_IN_QTAG_ATTR_LOOKING_FOR_VALUE

TAGPARSESTATE_IN_QTAG_SINGLE_QUOTES_ATTR_VALUE

TAGPARSESTATE_IN_QTAG_DOUBLE_QUOTES_ATTR_VALUE

TAGPARSESTATE_IN_QTAG_UNQUOTED_ATTR_VALUE

TAGPARSESTATE_IN_BRACKET_TOKEN

TAGPARSESTATE_NEED_FINAL_BRACKET

TAGPARSESTATE_IN_BANG_TOKEN

TAGPARSESTATE_IN_CDATA_BODY

TAGPARSESTATE_SAWRIGHTBRACKET

TAGPARSESTATE_SAWSECONDRIGHTBRACKET

TAGPARSESTATE_IN_UNQUOTED_ATTR_VALUE_SAW_SLASH

currentState

bTagDepth

accumBuffer

currentTagNameBuffer

currentAttrNameBuffer

currentValueBuffer

currentTagName

currentAttrName

currentAttrList

inAmpersand

ampBuffer

mapLookup

Constructor Detail

TagParseState

Method Detail

dealWithCharacter

acceptNewTag

newBuffer

outputAmpBuffer

dumpValues

noteTag

noteEndTag

noteQTag

noteBTag

noteEndBTag

noteEscaped

noteEndEscaped

noteBTagToken

noteNormalCharacter

noteEscapedCharacter

attributeDecode

mapChunk

isWhitespace

isPunctuation