WikipediaScanner (Bliki (Core) 3.1.1-SNAPSHOT-20170407 API)

java.lang.Object
- info.bliki.wiki.filter.WikipediaScanner

Direct Known Subclasses:: AbstractParser

public class WikipediaScanner
extends Object

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`EOF`
`protected int`	`fScannerPosition`
`protected char[]`	`fSource` The corresponding `char[]` array for the string source
`protected String`	`fStringSource` The `String` of the given raw wiki text
`protected IWikiModel`	`fWikiModel`

Constructor Summary

Constructors
Constructor and Description

WikipediaScanner(String src)

WikipediaScanner(String src, int position)

Constructors
Constructor and Description
`WikipediaScanner(String src)`
`WikipediaScanner(String src, int position)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static int`	`findNestedEnd(char[] sourceArray, char startCh, char endChar, int startPosition)` Read until the end of a nested block i.e.
`static int`	`findNestedEndSingle(char[] sourceArray, char startCh, char endChar, int startPosition)` Read until the end of a nested block i.e.
`static int[]`	`findNestedParamEnd(char[] sourceArray, int startPosition)` Find the end of a template parameter declaration or the end of a template declaration.
`static int`	`findNestedTemplateEnd(char[] sourceArray, int startPosition)`
`int`	`getPosition()`
`int`	`indexEndOfComment()` Get the offset position behind the next closing HTML comment tag (-->).
`int`	`indexEndOfNowiki()` Get the offset position behind the next </nowiki> tag.
`int`	`indexEndOfTable()` Get the offset position behind the corresponding wiki table closing tag (i.e.
`protected int`	`indexOfUntilNoLetter(char testChar, int fromIndex)` Read the characters until no more letters are found or the given `testChar` is found.
`protected WikiTagNode`	`makeTag(int start, int end, ArrayList<NodeAttribute> attributes)` Create a tag node based on the current cursor and the one provided.
`int`	`nextNewline()`
`int`	`nextNewlineCell(WPCell cell)`
`protected List<NodeAttribute>`	`parseAttributes(int start, int end)`
`protected WikiTagNode`	`parseTag(int start)` Parse a tag.
`protected int`	`readSpecialWikiTags(int start)`
`protected int`	`readUntilIgnoreCase(int start, String startString, String endString)` Read the characters until the concatenated start and end substring is found.
`void`	`setModel(IWikiModel wikiModel)`
`void`	`setPosition(int newPos)`
`protected static List<String>`	`splitByChar(char splitChar, char[] srcArray, int currOffset, int endOffset, List<String> resultList, int maxParts)` Split the given `srcArray` character array by the given character.
`static List<String>`	`splitByChar(char splitChar, String sourceString, List<String> resultList, int maxParts)` Split the given src string by pipe symbol (i.e.
`static List<String>`	`splitByPipe(char[] srcArray, int currOffset, int endOffset, List<String> resultList)` Split the given `srcArray` character array by pipe symbol (i.e.
`static List<String>`	`splitByPipe(String sourceString, List<String> resultList)` Split the given src string by pipe symbol (i.e.
`static boolean`	`startsWith(String str, int toffset, String prefix, boolean ignoreCase)` Check if a String starts with a specified prefix (optionally case insensitive).
`WPList`	`wpList()`
`WPTable`	`wpTable(ITableOfContent tableOfContentTag)` Scan a wikipedia table.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - EOF
```
public static final int EOF
```
    See Also:
    
    Constant Field Values
  - fScannerPosition
```
protected int fScannerPosition
```
  - fWikiModel
```
protected IWikiModel fWikiModel
```
  - fStringSource
```
protected final String fStringSource
```
    The String of the given raw wiki text
  - fSource
```
protected final char[] fSource
```
    The corresponding char[] array for the string source
- Constructor Detail
  - WikipediaScanner
```
public WikipediaScanner(String src)
```
  - WikipediaScanner
```
public WikipediaScanner(String src,
                        int position)
```
- Method Detail
  - setModel
```
public void setModel(IWikiModel wikiModel)
```
  - getPosition
```
public int getPosition()
```
  - setPosition
```
public void setPosition(int newPos)
```
  - wpTable
```
public WPTable wpTable(ITableOfContent tableOfContentTag)
```
    Scan a wikipedia table. See: Help - Table
    
    Parameters:
    
    tableOfContentTag -
    
    Returns:
    
    null if no wiki table was found
  - wpList
```
public WPList wpList()
```
  - nextNewline
```
public int nextNewline()
```
  - nextNewlineCell
```
public int nextNewlineCell(WPCell cell)
```
  - indexEndOfComment
```
public int indexEndOfComment()
```
    Get the offset position behind the next closing HTML comment tag (-->).
    
    Returns:
    
    the offset position behind the next closing HTML comment tag or -1 if no tag could be found.
  - indexEndOfNowiki
```
public int indexEndOfNowiki()
```
    Get the offset position behind the next </nowiki> tag.
    
    Returns:
    
    the offset position behind the </nowiki> tag or -1 if no tag could be found.
  - indexEndOfTable
```
public int indexEndOfTable()
```
    Get the offset position behind the corresponding wiki table closing tag (i.e. |}). The scanner detects HTML comment tags, <nowiki> tags and nested wiki table tags (i.e. {|... {|... ...|} ...|}).
    
    Returns:
    
    the offset position behind the corresponding wiki table closing tag or -1 if no corresponding tag could be found.
  - startsWith
```
public static boolean startsWith(String str,
                                 int toffset,
                                 String prefix,
                                 boolean ignoreCase)
```
    Check if a String starts with a specified prefix (optionally case insensitive).
    
    Parameters:
    
    str - the String to check, may be null
    
    toffset - the starting offset of the subregion the String to check
    
    prefix - the prefix to find, may be null
    
    ignoreCase - inidicates whether the compare should ignore case (case insensitive) or not.
    
    Returns:
    
    true if the String starts with the prefix or both null
    
    See Also:
    
    String.startsWith(String)
  - splitByPipe
```
public static List<String> splitByPipe(String sourceString,
                                       List<String> resultList)
```
    Split the given src string by pipe symbol (i.e. "|")
    
    Parameters:
    
    sourceString -
    
    resultList - the list which contains the splitted strings
    
    Returns:
    
    splitted strings
  - splitByPipe
```
public static List<String> splitByPipe(char[] srcArray,
                                       int currOffset,
                                       int endOffset,
                                       List<String> resultList)
```
    Split the given srcArray character array by pipe symbol (i.e. "|").
    
    Parameters:
    
    srcArray - the array to split
    
    currOffset - start position in srcArray
    
    endOffset - end position in srcArray
    
    resultList - the list which contains the splitted strings
    
    Returns:
    
    splitted strings
  - splitByChar
```
public static List<String> splitByChar(char splitChar,
                                       String sourceString,
                                       List<String> resultList,
                                       int maxParts)
```
    Split the given src string by pipe symbol (i.e. "|")
    
    Parameters:
    
    splitChar - the character to split by
    
    sourceString - the string to split
    
    resultList - the list which contains the splitted strings
    
    maxParts - max number of parts to split the source into (less than 0 for infinite number of parts, otherwise only values greater than 0 allowed!)
    
    Returns:
    
    splitted strings
  - splitByChar
```
protected static List<String> splitByChar(char splitChar,
                                          char[] srcArray,
                                          int currOffset,
                                          int endOffset,
                                          List<String> resultList,
                                          int maxParts)
```
    Split the given srcArray character array by the given character.
    
    Parameters:
    
    splitChar - the character to split by
    
    srcArray - the array to split
    
    currOffset - start position in srcArray
    
    endOffset - end position in srcArray
    
    resultList - the list which contains the splitted strings
    
    maxParts - max number of parts to split the source into (less than 0 for infinite number of parts, otherwise only values greater than 0 allowed!)
    
    Returns:
    
    splitted strings
  - findNestedEnd
```
public static int findNestedEnd(char[] sourceArray,
                                char startCh,
                                char endChar,
                                int startPosition)
```
    Read until the end of a nested block i.e. something like [[...[[ ]]...]]
    
    Parameters:
    
    sourceArray -
    
    startCh -
    
    endChar -
    
    startPosition -
    
    Returns:
    
    the position of the nested end charcters or -1 if not found
  - findNestedEndSingle
```
public static int findNestedEndSingle(char[] sourceArray,
                                      char startCh,
                                      char endChar,
                                      int startPosition)
```
    Read until the end of a nested block i.e. something like {{{...{...{{ }}...}...}}}
    
    Parameters:
    
    sourceArray -
    
    startCh -
    
    endChar -
    
    startPosition -
    
    Returns:
    
    the position of the nested end charcters or -1 if not found
  - findNestedTemplateEnd
```
public static int findNestedTemplateEnd(char[] sourceArray,
                                        int startPosition)
```
    Returns:
    
    the template end position or -1 if there is no end. TODO: This logic needs to be improved, there are very likely cases where this does not work as intended. Template parameters can take {} as well, e.g. {{foo:bar|{|}|{}|baz}}.
  - findNestedParamEnd
```
public static int[] findNestedParamEnd(char[] sourceArray,
                                       int startPosition)
```
    Find the end of a template parameter declaration or the end of a template declaration.
    
    Parameters:
    
    sourceArray -
    
    startPosition -
    
    Returns:
    
    an array of two integers. If array[0] > 0 the scanner has found the end position of a template parameter declaration. If array[1] > 0 the scanner has found the end position of a template declaration.
  - parseTag
```
protected WikiTagNode parseTag(int start)
```
    Parse a tag. Parse the name and attributes from a start tag.
    From the HTML 4.01 Specification, W3C Recommendation 24 December 1999 https://www.w3.org/TR/html4/intro/sgmltut.html#h-3.2.2
    3.2.2 Attributes
    Elements may have associated properties, called attributes, which may have values (by default, or set by authors or scripts). Attribute/value pairs appear before the final ">" of an element's start tag. Any number of (legal) attribute value pairs, separated by spaces, may appear in an element's start tag. They may appear in any order.
    In this example, the id attribute is set for an H1 element: <H1 id="section1"> This is an identified heading thanks to the id attribute </H1> By default, SGML requires that all attribute values be delimited using either double quotation marks (ASCII decimal 34) or single quotation marks (ASCII decimal 39). Single quote marks can be included within the attribute value when the value is delimited by double quote marks, and vice versa. Authors may also use numeric character references to represent double quotes (") and single quotes ('). For doublequotes authors can also use the character entity reference ".
    In certain cases, authors may specify the value of an attribute without any quotation marks. The attribute value may only contain letters (a-z and A-Z), digits (0-9), hyphens (ASCII decimal 45), periods (ASCII decimal 46), underscores (ASCII decimal 95), and colons (ASCII decimal 58). We recommend using quotation marks even when it is possible to eliminate them.
    Attribute names are always case-insensitive.
    Attribute values are generally case-insensitive. The definition of each attribute in the reference manual indicates whether its value is case-insensitive.
    All the attributes defined by this specification are listed in the attribute index.
    
    This method uses a state machine with the following states:
    1. state 0 - outside of any attribute
    2. state 1 - within attributre name
    3. state 2 - equals hit
    4. state 3 - within naked attribute value.
    5. state 4 - within single quoted attribute value
    6. state 5 - within double quoted attribute value
    7. state 6 - whitespaces after attribute name could lead to state 2 (=)or state 0
    The starting point for the various components is stored in an array of integers that match the initiation point for the states one-for-one, i.e. bookmarks[0] is where state 0 began, bookmarks[1] is where state 1 began, etc. Attributes are stored in a Vector having one slot for each whitespace or attribute/value pair. The first slot is for attribute name (kind of like a standalone attribute).
    Parameters:
    
    start - The position at which to start scanning.
    
    Returns:
    
    The parsed tag.
    
    Throws:
    
    ParserException - If a problem occurs reading from the source.
  - parseAttributes
```
protected List<NodeAttribute> parseAttributes(int start,
                                              int end)
```
  - makeTag
```
protected WikiTagNode makeTag(int start,
                              int end,
                              ArrayList<NodeAttribute> attributes)
```
    Create a tag node based on the current cursor and the one provided.
    
    Parameters:
    
    start - The starting point of the node.
    
    end - The ending point of the node.
    
    attributes - The attributes parsed from the tag.
    
    Returns:
    
    The new Tag node.
    
    Throws:
    
    ParserException - If the nodefactory creation of the tag node fails.
  - readSpecialWikiTags
```
protected int readSpecialWikiTags(int start)
```
  - readUntilIgnoreCase
```
protected final int readUntilIgnoreCase(int start,
                                        String startString,
                                        String endString)
```
    Read the characters until the concatenated start and end substring is found. The end substring is matched ignoring case considerations.
    
    Parameters:
    
    startString - the start string which should be searched in exact case mode
    
    endString - the end string which should be searched in ignore case mode
    
    Returns:
  - indexOfUntilNoLetter
```
protected int indexOfUntilNoLetter(char testChar,
                                   int fromIndex)
```
    Read the characters until no more letters are found or the given testChar is found. If testChar was found, return the offset position.
    
    Parameters:
    
    testCh - the test character
    
    fromIndex - read from this offset
    
    Returns:
    
    -1 if the character could not be found or no more letter character were found.

Class WikipediaScanner

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

EOF

fScannerPosition

fWikiModel

fStringSource

fSource

Constructor Detail

WikipediaScanner

WikipediaScanner

Method Detail

setModel

getPosition

setPosition

wpTable

wpList

nextNewline

nextNewlineCell

indexEndOfComment

indexEndOfNowiki

indexEndOfTable

startsWith

splitByPipe

splitByPipe

splitByChar

splitByChar

findNestedEnd

findNestedEndSingle

findNestedTemplateEnd

findNestedParamEnd

parseTag

parseAttributes

makeTag

readSpecialWikiTags

readUntilIgnoreCase

indexOfUntilNoLetter