org.antlr.v4.tool

Class GrammarParserInterpreter

java.lang.Object

org.antlr.v4.runtime.Recognizer<Token,ParserATNSimulator>

org.antlr.v4.runtime.Parser

org.antlr.v4.runtime.ParserInterpreter

org.antlr.v4.tool.GrammarParserInterpreter

public class GrammarParserInterpreter
extends ParserInterpreter

A heavier weight ParserInterpreter that creates parse trees that track alternative numbers for subtree roots.

Since:

4.5.1

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GrammarParserInterpreter.BailButConsumeErrorStrategy We want to stop and track the first error but we cannot bail out like BailErrorStrategy as consume() constructs trees.

Nested classes/interfaces inherited from class org.antlr.v4.runtime.Parser

Parser.TraceListener, Parser.TrimToSizeListener

Field Summary

Fields

Modifier and Type	Field and Description
protected BitSet	decisionStatesThatSetOuterAltNumInContext
protected Grammar	g The grammar associated with this interpreter.
protected int[][]	stateToAltsMap Cache LeftRecursiveRule.getPrimaryAlts() and LeftRecursiveRule.getRecursiveOpAlts() for states in decisionStatesThatSetOuterAltNumInContext.

Fields inherited from class org.antlr.v4.runtime.ParserInterpreter

_parentContextStack, atn, decisionToDFA, grammarFileName, overrideDecision, overrideDecisionAlt, overrideDecisionInputIndex, overrideDecisionReached, overrideDecisionRoot, rootContext, ruleNames, sharedContextCache, tokenNames

Fields inherited from class org.antlr.v4.runtime.Parser

_buildParseTrees, _ctx, _errHandler, _input, _parseListeners, _precedenceStack, _syntaxErrors, matchedEOF

Fields inherited from class org.antlr.v4.runtime.Recognizer

_interp, EOF

Constructor Summary

Constructors

Constructor and Description
GrammarParserInterpreter(Grammar g, ATN atn, TokenStream input)
GrammarParserInterpreter(Grammar g, String grammarFileName, Vocabulary vocabulary, Collection<String> ruleNames, ATN atn, TokenStream input)

Method Summary

Methods

Modifier and Type	Method and Description
protected InterpreterRuleContext	createInterpreterRuleContext(ParserRuleContext parent, int invokingStateNumber, int ruleIndex)
static ParserInterpreter	deriveTempParserInterpreter(Grammar g, Parser originalParser, TokenStream tokens) Derive a new parser from an old one that has knowledge of the grammar.
BitSet	findOuterMostDecisionStates() identify the ATN states where we need to set the outer alt number.
static List<ParserRuleContext>	getAllPossibleParseTrees(Grammar g, Parser originalParser, TokenStream tokens, int decision, BitSet alts, int startIndex, int stopIndex, int startRuleIndex) Given an ambiguous parse information, return the list of ambiguous parse trees.
static List<ParserRuleContext>	getLookaheadParseTrees(Grammar g, ParserInterpreter originalParser, TokenStream tokens, int startRuleIndex, int decision, int startIndex, int stopIndex) Return a list of parse trees, one for each alternative in a decision given the same input.
void	reset()
protected int	visitDecisionState(DecisionState p) Override this method so that we can record which alternative was taken at each decision point.

Methods inherited from class org.antlr.v4.runtime.ParserInterpreter

addDecisionOverride, enterRecursionRule, getATN, getATNState, getGrammarFileName, getOverrideDecisionRoot, getRootContext, getRuleNames, getTokenNames, getVocabulary, parse, recover, recoverInline, visitRuleStopState, visitState

Methods inherited from class org.antlr.v4.runtime.Parser

addContextToParseTree, addParseListener, compileParseTreePattern, compileParseTreePattern, consume, dumpDFA, enterOuterAlt, enterRecursionRule, enterRule, exitRule, getATNWithBypassAlts, getBuildParseTree, getContext, getCurrentToken, getDFAStrings, getErrorHandler, getExpectedTokens, getExpectedTokensWithinCurrentRule, getInputStream, getInvokingContext, getNumberOfSyntaxErrors, getParseInfo, getParseListeners, getPrecedence, getRuleContext, getRuleIndex, getRuleInvocationStack, getRuleInvocationStack, getSourceName, getTokenFactory, getTokenStream, getTrimParseTree, inContext, isExpectedToken, isMatchedEOF, isTrace, match, matchWildcard, notifyErrorListeners, notifyErrorListeners, precpred, pushNewRecursionContext, removeParseListener, removeParseListeners, setBuildParseTree, setContext, setErrorHandler, setInputStream, setProfile, setTokenFactory, setTokenStream, setTrace, setTrimParseTree, triggerEnterRuleEvent, triggerExitRuleEvent, unrollRecursionContexts

Methods inherited from class org.antlr.v4.runtime.Recognizer

action, addErrorListener, getErrorHeader, getErrorListenerDispatch, getErrorListeners, getInterpreter, getRuleIndexMap, getSerializedATN, getState, getTokenErrorDisplay, getTokenType, getTokenTypeMap, removeErrorListener, removeErrorListeners, sempred, setInterpreter, setState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

g

protected final Grammar g

The grammar associated with this interpreter. Unlike the ParserInterpreter from the standard distribution, this can reference Grammar, which is in the tools area not purely runtime.

decisionStatesThatSetOuterAltNumInContext

protected BitSet decisionStatesThatSetOuterAltNumInContext

stateToAltsMap

protected int[][] stateToAltsMap

Cache LeftRecursiveRule.getPrimaryAlts() and LeftRecursiveRule.getRecursiveOpAlts() for states in decisionStatesThatSetOuterAltNumInContext. It only caches decisions in left-recursive rules.

Constructor Detail

GrammarParserInterpreter

public GrammarParserInterpreter(Grammar g,
                        String grammarFileName,
                        Vocabulary vocabulary,
                        Collection<String> ruleNames,
                        ATN atn,
                        TokenStream input)

GrammarParserInterpreter

public GrammarParserInterpreter(Grammar g,
                        ATN atn,
                        TokenStream input)

Method Detail

createInterpreterRuleContext

protected InterpreterRuleContext createInterpreterRuleContext(ParserRuleContext parent,
                                                  int invokingStateNumber,
                                                  int ruleIndex)

Overrides:

createInterpreterRuleContext in class ParserInterpreter

reset

public void reset()

Overrides:

reset in class ParserInterpreter

findOuterMostDecisionStates

public BitSet findOuterMostDecisionStates()

identify the ATN states where we need to set the outer alt number. For regular rules, that's the block at the target to rule start state. For left-recursive rules, we track the primary block, which looks just like a regular rule's outer block, and the star loop block (always there even if 1 alt).

visitDecisionState

protected int visitDecisionState(DecisionState p)

Override this method so that we can record which alternative was taken at each decision point. For non-left recursive rules, it's simple. Set decisionStatesThatSetOuterAltNumInContext indicates which decision states should set the outer alternative number.

Left recursive rules are much more complicated to deal with: there is typically a decision for the primary alternatives and a decision to choose between the recursive operator alternatives. For example, the following left recursive rule has two primary and 2 recursive alternatives.

e : e '*' e | '-' INT | e '+' e | ID ;

ANTLR rewrites that rule to be

e[int precedence] : ('-' INT | ID) ( {...}? '*' e[5] | {...}? '+' e[3] )* ;

So, there are two decisions associated with picking the outermost alt. This complicates our tracking significantly. The outermost alternative number is a function of the decision (ATN state) within a left recursive rule and the predicted alternative coming back from adaptivePredict(). We use stateToAltsMap as a cache to avoid expensive calls to getRecursiveOpAlts().

Overrides:

visitDecisionState in class ParserInterpreter

getAllPossibleParseTrees

public static List<ParserRuleContext> getAllPossibleParseTrees(Grammar g,
                                               Parser originalParser,
                                               TokenStream tokens,
                                               int decision,
                                               BitSet alts,
                                               int startIndex,
                                               int stopIndex,
                                               int startRuleIndex)
                                                        throws RecognitionException

Given an ambiguous parse information, return the list of ambiguous parse trees. An ambiguity occurs when a specific token sequence can be recognized in more than one way by the grammar. These ambiguities are detected only at decision points. The list of trees includes the actual interpretation (that for the minimum alternative number) and all ambiguous alternatives. The actual interpretation is always first. This method reuses the same physical input token stream used to detect the ambiguity by the original parser in the first place. This method resets/seeks within but does not alter originalParser. The trees are rooted at the node whose start..stop token indices include the start and stop indices of this ambiguity event. That is, the trees returned will always include the complete ambiguous subphrase identified by the ambiguity event. The subtrees returned will also always contain the node associated with the overridden decision. Be aware that this method does NOT notify error or parse listeners as it would trigger duplicate or otherwise unwanted events. This uses a temporary ParserATNSimulator and a ParserInterpreter so we don't mess up any statistics, event lists, etc... The parse tree constructed while identifying/making ambiguityInfo is not affected by this method as it creates a new parser interp to get the ambiguous interpretations. Nodes in the returned ambig trees are independent of the original parse tree (constructed while identifying/creating ambiguityInfo).

Parameters:

g - From which grammar should we drive alternative numbers and alternative labels.

originalParser - The parser used to create ambiguityInfo; it is not modified by this routine and can be either a generated or interpreted parser. It's token stream *is* reset/seek()'d.

tokens - A stream of tokens to use with the temporary parser. This will often be just the token stream within the original parser but here it is for flexibility.

decision - Which decision to try different alternatives for.

alts - The set of alternatives to try while re-parsing.

startIndex - The index of the first token of the ambiguous input or other input of interest.

stopIndex - The index of the last token of the ambiguous input. The start and stop indexes are used primarily to identify how much of the resulting parse tree to return.

startRuleIndex - The start rule for the entire grammar, not the ambiguous decision. We re-parse the entire input and so we need the original start rule.

Returns:

The list of all possible interpretations of the input for the decision in ambiguityInfo. The actual interpretation chosen by the parser is always given first because this method retests the input in alternative order and ANTLR always resolves ambiguities by choosing the first alternative that matches the input. The subtree returned

Throws:

RecognitionException - Throws upon syntax error while matching ambig input.

Since:

4.5.1

getLookaheadParseTrees

public static List<ParserRuleContext> getLookaheadParseTrees(Grammar g,
                                             ParserInterpreter originalParser,
                                             TokenStream tokens,
                                             int startRuleIndex,
                                             int decision,
                                             int startIndex,
                                             int stopIndex)

Return a list of parse trees, one for each alternative in a decision given the same input. Very similar to getAllPossibleParseTrees(org.antlr.v4.tool.Grammar, org.antlr.v4.runtime.Parser, org.antlr.v4.runtime.TokenStream, int, java.util.BitSet, int, int, int) except that it re-parses the input for every alternative in a decision, not just the ambiguous ones (there is no alts parameter here). This method also tries to reduce the size of the parse trees by stripping away children of the tree that are completely out of range of startIndex..stopIndex. Also, because errors are expected, we use a specialized error handler that more or less bails out but that also consumes the first erroneous token at least. This ensures that an error node will be in the parse tree for display. NOTES: // we must parse the entire input now with decision overrides // we cannot parse a subset because it could be that a decision // above our decision of interest needs to read way past // lookaheadInfo.stopIndex. It seems like there is no escaping // the use of a full and complete token stream if we are // resetting to token index 0 and re-parsing from the start symbol. // It's not easy to restart parsing somewhere in the middle like a // continuation because our call stack does not match the // tree stack because of left recursive rule rewriting. grrrr!

Since:

4.5.1

deriveTempParserInterpreter

public static ParserInterpreter deriveTempParserInterpreter(Grammar g,
                                            Parser originalParser,
                                            TokenStream tokens)

Derive a new parser from an old one that has knowledge of the grammar. The Grammar object is used to correctly compute outer alternative numbers for parse tree nodes. A parser of the same type is created for subclasses of ParserInterpreter.