Constructor Detail

• TokenStreamCopyingHiddenTokenFilter

  public TokenStreamCopyingHiddenTokenFilter(TokenStream input)

Method Detail

• copy

  public void copy(int tokenType)

  Indicate that all tokens of type tokenType should be copied. The copy is put in the stream of hidden tokens, and the original is returned in the stream of normal tokens.

  Parameters:

  tokenType - integer representing the token type to copied

• partialCloneToken

  public CommonHiddenStreamToken partialCloneToken(CommonHiddenStreamToken t)

  Create a clone of the important parts of the given token. Note that this does NOT copy the hiddenBefore and hiddenAfter fields.

  Parameters:

  t - token to partially clone

  Returns:

  newly created partial clone

• linkAndCopyToken

  public void linkAndCopyToken(CommonHiddenStreamToken prev,
                      CommonHiddenStreamToken monitored)

• nextToken

  public Token nextToken()
                  throws TokenStreamException

  Return the next monitored token. Test the token following the monitored token. If following is another monitored token, save it for the next invocation of nextToken (like a single lookahead token) and return it then. If following is unmonitored, nondiscarded (hidden) channel token, add it to the monitored token. Note: EOF must be a monitored Token.

  Throws:

  TokenStreamException