Uses of Interface
opennlp.tools.tokenize.TokenContextGenerator
Packages that use TokenContextGenerator
Package
Description
Contains classes related to finding token or words in a string.
-
Uses of TokenContextGenerator in opennlp.tools.tokenize
Classes in opennlp.tools.tokenize that implement TokenContextGeneratorModifier and TypeClassDescriptionclass
A defaultTokenContextGenerator
which produces events for maxent decisions for tokenization.Methods in opennlp.tools.tokenize that return TokenContextGeneratorConstructors in opennlp.tools.tokenize with parameters of type TokenContextGeneratorModifierConstructorDescriptionTokSpanEventStream
(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, Pattern alphaNumeric, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator
.TokSpanEventStream
(ObjectStream<TokenSample> tokenSamples, boolean skipAlphaNumerics, TokenContextGenerator cg) Initializes a new event stream based on the data stream using aTokenContextGenerator
. -
Uses of TokenContextGenerator in opennlp.tools.tokenize.lang
Methods in opennlp.tools.tokenize.lang that return TokenContextGeneratorModifier and TypeMethodDescriptionFactory.createTokenContextGenerator
(String languageCode, Set<String> abbreviations) Initializes a customizedTokenContextGenerator
via a set ofabbreviations
.