It doesn't tokenize, it just splits; there's a difference -- if your tokens have embedded separators (consider, a space-delimited sequence of tokens, where tokens can have backslash-escaped ...

Most Java programmers have used the java.util.StringTokenizer class at some time or another. It is a handy class that basically tokenizes (breaks) the input string based on a separator, and supplies ...

The tokenize () method is a wrapper for the StringTokenizer class. The StringTokenizer constructor takes two String arguments: one for the input and one for the delimiter.