lundi 20 juin 2016

how to get line offset in StreamTokenizer?

I am working on a parser for my class that uses the StreamTokenizer class in java. In the case of a parsing error, I want to be able to print the exact line and offset of the character that begins the token where the error occurred. However, while the StreamTokenizer has a lineno() method to find which line the tokenizer is at, there is no method to find the character offset within that line.

I am hoping that somehow there is a way to get this offset using the available functions in either STreamTokenizer or BufferedReader, the input to the StreamTokenizer constructor.

So far, I have tried using something like this:

`
BufferedReader dataReader = new BufferedReader(new FileReader(filename));
StreamTokenizer st = new StreamTokenizer(dataReader);
st.eolIsSignificant(true);
`

Then, I made a wrapper around the

 StreamTokenizer.nextToken()

function so that it looks something like this:

 public int nextTokenSpec(StreamTokenizer st) throws IOException{
        int token = st.nextToken();

        if (token == StreamTokenizer.TT_EOL){
            Linker2.offsetCounter = 0;
            token = st.nextToken();
        } else{
            Linker2.offsetCounter += st.sval.length();
        }
        return token;
    }

note that Linker2 is a driver class that contains the main function where the above code (BufferedReader and StreamTokenizer) are invoked.

However, the problem with this is that it ignores the token delimiters, as it only increments based on the length of the tokens.

I suspect there may be some way to go directly to the BufferedReader to get info on this, but I am not sure.

Does anyone know how I can get the exact line offset of the StreamTokenizer function?

-Paul

Aucun commentaire:

Enregistrer un commentaire