erl_scan
The Erlang Token Scanner
This module contains functions for tokenizing characters into Erlang tokens.
Functions
string(CharList,StartLine]) -> {ok, Tokens, EndLine} | Error
string(CharList) -> {ok, Tokens, EndLine} | Error
CharList = string()
StartLine = EndLine = Line = integer()
Tokens = [{atom(),Line}|{atom(),Line,term()}]
Error = {error, ErrorInfo, EndLine}
Takes the list of characters CharList
and tries to scan (tokenize) them. Returns {ok, Tokens, EndLine}
, where Tokens
are the Erlang tokens
from CharList
. EndLine
is the last line where a token was found.
StartLine
indicates the initial line when scanning starts. string/1
is equivalent to string(CharList,1)
.
{error, ErrorInfo, EndLine}
is returned if an error occurs.
EndLine
indicates where the error occurred.
tokens(Continuation, CharList, StartLine) ->Return
Return = {done, Result, LeftOverChars} | {more, Continuation}
Continuation = [] | string()
CharList = string()
StartLine = EndLine = integer()
Result = {ok, Tokens, EndLine} | {eof, EndLine}
Tokens = [{atom(),Line}|{atom(),Line,term()}]
This is the re-entrant scanner which scans characters until a dot ('.' whitespace) has been reached. It returns:
{done, Result, LeftOverChars}
This return indicates that there is sufficient input data to get an
input. Result
is:
{ok, Tokens, EndLine}
The scanning was successful. Tokens
is the
list of tokens including dot.
{eof, EndLine}
End of file was encountered before any more tokens.
{error, ErrorInfo, EndLine}
An error occurred.
{more, Continuation}
More data is required for building a term.
Continuation
must be passed in a new call to
tokens/3
when more data is available.
reserved_word(Atom) -> bool()
Returns true
if Atom
is an Erlang reserved
word, otherwise false
.
format_error(ErrorDescriptor) -> string()
ErrorDescriptor = errordesc()
Takes an ErrorDescriptor
and returns a string which
describes the error or warning. This function is usually
called implicitly when processing an ErrorInfo
structure (see below).
Error Information
The ErrorInfo
mentioned above is the standard
ErrorInfo
structure which is returned from all IO
modules. It has the following format:
{ErrorLine, Module, ErrorDescriptor}
A string which describes the error is obtained with the following call:
apply(Module, format_error, ErrorDescriptor)
Notes
The continuation of the first call to the re-entrant input
functions must be []
. Refer to Armstrong, Virding and Williams, 'Concurrent Programming in Erlang', Chapter 13, for a complete description of how
the re-entrant input scheme works.