Re: Lexing and STk

From: Matt Wette <Matthew.R.Wette_at_jpl.nasa.gov>
Date: Mon, 02 Dec 1996 11:23:24 -0800

David Fox writes ...
> It so happened that I wanted to do a lexical analysis of an input file
> today, and I discovered that it is not too easy in STk. I tried using
> anchored regular expressions (e.g. "^[0-9][0-9]*") to match tokens,
> but then I had to chop each matched token off the string using
> substring before matching the next token, and substring always mallocs
> a whole new string. Perhaps STk should check for the special case of
> a substring which extends to the end of the original string, and store
> a bit to indicate that it the resulting string is a substring and
> doesn't own its data. Or is there a more sensible way of doing this
> that I'm not seeing?

Guile has "shared substrings" to deal with this problem:

Shared Substrings
*****************
 
  Whenever you extract a substring using `substring', a new string is
allocated and substring data is copied from the old string to the new
string.
 
  Sometimes, in order avoid the expense of copying, programmers will
write programs that pass around triples: a string, a starting position
within the string, and an ending position. This convention is awkward,
especially because many useful string primitives operate only on full
strings, not a string with substring indexes.
 
  Guile Scheme offers another way: shared substrings. A shared
substring is a read-only string built from an older string, and
substring indexes into that older string. A shared substring is not a
copy of the older string, but shares the older strings data directly.
Compared to copying substrings, shared substrings are cheap to allocate.
 
 - Function: make-shared-substring SOURCE-STRING START-POS ?END-POS?
     Return a shared substring of SOURCE-STRING, from START-POS to
     END-POS or the end of the source string
 
                (define some-string "the quick brown fox")
                (define x (make-shared-substring some-string 4 9))
                x => "quick"
          
                (string-set! some-string 7 #\r)
                x => "quirk"
          
                (string? some-string) => #t
                (string? x)
                (read-only-string? some-string) => #t
                (read-only-string? x) => #t
          
                (define y (make-shared-substring x 2))
                y => "irk"
 
  String operators which mutate their arguments do not work on shared
substrings but most other string operators do work. For example, if you
want to compare the substring [FROM ... TO - 1] of string A to the
string B, you can use the relatively inexpensive idiom:
 
     (string=? (make-shared-substring A FROM TO) B)
 
 
Received on Mon Dec 02 1996 - 20:24:59 CET