tango.text.Text

Kris

Text is a class for managing and manipulating Unicode character arrays.

Text maintains a current "selection", controlled via the select() and search() methods. Each of append(), prepend(), replace() and remove() operate with respect to the selection.

The search() methods also operate with respect to the current selection, providing a means of iterating across matched patterns. To set a selection across the entire content, use the select() method with no arguments.

Indexes and lengths of content always count code units, not code points. This is similar to traditional ascii string handling, yet indexing is rarely used in practice due to the selection idiom: substring indexing is generally implied as opposed to manipulated directly. This allows for a more streamlined model with regard to utf-surrogates.

Strings support a range of functionality, from insert and removal to utf encoding and decoding. There is also an immutable subset called TextView, intended to simplify life in a multi-threaded environment. However, TextView must expose the raw content as needed and thus immutability depends to an extent upon so-called "honour" of a callee. D does not enable immutability enforcement at this time, but this class will be modified to support such a feature when it arrives - via the slice() method.

The class is templated for use with char[], wchar[], and dchar[], and should migrate across encodings seamlessly. In particular, all functions in tango.text.Util are compatible with Text content in any of the supported encodings. In future, this class will become a principal gateway to the extensive ICU unicode library.

Note that several common text operations can be constructed through combining tango.text.Text with tango.text.Util e.g. lines of text can be processed thusly:

auto source = new Text!(char)("one\ntwo\nthree");

foreach (line; Util.lines(source.slice()))
         // do something with line

Speaking a bit like Yoda might be accomplished as follows:

auto dst = new Text!(char);

foreach (element; Util.delims ("all cows eat grass", " "))
         dst.prepend (element);

Below is an overview of the API and class hierarchy:

1 class Text(T) : TextView!(T)
2 {
3         // set or reset the content
4         Text set (T[] chars, bool mutable=true);
5         Text set (const(TextView) other, bool mutable=true);
6 
7         // retrieve currently selected text
8         T[] selection ();
9 
10         // set and retrieve current selection point
11         Text point (size_t index);
12         size_t point ();
13 
14         // mark a selection
15         Text select (int start=0, int length=int.max);
16 
17         // return an iterator to move the selection around.
18         // Also exposes "replace all" functionality
19         Search search (T chr);
20         Search search (T[] pattern);
21 
22         // format arguments behind current selection
23         Text format (T[] format, ...);
24 
25         // append behind current selection
26         Text append (T[] text);
27         Text append (const(TextView) other);
28         Text append (T chr, int count=1);
29         Text append (InputStream source);
30 
31         // transcode behind current selection
32         Text encode (char[]);
33         Text encode (wchar[]);
34         Text encode (dchar[]);
35 
36         // insert before current selection
37         Text prepend (T[] text);
38         Text prepend (const(TextView) other);
39         Text prepend (T chr, int count=1);
40 
41         // replace current selection
42         Text replace (T chr);
43         Text replace (T[] text);
44         Text replace (const(TextView) other);
45 
46         // remove current selection
47         Text remove ();
48 
49         // clear content
50         Text clear ();
51 
52         // trim leading and trailing whitespace
53         Text trim ();
54 
55         // trim leading and trailing chr instances
56         Text strip (T chr);
57 
58         // truncate at point, or current selection
59         Text truncate (int point = int.max);
60 
61         // reserve some space for inserts/additions
62         Text reserve (int extra);
63 
64         // write content to stream
65         Text write (OutputStream sink);
66 }
67 
68 class TextView(T) : UniText
69 {
70         // hash content
71         hash_t toHash ();
72 
73         // return length of content
74         size_t length ();
75 
76         // compare content
77         bool equals  (T[] text);
78         bool equals  (const(TextView) other);
79         bool ends    (T[] text);
80         bool ends    (const(TextView) other);
81         bool starts  (T[] text);
82         bool starts  (const(TextView) other);
83         int compare  (T[] text);
84         int compare  (const(TextView) other);
85         int opEquals (Object other);
86         int opCmp    (Object other);
87 
88         // copy content
89         T[] copy (T[] dst);
90 
91         // return content
92         T[] slice ();
93 
94         // return data type
95         typeinfo encoding ();
96 
97         // replace the comparison algorithm
98         Comparator comparator (Comparator other);
99 }
100 
101 class UniText
102 {
103         // convert content
104         abstract char[]  toString   (char[]  dst = null);
105         abstract wchar[] toString16 (wchar[] dst = null);
106         abstract dchar[] toString32 (dchar[] dst = null);
107 }
108 
109 struct Search
110 {
111         // select prior instance
112         bool prev();
113 
114         // select next instance
115         bool next();
116 
117         // return instance count
118         size_t count();
119 
120         // contains instance?
121         bool within();
122 
123         // replace all with char
124         void replace(T);
125 
126         // replace all with text (null == remove all)
127         void replace(T[]);
128 }