<?xml version="1.0" encoding="UTF-8"?>
<chapter id="hyphenator.upgrading">
    <title>Upgrading from the previous version of the <package>Hyphenator</package></title>
    <sect1>
        <title>Removed features</title>
        <para>
            Due to the UTF-8-capability of the new <package>Hyphenator</package>-package
            <methodname>setSpecialChars()</methodname> has been removed, as that simply was
            a workaround for non-ASCII-characters.
        </para>
        <para>
            Currently the feature to disable already hyphenated words and to replace
            custom-hyphenation-characters with the default-hyphen-character as disabled.
            This feature will come again in a later version, but has currently not been implemented
            in the new version. So, if you NEED that feature DO NOT UPGRADE!
        </para>
    </sect1>
    <sect1>
        <title>Different approach to tokenizing strings</title>
        <para>
            In the previous version of the <package>Hyphenator</package> a string has been tokenized into
            words by using whitespace and - to a certain extend - some special punktuation marks. 
            This has been done right inside the hyphenation-method which was a rather dirty trick
             and has proven extremely inflexible an inacurate
        </para>
        <para>
            Therefore this tokenizing has been completely rewritten for the new version. Now you can add 
            <classname>\Org\Heigl\Hyphenator\Tokenizer</classname>-Objects to your Hyphenato-Instance that will
            split the given string into Word-Tokens (those will be hyphenated later), Whitespace-Tokens and Non-Word-Tokens.
        </para>
        <para>
            Currently there are two tokenizers delivered with the package.
            <variablelist>
                <varlistentry><term><classname>WhitespaceTokenizer</classname></term><listitem>
                    Split the given string at every whitespace occurence creating <classname>WhitespaceTokens</classname>
                    and <classname>WordTokens</classname>. 
                </listitem></varlistentry>
                <varlistentry><term><classname>PunctuationTokenizer</classname></term>
                <listitem>
                    Split the given string at every occurence of a common set of punctuation-characters creating 
                    <classname>NonWordTokens</classname> for the punctuation-characters and <classname>WordTokens</classname>
                    for everything else
                </listitem>
                </varlistentry>
            </variablelist>
            But more Tokenizers will come and feel free to write your own ones.
        </para>
        <para>
            The above mentioned feature to disable hyphenation of already hyphenated words will be implemented
            using such a new tokenizer.
        </para>
    </sect1>
</chapter>