HTMLPurifier_Encoder Class Reference

A UTF-8 specific character encoder that handles cleaning and transforming. More...

List of all members.

Static Public Member Functions

static cleanUTF8 ($str, $force_php=false)
 Cleans a UTF-8 string for well-formedness and SGML validity.
static unichr ($code)
 Translates a Unicode codepoint into its corresponding UTF-8 character.
static convertToUTF8 ($str, $config, $context)
 Converts a string to UTF-8 based on configuration.
static convertFromUTF8 ($str, $config, $context)
 Converts a string from UTF-8 based on configuration.
static convertToASCIIDumbLossless ($str)
 Lossless (character-wise) conversion of HTML to ASCII.
static testEncodingSupportsASCII ($encoding, $bypass=false)
 This expensive function tests whether or not a given character encoding supports ASCII.
static cleanUTF8 ($str, $force_php=false)
 Cleans a UTF-8 string for well-formedness and SGML validity.
static unichr ($code)
 Translates a Unicode codepoint into its corresponding UTF-8 character.
static convertToUTF8 ($str, $config, $context)
 Converts a string to UTF-8 based on configuration.
static convertFromUTF8 ($str, $config, $context)
 Converts a string from UTF-8 based on configuration.
static convertToASCIIDumbLossless ($str)
 Lossless (character-wise) conversion of HTML to ASCII.
static testEncodingSupportsASCII ($encoding, $bypass=false)
 This expensive function tests whether or not a given character encoding supports ASCII.

Private Member Functions

 __construct ()
 Constructor throws fatal error if you attempt to instantiate class.
 __construct ()
 Constructor throws fatal error if you attempt to instantiate class.

Static Private Member Functions

static muteErrorHandler ()
 Error-handler that mutes errors, alternative to shut-up operator.
static muteErrorHandler ()
 Error-handler that mutes errors, alternative to shut-up operator.

Detailed Description

A UTF-8 specific character encoder that handles cleaning and transforming.

Note:
All functions in this class should be static.

Definition at line 7 of file Encoder.php.

Constructor & Destructor Documentation

HTMLPurifier_Encoder::__construct (  )  [private]

Constructor throws fatal error if you attempt to instantiate class.

Definition at line 13 of file Encoder.php.

HTMLPurifier_Encoder::__construct (  )  [private]

Constructor throws fatal error if you attempt to instantiate class.

Definition at line 2696 of file HTMLPurifier.standalone.php.

Member Function Documentation

static HTMLPurifier_Encoder::muteErrorHandler (  )  [static, private]

Error-handler that mutes errors, alternative to shut-up operator.

Definition at line 20 of file Encoder.php.

Referenced by convertFromUTF8(), convertToUTF8(), and testEncodingSupportsASCII().

static HTMLPurifier_Encoder::cleanUTF8 ( $  str,
$  force_php = false 
) [static]

Cleans a UTF-8 string for well-formedness and SGML validity.

It will parse according to UTF-8 and return a valid UTF8 string, with non-SGML codepoints excluded.

Note:
Just for reference, the non-SGML code points are 0 to 31 and 127 to 159, inclusive. However, we allow code points 9, 10 and 13, which are the tab, line feed and carriage return respectively. 128 and above the code points map to multibyte UTF-8 representations.

Fallback code adapted from utf8ToUnicode by Henri Sivonen and hsivonen@iki.fi at <http://iki.fi/hsivonen/php-utf8/> under the LGPL license. Notes on what changed are inside, but in general, the original code transformed UTF-8 text into an array of integer Unicode codepoints. Understandably, transforming that back to a string would be somewhat expensive, so the function was modded to directly operate on the string. However, this discourages code reuse, and the logic enumerated here would be useful for any function that needs to be able to understand UTF-8 characters. As of right now, only smart lossless character encoding converters would need that, and I'm probably not going to implement them. Once again, PHP 6 should solve all our problems.

Definition at line 47 of file Encoder.php.

Referenced by HTMLPurifier_Printer::escape(), HTMLPurifier_Lexer::normalize(), and HTMLPurifier_AttrDef_CSS_FontFamily::validate().

static HTMLPurifier_Encoder::unichr ( $  code  )  [static]

Translates a Unicode codepoint into its corresponding UTF-8 character.

Note:
Based on Feyd's function at <http://forums.devnetwork.net/viewtopic.php?p=191404#191404>, which is in public domain.

While we're going to do code point parsing anyway, a good optimization would be to refuse to translate code points that are non-SGML characters. However, this could lead to duplication.

This is very similar to the unichr function in maintenance/generate-entity-file.php (although this is superior, due to its sanity checks).

Definition at line 226 of file Encoder.php.

Referenced by HTMLPurifier_EntityParser::nonSpecialEntityCallback(), and HTMLPurifier_AttrDef_CSS_FontFamily::validate().

static HTMLPurifier_Encoder::convertToUTF8 ( $  str,
$  config,
$  context 
) [static]

Converts a string to UTF-8 based on configuration.

Definition at line 266 of file Encoder.php.

References muteErrorHandler(), and testEncodingSupportsASCII().

Referenced by HTMLPurifier::purify().

static HTMLPurifier_Encoder::convertFromUTF8 ( $  str,
$  config,
$  context 
) [static]

Converts a string from UTF-8 based on configuration.

Note:
Currently, this is a lossy conversion, with unexpressable characters being omitted.

Definition at line 293 of file Encoder.php.

References convertToASCIIDumbLossless(), muteErrorHandler(), and testEncodingSupportsASCII().

Referenced by HTMLPurifier::purify().

static HTMLPurifier_Encoder::convertToASCIIDumbLossless ( $  str  )  [static]

Lossless (character-wise) conversion of HTML to ASCII.

Parameters:
$str UTF-8 string to be converted to ASCII
Returns:
ASCII encoded string with non-ASCII character entity-ized
Warning:
Adapted from MediaWiki, claiming fair use: this is a common algorithm. If you disagree with this license fudgery, implement it yourself.
Note:
Uses decimal numeric entities since they are best supported.

This is a DUMB function: it has no concept of keeping character entities that the projected character encoding can allow. We could possibly implement a smart version but that would require it to also know which Unicode codepoints the charset supported (not an easy task).

Sort of with cleanUTF8() but it assumes that $str is well-formed UTF-8

Definition at line 339 of file Encoder.php.

Referenced by convertFromUTF8().

static HTMLPurifier_Encoder::testEncodingSupportsASCII ( $  encoding,
$  bypass = false 
) [static]

This expensive function tests whether or not a given character encoding supports ASCII.

7/8-bit encodings like Shift_JIS will fail this test, and require special processing. Variable width encodings shouldn't ever fail.

Parameters:
string $encoding Encoding name to test, as per iconv format
bool $bypass Whether or not to bypass the precompiled arrays.
Returns:
Array of UTF-8 characters to their corresponding ASCII, which can be used to "undo" any overzealous iconv action.

Definition at line 381 of file Encoder.php.

References muteErrorHandler().

Referenced by convertFromUTF8(), and convertToUTF8().

static HTMLPurifier_Encoder::muteErrorHandler (  )  [static, private]

Error-handler that mutes errors, alternative to shut-up operator.

Definition at line 2703 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Encoder::cleanUTF8 ( $  str,
$  force_php = false 
) [static]

Cleans a UTF-8 string for well-formedness and SGML validity.

It will parse according to UTF-8 and return a valid UTF8 string, with non-SGML codepoints excluded.

Note:
Just for reference, the non-SGML code points are 0 to 31 and 127 to 159, inclusive. However, we allow code points 9, 10 and 13, which are the tab, line feed and carriage return respectively. 128 and above the code points map to multibyte UTF-8 representations.

Fallback code adapted from utf8ToUnicode by Henri Sivonen and hsivonen@iki.fi at <http://iki.fi/hsivonen/php-utf8/> under the LGPL license. Notes on what changed are inside, but in general, the original code transformed UTF-8 text into an array of integer Unicode codepoints. Understandably, transforming that back to a string would be somewhat expensive, so the function was modded to directly operate on the string. However, this discourages code reuse, and the logic enumerated here would be useful for any function that needs to be able to understand UTF-8 characters. As of right now, only smart lossless character encoding converters would need that, and I'm probably not going to implement them. Once again, PHP 6 should solve all our problems.

Definition at line 2730 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Encoder::unichr ( $  code  )  [static]

Translates a Unicode codepoint into its corresponding UTF-8 character.

Note:
Based on Feyd's function at <http://forums.devnetwork.net/viewtopic.php?p=191404#191404>, which is in public domain.

While we're going to do code point parsing anyway, a good optimization would be to refuse to translate code points that are non-SGML characters. However, this could lead to duplication.

This is very similar to the unichr function in maintenance/generate-entity-file.php (although this is superior, due to its sanity checks).

Definition at line 2909 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Encoder::convertToUTF8 ( $  str,
$  config,
$  context 
) [static]

Converts a string to UTF-8 based on configuration.

Definition at line 2949 of file HTMLPurifier.standalone.php.

References muteErrorHandler(), and testEncodingSupportsASCII().

static HTMLPurifier_Encoder::convertFromUTF8 ( $  str,
$  config,
$  context 
) [static]

Converts a string from UTF-8 based on configuration.

Note:
Currently, this is a lossy conversion, with unexpressable characters being omitted.

Definition at line 2976 of file HTMLPurifier.standalone.php.

References convertToASCIIDumbLossless(), muteErrorHandler(), and testEncodingSupportsASCII().

static HTMLPurifier_Encoder::convertToASCIIDumbLossless ( $  str  )  [static]

Lossless (character-wise) conversion of HTML to ASCII.

Parameters:
$str UTF-8 string to be converted to ASCII
Returns:
ASCII encoded string with non-ASCII character entity-ized
Warning:
Adapted from MediaWiki, claiming fair use: this is a common algorithm. If you disagree with this license fudgery, implement it yourself.
Note:
Uses decimal numeric entities since they are best supported.

This is a DUMB function: it has no concept of keeping character entities that the projected character encoding can allow. We could possibly implement a smart version but that would require it to also know which Unicode codepoints the charset supported (not an easy task).

Sort of with cleanUTF8() but it assumes that $str is well-formed UTF-8

Definition at line 3022 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Encoder::testEncodingSupportsASCII ( $  encoding,
$  bypass = false 
) [static]

This expensive function tests whether or not a given character encoding supports ASCII.

7/8-bit encodings like Shift_JIS will fail this test, and require special processing. Variable width encodings shouldn't ever fail.

Parameters:
string $encoding Encoding name to test, as per iconv format
bool $bypass Whether or not to bypass the precompiled arrays.
Returns:
Array of UTF-8 characters to their corresponding ASCII, which can be used to "undo" any overzealous iconv action.

Definition at line 3064 of file HTMLPurifier.standalone.php.

References muteErrorHandler().

The documentation for this class was generated from the following files:
Generated on Thu Jun 19 18:47:30 2008 for HTMLPurifier by  doxygen 1.5.3