Forgivingly lexes HTML (SGML-style) markup into tokens. More...

Inheritance diagram for HTMLPurifier_Lexer:

Public Member Functions
	__construct ()
	parseData ($string)
	Parses special entities into the proper characters.
	tokenizeHTML ($string, $config, $context)
	Lexes an HTML string into tokens.
	normalize ($html, $config, $context)
	Takes a piece of HTML and normalizes it by converting entities, fixing encoding, extracting bits, and other good stuff.
	extractBody ($html)
	Takes a string of HTML (fragment or document) and returns the content.
	__construct ()
	parseData ($string)
	Parses special entities into the proper characters.
	tokenizeHTML ($string, $config, $context)
	Lexes an HTML string into tokens.
	normalize ($html, $config, $context)
	Takes a piece of HTML and normalizes it by converting entities, fixing encoding, extracting bits, and other good stuff.
	extractBody ($html)
	Takes a string of HTML (fragment or document) and returns the content.
Static Public Member Functions
static	create ($config)
	Retrieves or sets the default Lexer as a Prototype Factory.
static	create ($config)
	Retrieves or sets the default Lexer as a Prototype Factory.
Public Attributes
	$tracksLineNumbers = false
	Whether or not this lexer implements line-number/column-number tracking.
Static Protected Member Functions
static	escapeCDATA ($string)
	Translates CDATA sections into regular sections (through escaping).
static	escapeCommentedCDATA ($string)
	Special CDATA case that is especially convoluted for <script>
static	removeIEConditional ($string)
	Special Internet Explorer conditional comments should be removed.
static	CDATACallback ($matches)
	Callback function for escapeCDATA() that does the work.
static	escapeCDATA ($string)
	Translates CDATA sections into regular sections (through escaping).
static	escapeCommentedCDATA ($string)
	Special CDATA case that is especially convoluted for <script>
static	removeIEConditional ($string)
	Special Internet Explorer conditional comments should be removed.
static	CDATACallback ($matches)
	Callback function for escapeCDATA() that does the work.
Protected Attributes
	$_special_entity2str
	Most common entity to raw value conversion table for special entities.

Detailed Description

Forgivingly lexes HTML (SGML-style) markup into tokens.

A lexer parses a string of SGML-style markup and converts them into corresponding tokens. It doesn't check for well-formedness, although its internal mechanism may make this automatic (such as the case of HTMLPurifier_Lexer_DOMLex). There are several implementations to choose from.

A lexer is HTML-oriented: it might work with XML, but it's not recommended, as we adhere to a subset of the specification for optimization reasons. This might change in the future. Also, most tokenizers are not expected to handle DTDs or PIs.

This class should not be directly instantiated, but you may use create() to retrieve a default copy of the lexer. Being a supertype, this class does not actually define any implementation, but offers commonly used convenience functions for subclasses.

Note:: The unit tests will instantiate this class for testing purposes, as many of the utility functions require a class to be instantiated. This means that, even though this class is not runnable, it will not be declared abstract.

Note:: We use tokens rather than create a DOM representation because DOM would:

Require more processing and memory to create,
Is not streamable, and
Has the entire document structure (html and body not needed).

: However, DOM is helpful in that it makes it easy to move around nodes without a lot of lookaheads to see when a tag is closed. This is a limitation of the token system and some workarounds would be nice.

Definition at line 42 of file Lexer.php.

Constructor & Destructor Documentation

HTMLPurifier_Lexer::__construct ( )

Reimplemented in HTMLPurifier_Lexer_DOMLex, and HTMLPurifier_Lexer_DOMLex.

Definition at line 142 of file Lexer.php.

HTMLPurifier_Lexer::__construct ( )

Reimplemented in HTMLPurifier_Lexer_DOMLex, and HTMLPurifier_Lexer_DOMLex.

Definition at line 6335 of file HTMLPurifier.standalone.php.

Member Function Documentation

static HTMLPurifier_Lexer::CDATACallback ( $ matches ) [static, protected]

Callback function for escapeCDATA() that does the work.

Warning:: Though this is public in order to let the callback happen, calling it directly is not recommended. $matches PCRE matches array, with index 0 the entire match and 1 the inside of the CDATA section.

Returns:: Escaped internals of the CDATA section.

Definition at line 253 of file Lexer.php.

static HTMLPurifier_Lexer::CDATACallback ( $ matches ) [static, protected]

Callback function for escapeCDATA() that does the work.

Warning:: Though this is public in order to let the callback happen, calling it directly is not recommended. $matches PCRE matches array, with index 0 the entire match and 1 the inside of the CDATA section.

Returns:: Escaped internals of the CDATA section.

Definition at line 6446 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Lexer::create ( $ config ) [static]

Retrieves or sets the default Lexer as a Prototype Factory.

By default HTMLPurifier_Lexer_DOMLex will be returned. There are a few exceptions involving special features that only DirectLex implements.

Note:: The behavior of this class has changed, rather than accepting a prototype object, it now accepts a configuration object. To specify your own prototype, set Core.LexerImpl to it. This change in behavior de-singletonizes the lexer object.

Parameters:

$config Instance of HTMLPurifier_Config

Returns:: Concrete lexer.

Definition at line 6261 of file HTMLPurifier.standalone.php.

References $config.

static HTMLPurifier_Lexer::create ( $ config ) [static]

Retrieves or sets the default Lexer as a Prototype Factory.

By default HTMLPurifier_Lexer_DOMLex will be returned. There are a few exceptions involving special features that only DirectLex implements.

Note:: The behavior of this class has changed, rather than accepting a prototype object, it now accepts a configuration object. To specify your own prototype, set Core.LexerImpl to it. This change in behavior de-singletonizes the lexer object.

Parameters:

$config Instance of HTMLPurifier_Config

Returns:: Concrete lexer.

Definition at line 68 of file Lexer.php.

References $config.

Referenced by HTMLPurifier::purify().

static HTMLPurifier_Lexer::escapeCDATA ( $ string ) [static, protected]

Translates CDATA sections into regular sections (through escaping).

Parameters:

$string HTML string to process.

Returns:: HTML with CDATA sections escaped.

Definition at line 214 of file Lexer.php.

Referenced by normalize().

static HTMLPurifier_Lexer::escapeCDATA ( $ string ) [static, protected]

Translates CDATA sections into regular sections (through escaping).

Parameters:

$string HTML string to process.

Returns:: HTML with CDATA sections escaped.

Definition at line 6407 of file HTMLPurifier.standalone.php.

static HTMLPurifier_Lexer::escapeCommentedCDATA ( $ string ) [static, protected]

Special CDATA case that is especially convoluted for <script>

Definition at line 225 of file Lexer.php.

Referenced by normalize().

static HTMLPurifier_Lexer::escapeCommentedCDATA ( $ string ) [static, protected]

Special CDATA case that is especially convoluted for <script>

Definition at line 6418 of file HTMLPurifier.standalone.php.

HTMLPurifier_Lexer::extractBody ( $ html )

Takes a string of HTML (fragment or document) and returns the content.

Todo:: Consider making protected

Definition at line 314 of file Lexer.php.

References $html.

Referenced by normalize().

HTMLPurifier_Lexer::extractBody ( $ html )

Takes a string of HTML (fragment or document) and returns the content.

Todo:: Consider making protected

Definition at line 6507 of file HTMLPurifier.standalone.php.

References $html.

HTMLPurifier_Lexer::normalize	(	$	html,
		$	config,
		$	context
	)

Takes a piece of HTML and normalizes it by converting entities, fixing encoding, extracting bits, and other good stuff.

Todo:: Consider making protected

Definition at line 263 of file Lexer.php.

References $config, $html, HTMLPurifier_Encoder::cleanUTF8(), escapeCDATA(), escapeCommentedCDATA(), extractBody(), and removeIEConditional().

Referenced by HTMLPurifier_Lexer_PH5P::tokenizeHTML(), HTMLPurifier_Lexer_DirectLex::tokenizeHTML(), and HTMLPurifier_Lexer_DOMLex::tokenizeHTML().

HTMLPurifier_Lexer::normalize	(	$	html,
		$	config,
		$	context
	)

Takes a piece of HTML and normalizes it by converting entities, fixing encoding, extracting bits, and other good stuff.

Todo:: Consider making protected

Definition at line 6456 of file HTMLPurifier.standalone.php.

References $config, $html, HTMLPurifier_Encoder::cleanUTF8(), escapeCDATA(), escapeCommentedCDATA(), extractBody(), and removeIEConditional().

HTMLPurifier_Lexer::parseData ( $ string )

Parses special entities into the proper characters.

This string will translate escaped versions of the special characters into the correct ones.

Warning:: You should be able to treat the output of this function as completely parsed, but that's only because all other entities should have been handled previously in substituteNonSpecialEntities()

Parameters:

$string String character data to be parsed.

Returns:: Parsed character data.

Definition at line 6367 of file HTMLPurifier.standalone.php.

HTMLPurifier_Lexer::parseData ( $ string )

Parses special entities into the proper characters.

This string will translate escaped versions of the special characters into the correct ones.

Warning:: You should be able to treat the output of this function as completely parsed, but that's only because all other entities should have been handled previously in substituteNonSpecialEntities()

Parameters:

$string String character data to be parsed.

Returns:: Parsed character data.

Definition at line 174 of file Lexer.php.

Referenced by HTMLPurifier_Lexer_DOMLex::createStartNode(), HTMLPurifier_Lexer_DirectLex::parseAttributeString(), and HTMLPurifier_Lexer_DirectLex::tokenizeHTML().

static HTMLPurifier_Lexer::removeIEConditional ( $ string ) [static, protected]

Special Internet Explorer conditional comments should be removed.

Definition at line 236 of file Lexer.php.

Referenced by normalize().

static HTMLPurifier_Lexer::removeIEConditional ( $ string ) [static, protected]

Special Internet Explorer conditional comments should be removed.

Definition at line 6429 of file HTMLPurifier.standalone.php.

HTMLPurifier_Lexer::tokenizeHTML	(	$	string,
		$	config,
		$	context
	)

Lexes an HTML string into tokens.

Parameters:

$string String HTML.

Returns:: HTMLPurifier_Token array representation of HTML.

Reimplemented in HTMLPurifier_Lexer_DirectLex, HTMLPurifier_Lexer_DOMLex, HTMLPurifier_Lexer_PH5P, HTMLPurifier_Lexer_DOMLex, and HTMLPurifier_Lexer_DirectLex.

Definition at line 6397 of file HTMLPurifier.standalone.php.

HTMLPurifier_Lexer::tokenizeHTML	(	$	string,
		$	config,
		$	context
	)

Lexes an HTML string into tokens.

Parameters:

$string String HTML.

Returns:: HTMLPurifier_Token array representation of HTML.

Reimplemented in HTMLPurifier_Lexer_DirectLex, HTMLPurifier_Lexer_DOMLex, HTMLPurifier_Lexer_PH5P, HTMLPurifier_Lexer_DOMLex, and HTMLPurifier_Lexer_DirectLex.

Definition at line 204 of file Lexer.php.

Member Data Documentation

HTMLPurifier_Lexer::$_special_entity2str [protected]

Initial value:

            array(
                    '&quot;' => '"',
                    '&amp;'  => '&',
                    '&lt;'   => '<',
                    '&gt;'   => '>',
                    '&#39;'  => "'",
                    '&#039;' => "'",
                    '&#x27;' => "'"
            )

Most common entity to raw value conversion table for special entities.

Definition at line 149 of file Lexer.php.

HTMLPurifier_Lexer::$tracksLineNumbers = false

Whether or not this lexer implements line-number/column-number tracking.

If it does, set to true.

Reimplemented in HTMLPurifier_Lexer_DirectLex.

Definition at line 49 of file Lexer.php.

The documentation for this class was generated from the following files:

/home/ezyang/Dev/htmlpurifier/library/HTMLPurifier/Lexer.php
/home/ezyang/Dev/htmlpurifier/library/HTMLPurifier.standalone.php

Public Member Functions

Static Public Member Functions

Public Attributes

Static Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation