Forums - Internals

Mnify output (1 reply)

AJenbo — Wed, 22 May 2013 18:47:41 -0400

It might be nice to minify the output. I have a rather simple implementation for minifying xHTML 1.0:

//Combine all white space
$html = preg_replace('/\s+/us', ' ', $html);
//Strip all white space surrounding block level elements
$html =  preg_replace(
    '/\s*(<[\/]{0,1}(address|blockquote|center|dir|div|dl|fieldset|form|h1|h2|h3|h4|h5|h6|hr|isindex|menu|noframes|noscript|ol|p|pre|table|ul|dd|dt|frameset|li|tbody|td|tfoot|th|thead|tr|applet|button|del|iframe|insmap|object|script)(\s[^>]+?)*?>)\s*/us',
    '$1',
    $html
);
//Strip all whitespace surounding line breaks
$html = preg_replace('/\s*
\s*/us', '
', $html);

It's has a few backsides that i personally can live with

1: Does not respect literal (not written an an html entity) nbsp

2: Strips whitespace form "pre" content.

I think this should be rather fixable though

Edit: Line breaks, don't strip look-a-likes, less greedy regex

Edited 5 time(s). Last edit at 05/22/2013 08:00PM by AJenbo.

Linkify bug (3 replies)

nAS — Tue, 21 May 2013 18:25:48 -0400

Linkify directive doesn't work correctly when using no-breaking space or comma.

Example: http://bit.ly/11RAUaU

Proposed fix:

Linkify.php line 24 change from:

$bits = preg_split('#((?:https?|ftp)://[^\s\'"<>()]+)#S', $token->data, -1, PREG_SPLIT_DELIM_CAPTURE);

to:

$bits = preg_split('#((?:https?|ftp)://[^\s\'",<>()]+)#Su', $token->data, -1, PREG_SPLIT_DELIM_CAPTURE);

Invalid PHPDoc - HTMLPurifier.php(82) (no replies)

laurin1 — Fri, 17 May 2013 19:39:41 -0400

Line 82 of HTMLPurifier.php is

* @param $config Optional HTMLPurifier_Config object for all instances of

And should be:

* @param $config HTMLPurifier_Config object for all instances of

Invalid PHPDoc - Config.php(136) (1 reply)

laurin1 — Sat, 18 May 2013 11:49:16 -0400

Line 136 of Config.php is

* @return Default HTMLPurifier_Config object.

And should be:

* @return HTMLPurifier_Config object.

Scheme parsed incorrectly (1 reply)

Michael Gusev — Tue, 16 Apr 2013 16:57:32 -0400

Code below is parsed incorrectly:

Problem here because of incorrect detection of scheme in URL.

RFC 1738
2.1. The main parts of URLs

Scheme names consist of a sequence of characters. The lower case
letters "a"--"z", digits, and the characters plus ("+"), period
("."), and hyphen ("-") are allowed. For resiliency, programs
interpreting URLs should treat upper case letters as equivalent to
lower case in scheme names (e.g., allow "HTTP" as well as "http").

patch for fix

diff --git a/library/HTMLPurifier/URIParser.php b/library/HTMLPurifier/URIParser.php
index 7179e4ab8991077aa2ff4b3a4fca0a4eafd416e0..a7e5dd66eaab4fa6cc8b3daa3856ec450f1c35ce 100644
--- a/library/HTMLPurifier/URIParser.php
+++ b/library/HTMLPurifier/URIParser.php
@@ -30,7 +30,7 @@ class HTMLPurifier_URIParser
         // Note that ["<>] are an addition to the RFC's recommended
         // characters, because they represent external delimeters.
         $r_URI = '!'.
-            '(([^:/?#"<>]+):)?'. // 2. Scheme
+            '(([a-zA-Z0-9\.\+\-]+):)?'. // 2. Scheme
             '(//([^/?#"<>]*))?'. // 4. Authority
             '([^?#"<>]*)'.       // 5. Path
             '(\?([^#"<>]*))?'.   // 7. Query

HTMLPurifier_Strategy_MakeWellFormed is slow with big array of tokens (17 replies)

Michael Gusev — Thu, 23 May 2013 07:59:28 -0400

Hi there.

We met problem with big portions of broken html. If we have a lot of not closed tags then HTMLPurifier_Strategy_MakeWellFormed works really slow. Problem here in array_splice:

private function insertBefore($token) {
        array_splice($this->tokens, $this->t, 0, array($token));
}

private function remove() {
        array_splice($this->tokens, $this->t, 1);
}

if $this->tokens is array with 20k nodes then array_splice works really slow.

I have patch where I replaced array_splice by double linked list. It works up to 40 times faster then array_splice.

Could anyone direct me where I should send the patch or create PR?

Thank you.

Config Caching for Speed (3 replies)

xathien — Fri, 22 Mar 2013 19:36:13 -0400

I'll preface this by saying that we're probably using HTMLPurifier in a very nonstandard way. :) One example of a place we use this is in our Reporting system. We obviously want to allow our own scripts and custom HTML and such on the page, but any user-editable text on the page (of which there is much) we want to purify, right? We keep one HTMLPurifier instance around with one HTMLPurifier_Config, and then purify hundreds of strings, one-by-one, as we encounter them while parsing the report's definition. Simple, right? If there's a more preferred way of doing this, I'd certainly like to hear about it. :)

In the meantime, and as the hard-nosed optimizer on our Engineering team, I've done some profiling of how HTMLPurifier runs in our context. Since this page says I should report my findings, well...

So far I've optimized two things in HTMLPurifier_Config. Even though we use the same instance of the Config across all our strings (in my profiling examples, we're purifying 431 strings), each purify() call ends up calling HTMLPurifier_Config->getAll() twice. In the source code, it specifically says "This is a pretty inefficient function, avoid if you can". 863 calls to getAll() (resulting also in roughly 100k calls to php::explode) profiled at about 1.6 seconds cumulative execution. Since the Config is finalized once and never changed after, I put a quick cache on the front of the function that holds the $ret value of the function. This reduced getAll's overhead from 1.6s to about 6 milliseconds.

My other optimization is in the get() method. Every get() would result in several simple error checks, and then a recursive call to parent configs looking for the key in question. In my example, my 431 string purifications end up calling HTMLPurifier_Config->get() about 37k times, resulting in around 73k recursive calls and calls to ->has() on PropertyList. My profiles show this around 3.2s total time. Putting a small cache on the front of the get method reduces the number of recursive calls to about 5k, and the time footprint is reduced to about 329ms.

If anyone requests, I'll post the changes to my code. They're a bit hacked in and perhaps not up to HTMLPurifier standards, so maybe my overly-verbose explanation is sufficient. :)

Enjoy!

XSS against nature (1 reply)

AvenidaGez — Sun, 24 Mar 2013 17:21:01 -0400

Having years programing all this about XSS makes me sad day by day It is understandable there is no security at any matter, the car the house, anything someway or another is insecure, but getting up to this levels goes beyond my mind. Of course there are many reasons to protect. From the more difficult aspects I have known is Overflow which is related to most of all to system structure Code injection? Many years ago, I thougt there could be in the engine PHP or whatever, a prefix and/or sufix for all commands, as simple to have in php.ini related to the engine And of course, from manual to php editors handle those pre-sufixes That way there were no problem with any injection, hard to find your assigned prefixes to commands fast code, no need to clean, no need to nothing. Isn't it the same what does Java Pcode, PHP interpreter, javascript or anyother? if is not compiled is interpreted, if is interpreted you have a translation table, then could be prefixes, renames, and sufixes which I don't believe a hacker will take time to find-out For now, I need how to let a visitor let write code in comments (not functional code) with this thing that looks like antivirus which one ends turning off because at the end does not lets you work, I do not find how We need to be secure to some degree, but not living in a self sanitized jail against viruses.

Bug in 'A' tag correction. (3 replies)

Joel — Sat, 02 Mar 2013 14:55:59 -0500

Hello, I believe I've discovered a bug (in 4.5.0) in regards to correction of improperly formatted URLs.

Given the following code:

http://www.example.com>example

HTML Purifier will correct it to:

http://www.example.com">

It properly adds in the missing quotation mark, but then strangely, removes the text that is enclosed in the 'A' tag.

Thanks for making a fine product.

include path override? (1 reply)

smalyshev — Wed, 31 Oct 2012 17:40:43 -0400

I'm using HTMLPurifier.standalone.php and I see it does in its code this:

set_include_path(HTMLPURIFIER_PREFIX . PATH_SEPARATOR . get_include_path());

This puts purifier path at the beginning of the include path. However, this has bad effect inside a big application, since it means every time it includes a file, it looks in purifier directory first. Is this line really necessary? As far as I can see, files are loaded using HTMLPURIFIER_PREFIX anyway, so no reason to add anything to include path. Is there any reason to do it that I fail to see?

Multiple values possible for URI.Host (2 replies)

daniels — Fri, 29 Jun 2012 10:42:09 -0400

Hi,

I would like to use HTML Purifier with the option "HTML.TargetBlank" for example but taking in account multiples domains in the "URI.Host" option for different site languages.

Do you think about to develop this behavior?

Extracting elements (2 replies)

Vaibhav Kaushal — Mon, 30 Jul 2012 08:56:26 -0400

Hello!

I was wondering if there is a way to extract elements from the purified text. Something like:

$strPurified = $purifier->Purify($DirtyHtml); $arrElements = HTMLPurifier::Extract($strPurifier, 'a,img,b');

and then use something like this:

$strFirstLinkInText = $arrElements['a'][0];

Wouldn't that be a great addition? Since HTMLPurifier already is able to completely tear apart HTML and rejoin it, this would be a great addition for implementing some functionality on the server side which normally we should not want be done on the client side.

Regards, Vaibhav

DisplayRemoteLinkURI injector (9 replies)

tpaksu — Sat, 24 Mar 2012 23:20:27 -0400

I've created a new injector named AutoFilter.DisplayRemoteLinkURI and explained at the link below about how to add it. It works like DisplayLinkURI but just for remote URL's. Local URL's stay the same. And I couldn't figure how to use DisableExternal URIFilter inside it, so I wrote a temporary function to check if link is remote or local.

http://stackoverflow.com/a/9804323/1262700

Just wanted to inform you. If it's hacking the core and not allowed, I'd remove that.

URI.DisableResources not loading (2 replies)

Brent C — Fri, 02 Mar 2012 13:26:34 -0500

I'v been trying to get the URI.DisableResources directive to work, but setting the config directive to true didn't seem to actual change anything. Digging into the code (which I'm very unfamiliar with as this is my first time working with HTMLPurifier), it seems to me that the URI.DisableResources filter is not being loaded into the HTMLPurifier_URIDefinition class.

registerFilter(new HTMLPurifier_URIFilter_DisableExternal());
        $this->registerFilter(new HTMLPurifier_URIFilter_DisableExternalResources());
        $this->registerFilter(new HTMLPurifier_URIFilter_HostBlacklist());
        $this->registerFilter(new HTMLPurifier_URIFilter_SafeIframe());
        $this->registerFilter(new HTMLPurifier_URIFilter_MakeAbsolute());
        $this->registerFilter(new HTMLPurifier_URIFilter_Munge());
    }

Shouldn't there be a line in there for URIFilter_DisableResources?

$this->registerFilter(new HTMLPurifier_URIFilter_DisableResources());

-Brent

Deprecating trigger_error calls in the Tarot cards? (8 replies)

Yzmir Ramirez — Mon, 11 Apr 2011 15:05:16 -0400

In some environments setting up your own PHP error_handler using set_error_handler() is not an option. Have you considered converting all the trigger_error() calls to throw exceptions?

There are 51 locations that have trigger_error calls and 23 that throw *Exception in 4.2.0.

Is that a 5.x change? Or something that could be rolled into ~~4.3.x~~ 4.4.0?

AutoFormat.LinkifyEmail Option (3 replies)

bfroehle — Tue, 15 Feb 2011 13:07:40 -0500

I've added an option to automatically linkify e-mail addresses. Here's the commit message:

    Add AutoFormat.LinkifyEmail Option.
    
    If enabled, automatically provide links to e-mail addresses.
    For example,
      Send an e-mail to user@example.com.
    is transformed to
      Send an e-mail to user@example.com.
    
    Signed-off-by: Bradley M. Froehle

The code is available in the 'linkify-email' branch of http://repo.or.cz/w/htmlpurifier/bfroehle.git.

HTML.SafeIframe Option (26 replies)

bfroehle — Mon, 26 Dec 2011 08:48:11 -0500

I've attempted to extend HTMLPurifier to allow (some) iframes. Here's the commit message:

Add new HTML.SafeIframe and URI.IframeHostWhitelist options.

Many online video providers (YouTube, Vimeo) and other web applications (Google Maps, Google Calendar, etc) provide embed code in iframe format. This introduces two new settings:
HTML.SafeIframe / bool, default: FALSE

Whether or not to display iframe content.

URI.IframeHostWhitelist / list, default: array()

A list of whitelisted hosts for iframe content. Iframes are allowed only if the host explicitly matches an element of this array.

The code is available in the 'iframe' branch of http://repo.or.cz/w/htmlpurifier/bfroehle.git.

Cache directory permissions setting (8 replies)

skodak — Thu, 13 Jan 2011 21:33:33 -0500

Hello,

at present the cache directory permissions are hardcoded to be 0755, sometimes it is useful to specify custom permissions (executing PHP from two different user accounts, phpunit, safe mode, etc.).

I have hacked the standard HTML Purifier to use custom permissions for the Moodle project - https://github.com/moodle/custom-htmlpurifier/compare/882ffed9babb9ddc20bfb0979b14bb52d64c96c4...MOODLE_20_STABLE

Is this interesting? Could I do anything to get something similar included in HTML Purifier?

Thanks for all your hard work on this library!

Petr

Iterative traversal of DOM [patch] (7 replies)

Darhazer — Wed, 19 Jan 2011 17:10:38 -0500

Hi,

Since there are some deep DOMs you can hit the maximum nesting level limit in tokenizeDOM (we've experienced this even with maximum nesting level of 300). So here is an iterative version of the same function with simple quoue/dequoue approach. I was going to upload a .patch file, but no file upload is permitted.

NOTE: createStartNode() and createEndNode() are helper functions (actually parts of the old body of tokenizeDOM) and do not exists in the original code. You have to replace the original tokenizeDOM() function with these 3 functions:

/**
 * Iterative function that tokenizes a node, putting it into an accumulator.
 * To iterate is human, to recurse divine - L. Peter Deutsch
 * @param $node     DOMNode to be tokenized.
 * @param $tokens   Array-list of already tokenized tokens.
 * @returns Tokens of node appended to previously passed tokens.
 */
protected function tokenizeDOM($node, &$tokens) {

	$level = 0;
	$nodes = array($level => array($node));
	$closingNodes = array();
	do {
		while (!empty($nodes[$level])) {
			$node = array_shift($nodes[$level]); // FIFO
			$collect = $level > 0 ? true : false;
			$needEndingTag = $this->createStartNode($node, $tokens, $collect);
			if ($needEndingTag) {
				$closingNodes[$level][] = $node;
			}
			if ($node->childNodes && $node->childNodes->length) {
				$level++;
				$nodes[$level] = array();
				foreach ($node->childNodes as $childNode) {
					array_push($nodes[$level], $childNode);
				}
			}
		}
		$level--;
		if ($level && isset($closingNodes[$level])) {
			while($node = array_pop($closingNodes[$level])) {
				$this->createEndNode($node, $tokens);
			}
		}
	} while ($level > 0);
}

/**
 * @param $node  DOMNode to be tokenized.
 * @param $tokens   Array-list of already tokenized tokens.
 * @param $collect  Says whether or start and close are collected, set to
 *					false at first recursion because it's the implicit DIV
 *					tag you're dealing with.
 * @returns bool if the token needs an endtoken
 */
protected function createStartNode($node, &$tokens, $collect)
{
	// intercept non element nodes. WE MUST catch all of them,
	// but we're not getting the character reference nodes because
	// those should have been preprocessed
	if ($node->nodeType === XML_TEXT_NODE) {
		$tokens[] = $this->factory->createText($node->data);
		return false;
	} elseif ($node->nodeType === XML_CDATA_SECTION_NODE) {
		// undo libxml's special treatment of