Test failures on Mac

Here you go...

All HTML Purifier tests on PHP 5.2.6
1) Equal expectation fails at character 0 with [Invalid encoding utf8] and [] -> Expected error not caught
        in test_convertToUTF8_spuriousEncoding
        in HTMLPurifier_EncoderTest
2) Identical expectation [Integer: 1] fails with [Integer: -1] because [Integer: 1] differs from [Integer: -1] by 2 at [/Users/dgm/workspace/htmlpurifier/tests/HTMLPurifier/LengthTest.php line 68]
        in testCompareTo
        in HTMLPurifier_LengthTest
FAILURES!!!
Test cases run: 195/195, Passes: 2426, Failures: 2, Exceptions: 0

I'm testing on a mac, fwiw.

Ok, so I'm inclined to believe that the first error is just a bad test-case, although I don't know why your case complains and mine doesn't; well, I do, sort of: your version of iconv somehow supports the "utf8" identifier as UTF-8. Maybe they fixed that in a new version. Anyway, say bye bye to that test.

As for the second one, it looks like we've got an integer overflow. Are you on a 32-bit or 64-bit architecture?

I believe it's 32 bit... Intel Core Duo, Macbook Pro.

Hmm.

So this is reaaally difficult to test, mainly because I don't know what the system differences are. I can make a guess, however, as to the reason. Patch forthcoming.

when I change the str_repeat to 3, it works but 4 fails: str_repeat('2', 4)

BTW, I feel your pain. The first time someone tried running Maia Mailguard on a mac server, we had all sorts of problems.. for one thing, the file system is case insensitive, and we had a naming conflict between a config.php and Config.php in another include path. That was before I bought this laptop. :)

... Weeird. Would you mind putting a probe in HTMLPurifier/Length.php compareTo and checking out what $l, (float) $l, (float) $n are?

Adding probes like this:

    public function compareTo($l) {
        var_dump((float)$l->n);
        var_dump((float)$this->n);
        if ($l === false) return false;
        if ($l->unit !== $this->unit) {
            $converter = new HTMLPurifier_UnitConverter();
            $l = $converter->convert($l, $this->unit);
            if ($l === false) return false;
        }
        var_dump((float)$this->n);
        var_dump((float)$l->n);
        return $this->n - $l->n;
    }

yields (for the failed test):

float(2.2222222222222E+37)
float(100)
float(100)
float(1)

If I take out the float casts, I get:

string(38) "22222222222222222222222222222222222222"
string(3) "100"
string(3) "100"
string(2) "1f"

That 1f looks funny. ;)

Yeah... that's probably the culprit. :-)

On my computer, I get "2133333333333333333333333333333333333300", so things work out correctly. Checking UnitConverter now.

It looks like there's a bug with sprintf(). Go to UnitConverter.php, line 237. I'd like to know what the value of '%.' . $scale . 'f' is.

 private function scale($r, $scale) {
        var_dump('%.' . $scale . 'f');
        var_dump(sprintf('%.' . $scale . 'f', (float) $r));
        return sprintf('%.' . $scale . 'f', (float) $r);
    }

The last two runs before the assert error:

string(5) "%.10f"
string(51) "2133333333333333285307942900395392630784.0000000000"
string(5) "%.-1f"
string(2) "1f"

BTW, the utf8 one is no longer barking at me.

Maybe this would help, I generated a call stack for when it gets to that -1 scale:

Call Stack:
    0.0009   1. {main}() /Users/dgm/workspace/htmlpurifier/tests/index.php:0
    3.4522   2. TestSuite->run() /Users/dgm/workspace/htmlpurifier/tests/index.php:175
    8.9501   3. SimpleTestCase->run() /Users/dgm/workspace/simpletest/test_case.php:557
    8.9694   4. SimpleExceptionTrappingInvoker->invoke() /Users/dgm/workspace/simpletest/test_case.php:144
    8.9695   5. SimpleInvokerDecorator->invoke() /Users/dgm/workspace/simpletest/exceptions.php:43
    8.9695   6. SimpleErrorTrappingInvoker->invoke() /Users/dgm/workspace/simpletest/invoker.php:126
    8.9695   7. SimpleInvokerDecorator->invoke() /Users/dgm/workspace/simpletest/errors.php:42
    8.9695   8. SimpleInvoker->invoke() /Users/dgm/workspace/simpletest/invoker.php:126
    8.9697   9. HTMLPurifier_LengthTest->testCompareTo() /Users/dgm/workspace/simpletest/invoker.php:68
    8.9738  10. HTMLPurifier_LengthTest->assertComparison() /Users/dgm/workspace/htmlpurifier/tests/HTMLPurifier/LengthTest.php:68
    8.9741  11. HTMLPurifier_Length->compareTo() /Users/dgm/workspace/htmlpurifier/tests/HTMLPurifier/LengthTest.php:59
    8.9741  12. HTMLPurifier_UnitConverter->convert() /Users/dgm/workspace/htmlpurifier/library/HTMLPurifier/Length.php:107
    8.9743  13. HTMLPurifier_UnitConverter->round() /Users/dgm/workspace/htmlpurifier/library/HTMLPurifier/UnitConverter.php:161
    8.9743  14. HTMLPurifier_UnitConverter->scale() /Users/dgm/workspace/htmlpurifier/library/HTMLPurifier/UnitConverter.php:229

So, $scale shouldn't be negative. Can you var_dump round() with prejudice? Specifically, I'm interested in $rp, $n and $new_log.

I figured as much, and I tried that last night, but I got lost in all the variables. I guess I don't understand what it's calculating. I do seem to remember a $log variable that came up with 37, which was less precision I'll see if I can trace it out better.

My math is a bit rusty, could you describe what is supposed to be happening in round and scale?

Ok!

UnitConverter is designed to account properly for very large numbers. In order to do this most effectively, we need bcmath(); in the even bcmath is not available, however, we can still cludge things with less precise floating point arithmetic, since we generally don't deal with 10³⁰⁸ inches, or anything like that. However, since we can't trust the digits near the far right of a floating point number, scale() converts it into a string representation with an appropriate amount of real digits, and then zero padding the rest of the way.

However, simply truncating the number is not enough: we should also be rounding appropriately, so that 1.7 with one significant digit becomes 2, not 1. So the round() function performs this with both arbitrary precision (bcmath) and floating point precision.

Ok, this will be a bit long. I put a var_dump after each operation in round. I was a bit confused until I realized the last line in round called the php builtin round, not a recursive call to itself. whew!

    private function round($n, $sigfigs) {
        var_dump(array("entering round", $n, $sigfigs));
        $new_log = (int) floor(log(abs($n), 10)); // Number of digits left of decimal - 1
        var_dump(array("new_log calculated:", $new_log));
        $rp = $sigfigs - $new_log - 1; // Number of decimal places needed
        var_dump(array("rp calculated:", $rp));
        $neg = $n < 0 ? '-' : ''; // Negative sign
        var_dump(array("bcmath:",$this->bcmath));
        if ($this->bcmath) {
            if ($rp >= 0) {
                $n = bcadd($n, $neg . '0.' .  str_repeat('0', $rp) . '5', $rp + 1);
                $n = bcdiv($n, '1', $rp);
            } else {
                // This algorithm partially depends on the standardized
                // form of numbers that comes out of bcmath.
                $n = bcadd($n, $neg . '5' . str_repeat('0', $new_log - $sigfigs), 0);
                $n = substr($n, 0, $sigfigs + strlen($neg)) . str_repeat('0', $new_log - $sigfigs + 1);
            }
            return $n;
        } else {
            var_dump(array("scale called with round:", $n, $sigfigs, $new_log, $rp,"result of round:",round($n, $sigfigs - $new_log - 1)));
            return $this->scale(round($n, $sigfigs - $new_log - 1), $rp + 1);
        }
    }

string(13) "starting test"
array(3) {
  [0]=>
  string(14) "entering round"
  [1]=>
  string(12) "1.0416666700"
  [2]=>
  int(4)
}
array(2) {
  [0]=>
  string(19) "new_log calculated:"
  [1]=>
  int(0)
}
array(2) {
  [0]=>
  string(14) "rp calculated:"
  [1]=>
  int(3)
}
array(2) {
  [0]=>
  string(7) "bcmath:"
  [1]=>
  bool(false)
}
array(7) {
  [0]=>
  string(24) "scale called with round:"
  [1]=>
  string(12) "1.0416666700"
  [2]=>
  int(4)
  [3]=>
  int(0)
  [4]=>
  int(3)
  [5]=>
  string(16) "result of round:"
  [6]=>
  float(1.042)
}
array(3) {
  [0]=>
  string(14) "entering round"
  [1]=>
  string(51) "2133333333333333285307942900395392630784.0000000000"
  [2]=>
  int(38)
}
array(2) {
  [0]=>
  string(19) "new_log calculated:"
  [1]=>
  int(39)
}
array(2) {
  [0]=>
  string(14) "rp calculated:"
  [1]=>
  int(-2)
}
array(2) {
  [0]=>
  string(7) "bcmath:"
  [1]=>
  bool(false)
}
array(7) {
  [0]=>
  string(24) "scale called with round:"
  [1]=>
  string(51) "2133333333333333285307942900395392630784.0000000000"
  [2]=>
  int(38)
  [3]=>
  int(39)
  [4]=>
  int(-2)
  [5]=>
  string(16) "result of round:"
  [6]=>
  float(2.13333333333E+39)
}

The most recent commit may interest you. Please check it out!

Ok, so a negative scale is valid then? I was looking for the negative factor being an error. All tests pass now. :)

In the process, I managed to discover xdebug and a gui debugger, and was able to step through the program. A worthwhile experience I suppose. :)

Hooray!

Yep, a negative scale is valid; it simply means to the left of the decimal point. Btw, XDebug is a really awesome debugger, so by all means take advantage of it. What gui are you using?

I just discovered MacGDBp. http://www.bluestatic.org/software/macgdbp/index.php

Meh, Mac only. :-) I'm on Windows.