Sets a tolerance such that if two numbers differ by less than the precision (exclusive), they will be considered equal. Specifically, n1 == n2 if Abs (n1 - n2) < p, where p is the precision.
The default precision is 0.000001.
n1 == n2 iff abs(n1-n2) + DBL_EPSILON <= precisionwhere DBL_EPSILON is the smallest number such that 1.0+DBL_EPSILON != 1.0. It is typically defined as: 2.2204460492503131e-016.
To normalize the comparison, apply a suitable multiplier to the precision and to the values being compared. The multiplier should be based on 1e-15, which is the first power of 10 greater than DBL_EPSILON. The example below demonstrates the use of such a multiplier.
[ ] const MIN_PRECISION = 1e-15 // first power of 10 greater than DBL_EPSILON
[ ]
[-] testcase Example () appstate none
[-] LIST OF NUMBER ln1 = {...}
[ ] 1
[ ] .045
[ ] .04
[ ] 1.0e-20
[ ] 1.0e-20
[-] LIST OF NUMBER ln2 = {...}
[ ] 4
[ ] .04
[ ] .05
[ ] 1.1e-20
[ ] 1.1e-20
[-] LIST OF REAL lrPrecisions = {...}
[ ] 10
[ ] .01
[ ] .01
[ ] 0.00000000000000000001
[ ] 1.0e-21
[ ] REAL r1, r2, rPrecision, rMultiplier
[ ] INTEGER i
[ ]
[-] for i = 1 to ListCount (ln1)
[-] if i <= 3
[ ] Print ("Comparing {ln1[i]} to {ln2[i]} with precision {lrPrecisions[i]}")
[-] else
[ ] Printf ("Comparing %1.24f to %1.24f with precision %1.24f\n", ln1[i], ln2[i], lrPrecisions[i])
[ ] r1 = ln1[i] // will implicitly convert INTEGER to REAL
[ ] r2 = ln2[i] // will implicitly convert INTEGER to REAL
[ ] rPrecision = lrPrecisions[i]
[ ] // If necessary, normalize the comparison.
[ ] // If omitted, the fourth comparison (1.0e-20 to 1.1e-20) will fail.
[ ] rMultiplier = MIN_PRECISION / rPrecision
[-] if rMultiplier > 1.0
[ ] r1 *= rMultiplier
[ ] r2 *= rMultiplier
[ ] rPrecision *= rMultiplier
[ ] SetPrecision (rPrecision)
[ ] // The following verification should raise an exception
[ ] // for the third comparison (.04 to .05 with precision .01)
[ ] // and fifth comparison (1.0e-20 to 1.1e-20 with precision 1.0e-21)
[-] do
[ ] Verify (r1, r2, "the numbers match within the precision")
[ ] // Could also use:
[ ] // Verify (r1 == r2, TRUE, "the numbers match within the precision")
[-] except
[ ] LogError (ExceptData ())
The results file is:
Script OnLineHelpExample[1].t - 2 errors Machine: (local) Started: 03:49:33PM on 08-Nov-2006 Elapsed: 0:00:06 Passed: 0 tests (0%) Failed: 1 test (100%) Totals: 1 test, 2 errors, 0 warnings Testcase Example - 2 errors Comparing 1 to 4 with precision 10.000000 Comparing 0.045000 to 0.040000 with precision 0.010000 Comparing 0.040000 to 0.050000 with precision 0.010000 *** Error: Verify the numbers match within the precision failed - got 0.040000, expected 0.050000 Comparing 0.000000000000000000010000 to 0.000000000000000000011000 with precision 0.000000000000000000010000 Comparing 0.000000000000000000010000 to 0.000000000000000000011000 with precision 0.000000000000000000001000 *** Error: Verify the numbers match within the precision failed - got 0.000000, expected 0.000000