ΣSigmaStudio Ask AI Русский Log in

Error in ADAU1542 datasheet Numeric formats

SigmaDSP & SigmaStudio alexsconway Mar 12, 2021 00:00 1 replies Original on forum ↗
Subscribe
alexsconway Question
Mar 12, 2021 00:00

On page 89 (rev D), there is a table enumerating examples of 8.24 values and their equivalent in scientific notation.

There are two errors:

1. 1LSB below 0.0 should be -0.0000000596046

2. 1LSB above 0.0 should be +0.0000000596046

And in the cases of -0.1 and 0.1, maybe it should be made clear that the values printed cannot be exemplars becasue they fall foul of both floating point inaccuracies and round-off errors when converting to and from formats with differing fractional resolutions

Alex

Audio Signal Processors

adau1452

numeric

microcontroller

fixed point

SigmaDSP Audio Processors

Show More

KJBob Reply
Mar 14, 2021 00:00

Hello Alex,

Yes you're correct -- the LSB is 2^(-24) = 0.0000000596.... The datasheet truncated (not rounded) it to only one significant figure. At least they got the seven zeros after the decimal point correct, and this LSB value is about -144.5 dB below full scale anyhow.

Best regards,

Bob