LoginLogin
Nintendo shutting down 3DS + Wii U online services, see our post

LOG function incorrect result

Root / Programming Questions / [.]

NeatNitCreated:
This took me a few hours to catch...
? LOG(&H20000000, 2)
29
? CEIL(LOG(&H20000000, 2))
30
The result should be 29 EXACTLY, not 29 + ½⁴⁸ as it spits out right now. I'm guessing this is a rounding error from doing log(x)/log(2), but it broke my code big time. By the way... Is there a better way, given an integer that I know has just one bit on (i.e. is a power of 2), to find which bit it is? I'm looking for execution speed. Currently I'm doing it in a big if-elseif-elseif... thing.

You could try something silly with BIN$ and INSTR.
?31-INSTR(BIN$(&H20000000,32),"1")
29
It's shorter but I don't know if it meets your criteria for speed. It also works with the sign bit, if you need that I guess. EDIT: why not FLOOR the result instead of CEIL if the rounding error is an issue? Are you also dealing with negatives (somehow?) You can try using OR 0 to coerce the result into an integer instead of using a rounding function. This has the effect of just truncating the decimals, but obviously will break on really large numbers (but you'll never have anything greater than 31, will you?)
?CEIL(LOG(&H20000000,2) OR 0)
29
The type conversion will have an effect on speed, but whether that's better or worse than you have now I of course don't know.

The INSTR$ method is creative :o The reason this was a problem is precisely because I'm covering to integers, but I was using (31 - LOG(M, 2)) (where M is my bitmask), so truncating that got rid of most of a 1. Honestly I regret using big endian for what I'm doing, it was nice to type into &B format but the math turned out to be much more annoying to deal with. Bottom line though, I'm probably just gonna leave it with the giant stack of ELSEIF it has right now. I haven't tested very thorough but I have a very strong hunch that it's as optimized as it can get. SB does have some sort of compilation, right?

What are you even trying to do anyway?

What are you even trying to do anyway?
Project Euler - https://projecteuler.net I'm making myself a utility library, and I'm handling prime numbers. I'm using the sieve method https://en.m.wikipedia.org/wiki/Sieve_of_Eratosthenes to find all prime up to a limit, and for each number I have to store whether it's prime or not prime. Originally I just had an array with every element being true or false, but that is a lot of wasted memory. So now I made it a bitmask, where each bit represents whether the number corresponding to that bit is prime. So I have to do the math to convert back and forth from this format.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
BIT%=INTEGER% AND 1<<INDEX%
The order of operations wasn't chosen randomly, you know.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
But since we already know which bit we want, wouldn’t we just want a 1 or 0, rather than a power of 2?

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
But since we already know which bit we want, wouldn’t we just want a 1 or 0, rather than a power of 2?
&& only checks that both numbers aren't 0, you need AND to make sure the bits match.

To use that I'd have to test that in a loop until I hit the right one. AFAIK (haven't actually tested), a big hardcoded ELSEIF thing is faster to execute:
'OPTION DEFINT at the top
VAR M ' the mask
VAR O ' the result
IF     M == &H80000000 THEN O = 1
ELSEIF M == &H40000000 THEN O = 2
ELSEIF M == &H20000000 THEN O = 3
' ...
ELSEIF M == &H00000004 THEN O = 30
ELSEIF M == &H00000002 THEN O = 31
ELSEIF M == &H00000001 THEN O = 32
I'm using a variation of that. Note that it's technically all one line, with word wrap turned on. Edit 2017-12-22: changed AND to ==, since that's how my code is

Why write it on one line? This isn't PTC where writing all your code without whitespace is faster.

Because it looks much nicer. No line break after THEN and no ENDIF.

He's doing the silly thing where you add extra spaces to make the code line up with line wrap enabled.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
BIT%=INTEGER% AND 1<<INDEX%
The order of operations wasn't chosen randomly, you know.
(a bit of a late reply) When in doubt, use parentheses. Even if the order of operations does the exact same thing, the human reading the code might not remember that. Adding the parentheses means there's no chance of confusion - this part happens first.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
BIT%=INTEGER% AND 1<<INDEX%
The order of operations wasn't chosen randomly, you know.
(a bit of a late reply) When in doubt, use parentheses. Even if the order of operations does the exact same thing, the human reading the code might not remember that. Adding the parentheses means there's no chance of confusion - this part happens first.
Except when your screen is only 46(?) characters wide, and you want to avoid line wrapping (or worse).

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
But since we already know which bit we want, wouldn’t we just want a 1 or 0, rather than a power of 2?
&& only checks that both numbers aren't 0, you need AND to make sure the bits match.
I’m still trying to learn: is my version fine for checking one bit of data? AND gates still just check for two high inputs, whether we are ANDing the entire value or just one bit.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
But since we already know which bit we want, wouldn’t we just want a 1 or 0, rather than a power of 2?
&& only checks that both numbers aren't 0, you need AND to make sure the bits match.
I’m still trying to learn: is my version fine for checking one bit of data? AND gates still just check for two high inputs, whether we are ANDing the entire value or just one bit.
&B10 AND &B01 => 0 &B10 && &B01 => 1 AND is bitwise, && is Boolean. Huge difference. I suggest you toy around with it, read up on it, etc, and if anything is unclear ask away.

You can check the state of a bit with this:
BIT% = INTEGER% && POW(2,INDEX%)
The use of && here is incorrect, it should be bitwise AND. I'd do it like this:
BIT%=INTEGER% AND (1<<INDEX%)
But since we already know which bit we want, wouldn’t we just want a 1 or 0, rather than a power of 2?
&& only checks that both numbers aren't 0, you need AND to make sure the bits match.
I’m still trying to learn: is my version fine for checking one bit of data? AND gates still just check for two high inputs, whether we are ANDing the entire value or just one bit.
&B10 AND &B01 => 0 &B10 && &B01 => 1 AND is bitwise, && is Boolean. Huge difference. I suggest you toy around with it, read up on it, etc, and if anything is unclear ask away.
Ahhh, my bad. I sure will be fiddling around with them. Thanks for the explaination!

I'm moving this to Programming Questions because the bug is just a rounding error, and most of the discussion is about alternative methods, not the "bug" itself.