<br><br><div class="gmail_quote">On Fri, Jan 25, 2013 at 4:13 AM, IZ3GME Marco <span dir="ltr"><<a href="mailto:iz3gme.marco@gmail.com" target="_blank">iz3gme.marco@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
It's a rounding problem due to the internal representation of float.<br>
<br>
I tried the following version of convert_freq_to_bytes which seems to<br>
works but may be Dan or some other python expert have a better solution.<br>
<br>
def convert_freq_to_bytes(freq):<br>
<div class="im"> bytes = [ 0 for x in range(0,8) ] # init list with 8 times 0<br>
</div> real_freq = "%3.5f" % freq # now is a string<br>
real_freq = int(real_freq[0:3]+ real_freq[4:])<br>
# and now an int<br>
<div class="im"> for i in range(7, -1, -1): # go from 7 to 0<br>
bytes[i] = real_freq%10 # extract last digit<br>
real_freq /= 10 # throw away last digit<br>
return bytes<br>
<br>
</div>73 de IZ3GME Marco<br>
<div class="im HOEnZb"><br></div></blockquote><div>I still get 146.63999 out for 146.64000 in.<br><br>In settings.py, I had changed 'precision=4' to 'precision=5'<br><br>class RadioSettingValueFloat(RadioSettingValue):<br>
"""A floating-point setting"""<br> def __init__(self, minval, maxval, current, resolution=0.001, precision=4):<br> <br>The reason I did this was to get 5 places past the decimal. Maybe that was the wrong way to accomplish that. Anyway, I changed back to 'precision=4' and now the result is 146.6400, which is better than 146.63999, but still not exactly what I am looking for.<br>
<br>Thanks again. You help and guidance is greatly appreciated.<br>Jim KC9HI<br><br><br></div></div>