[chirp_devel] def convert_freq_to_bytes(real_freq):

[Jim Unroe]

On Fri, Jan 25, 2013 at 4:13 AM, IZ3GME Marco <iz3gme.marco at gmail.com>wrote:

> It's a rounding problem due to the internal representation of float.
>
> I tried the following version of convert_freq_to_bytes which seems to
> works but may be Dan or some other python expert have a better solution.
>
>          def convert_freq_to_bytes(freq):
>              bytes = [ 0 for x in range(0,8) ]   # init list with 8 times 0
>              real_freq = "%3.5f" % freq         # now is a string
>              real_freq = int(real_freq[0:3]+ real_freq[4:])
>                                                 # and now an int
>              for i in range(7, -1, -1):          # go from 7 to 0
>                  bytes[i] = real_freq%10         # extract last digit
>                  real_freq /= 10                 # throw away last digit
>              return bytes
>
> 73 de IZ3GME Marco
>
> I still get 146.63999 out for 146.64000 in.

In settings.py, I had changed 'precision=4' to 'precision=5'

class RadioSettingValueFloat(RadioSettingValue):
    """A floating-point setting"""
    def __init__(self, minval, maxval, current, resolution=0.001,
precision=4):

The reason I did this was to get 5 places past the decimal. Maybe that was
the wrong way to accomplish that. Anyway, I changed back to 'precision=4'
and now the result is 146.6400, which is better than 146.63999, but still
not exactly what I am looking for.

Thanks again. You help and guidance is greatly appreciated.
Jim KC9HI
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://intrepid.danplanet.com/pipermail/chirp_devel/attachments/20130125/fd27acbe/attachment-0001.html