[chirp_devel] any Chirp developers in NJ, NY, or PA?

Thu Jan 4 13:06:49 PST 2018

Hi Rob, welcome to the world of reverse engineering techniques in python.

/Grab a cup of coffe/tea/soda as this is a medium/log sized email. /

A Chirp's dev as a elmer in the "next block" is a pure gold mine, but as 
this is a rare case I suggest you to dig on this:

  * Get comfortable with python language, its learning curve is one of
    the most pleasant and fast I have found

I learned python building a driver for the Feidaxin FD-268A 
(https://chirp.danplanet.com/issues/2169) and two years after that (now) 
I have supported a few other models of Feidaxin, the first wave of 
B-TECHs and its long clone family (remotely with the great help of Jim 
Unroe), The Kenwood Family 60 and 60G and recently the Baofeng BF-T1 
(with help of the users https://chirp.danplanet.com/issues/4933)

When you get the swing and you have the help of the users you can do 
drivers without touching the radio it self! for example the B-TECHs and 
the recent Baofeng BF-T1.

The best learning material is the Chirp code base, you will have buch of 
examples to play with.

Review the Chirp's wiki page about developing a new model, there you 
will learn how to setup a correct environment to start, an a few trick more.

Google is your friend, ask anything about python & you will get a few 
dozens of hits for every question you make.

  * Get some background on some important things you will need, as usual
    google is your friend.

You need to understand some concepts like (to start):

      * Numeric representations y different bases (hexadecimal and bit
        tinkering)
      * Boolean and arithmetic operations (and, or, xor and it's
        combinations)
      * String and numeric representation in python, including
        conversions between formats.
      * Data encoding: Little/Big endian, BCD, etc.
      * Serial communications.
      * Regular expression matching (for the serial capture log cleaning)

My roadmap for someone that start to develop is like this (very similar 
to other suggestions)

  * Get a already supported & cheap radio, and do a reverse engineering
    by yourself, use a simple radio if possible (not a driver with
    multiple variants) use the already made driver as a cheat sheet, but
    not too much... don't be surprised to find bugs or to add new
    features not supported by the original developer)
  * Let me suggest you an inexpensive and easy radio for this: Baofeng
    BF-T1.
  * Then go to a more complicated model to understand the concept of
    supporting multiple radios on a single driver and a few more new
    techniques. ( a few candidates for this stage: Baofeng UV-5R, B-TECH)
  * Very important point: look and learn from your radio family, the one
    you pretend to support (kenwood, yaesu, baofeng, etc.) All families
    has a very similar logic and structure.
  * Now got your hands dirty with the dev stage.

The main dev steps are this (at least for me):

1 - Get a serial capture log for the download and upload processes.

Go to http://chirp.danplanet.com/attachments/2257/how%20to%20portmon.doc 
and learn how to capture a log, if you don't has a Windows at 32 bits 
then use other port serial monitors, free alternatives are usbpcap and 
wireshark (there are some freeware tools out there too, google "serial 
monitor eltima") all comments on the settings for the capture about 
logging time and hex + ascii are valid.

In this point a good editor text editor (geany, notepad++, gedit) with 
regular expression support will help you to "clean" and get a log with 
just the needed data and no more comments and notices...

2 - Analyze the serial stream logs, you need to figure out the serial 
details (baudrate, transfer bits, parity & stop bits) and more

Usually the radios has a magic string and then a ident string, not all 
works the same, Yaesu has a key combinations to send/capture data, etc...

The magic string is the string you send to the radio to put it on clone 
mode, usually they answer back with a fer chars.
The ident string is a special request to retrieve radio identifications 
as full model and some other useful info (band data, options installed, 
firmware version, etc.)

3 - Figure out the clone logic.

Usually you has to send a crafted string of a few bytes with a command, 
an address to start to read and the amount of data you want; you will 
identify it easily as they repeat itself incrementing the address part 
of the string.

Following this request the radio answer with a block of data that 
may/may not contain a kind of echo of the request and then the data 
payload you want to read.

You can search the drivers already in chirp to identify similar drivers 
you can reuse for that and to understand the clone logic.

4 - Build a skeleton of the driver for only download and radio image.

In this step you can reuse a similar driver, concentrate into download 
an image.

5 - Get upload support.

Do step 3 but for the upload process, build the upload procedures, test 
and validate.

At this point you are wondering if writing an incorrect data stream will 
damage the radio... for this there are 50% chance of brick your radio 
for some models, but that's 100% recoverable:

Before playing with upload get a copy of your radio mem (codeplug) with 
your OEM software; if you brick your radio, simply power cycle it and 
upload the codeplug with the OEM software. Done, you are set again to play.

Note that we are not changing a bit of the radio memory yet, so download 
and upload must not affect the functioning of the radio.

6 - Reverse-engineer the memory data.

If you are working in Linux (which I encourage to do so) you have a few 
tools at disposal, "hd" command utility is your best friend and the 
hexdiff script is a joy 
(https://chirp.danplanet.com/projects/chirp/wiki/DevelopersAdd_a_Radio)

Follow this steps: (use a paper and pencil or a digital notepad to write 
down your findings if you are not familiar yet with MEM-FORMAT)

6.1 OEM soft: program just one channel with a simplex freq on it, save 
to radio
6.2 Chirp: read image, save as base.img
6.3 OEM soft: same channel, change the RX freq, save to radio
6.4 Chirp: read image as new
6.5 Use hexdiff to compare images... now you have found where it store 
channel #1.

Using that algorithm determine channel structure and data:

  * TX and Rx are stored as is or base +/- offset
  * What about Rx tones, Digital tone, inverted tone?
  * Same for Tx tone...
  * Repeat with every other channel property (scan/skip, power, name)

And go for a mid and then the final (last channel) to determine 
boundaries of the channel data... then learn

Now see some other drivers to learn/improve how to represent your 
findings on MEM_FORMAT language...

Don't forget to COMMENT EVERYTHING that is not obvious in the code reading,

7 - Build the memory parse functions

Use other drivers to represent your MEM_FORMAT in channel data for the 
UI. At this point you will get chirp represent your data in a visual 
way, polish and validate it, test the limits of data and validate for that.

8 - Re-do point 6 but for Settings....

Learn how to represent them from other drivers code.

9 - Now try to validate the driver against the Chirp's test bed, with 
run_test and/or tox...

You will find more and more errors in your code, validations are need 
here and there, prep up your code...

Test it extensively against your hardware...

10 - Submit a patch.

See the wiki to know how to deal with mercurial.

Every radio is different, you will even find bugs in serial 
communication (first ident try result in a bad checksum) etc, the hell 
is in the details and you need to triage them to find the solution.

If you are unlucky you will need to make yourself a new way, like a OM 
recently did in finding the brick of the encryption schema in the Wouxun 
KG-UC8D Plus, and opening a door for me too develop a driver for the 
KG-UV8E and T and possible for the UV9D Plus also. 
(https://chirp.danplanet.com/issues/3943#change-15199)

This is my 10c contribution, don't give up. Ask me for help any time you 
want (direct mail or via the list), I'm not in the "next block" or 
"around the corner" but I will help you as much as is possible to me. 
I'm not 100% of the time online, as Internet is a scarce and expensive 
resource here in Cuba, so you may wait for and answer from me, I try to 
connect at least twice a day to check when an active development is on 
going.

Finally take a time to see this video to learn a few tips to deal with 
an Open Source Community with his soft and rough edges: 
https://www.youtube.com/watch?v=7XTHdcmjenI

73 Pavel CO7WT...

El 04/01/18 a las 12:39, Rob Owens via chirp_devel escribió:
> I now have three radios that are not supported by Chirp.  Nobody has
> stepped up to add support, so I'm trying to teach myself how to do it.
> But I'm sure my learning curve would be much quicker if I had somebody
> experienced to work with for a day or so.
>
> Is there anybody near northwestern New Jersey who would be willing to
> help me get started?  I'm not looking for someone to do the work for me,
> just somebody to help remove the cloud of uncertainty I'm stuck in.
>
> -Rob
> _______________________________________________
> chirp_devel mailing list
> chirp_devel at intrepid.danplanet.com
> http://intrepid.danplanet.com/mailman/listinfo/chirp_devel
> Developer docs: http://chirp.danplanet.com/projects/chirp/wiki/Developers

-- 
73 CO7WT, Pavel.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://intrepid.danplanet.com/pipermail/chirp_devel/attachments/20180104/4f3f3c41/attachment-0001.html 