Jump to content


Photo

Restarting PipXtream Firmware Development


  • Please log in to reply
324 replies to this topic

#1 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 16 February 2012 - 07:54 PM

With the imminent release of prototype hardware into the wild, it's time to (re)start discussion of the firmware. Please add any information you have on what the current state of the firmware is, and what ideas we might have for improving it.

Feature wish list:
  • Relay a PPM signal between two modems.
    • Needs to take priority over other traffic
    • Might need additional error correction to help ensure that very few packets are lost
  • Error correction over the wireless link
  • RSSI indication
    • Displayed in GCS and on OSD
  • Congestion indication
    • Displayed in GCS and on OSD
PipXtreme Firmware V2.0-alpha1 (4936cd8fd2f12dc247af4295737f4efdcec6d377)
  • Upgraded to use PIOS interface and FreeRTOS
  • Working RFM22B radio driver (using interrupts)
  • Reed-Solomon error corection
  • Single connectionless link (broadcast only, no pairing supported)
  • 57600 baud serial link, 115200 baud wireless link
Attached File  fw_pipxtreme-2.0-alpha1.opfw   53.4K   4 downloads

PipXtreme Firmware V2.0-alpha2 (ed1937958e3ceed1790b583e1e5c59fd83d7f227)
  • Added watchdog timeout
  • Added RTC supervisor to RFM22B driver to restart transfers that lockup
  • Significant start at configuration GUI.
Attached File  fw_pipxtreme-2.0-alpha2.opfw   54.84K   1 downloads


PipXtreme Firmware V2.0-alpha2 (7082eda17526462539d81cb34abb627ad856b625)
  • Improved transfer rate and error rate
  • Settings saving to flash
  • Near complete status and configuration GUI
Configuration GUI:

Attached File  CropperCapture1.png   59.73K   17 downloads

Attached File  fw_pipxtreme-2.0-alpha3.opfw   60.67K   3 downloads

#2 XXL-Wing

XXL-Wing

    Lead Developer

  • Members
  • PipPipPip
  • 200 posts
  • LocationNiederglatt
  • Country: flag of Switzerland Switzerland

Posted 17 February 2012 - 06:02 AM

count me in on the developer list.
just placing my order now :-)

for me the features would also include:

* RSSI and medium congestion Indication to user
* rolling window including sync bits
* use a time slot based approach together with the rest time used for non-critical data (this would also allow multiple modems in one communication)

for ECC i would prefer Reed-Solomon, this proved very reliable and fast for our UAV guys in the UK.
No matter how good your backend systems are, the users will only remember your front end. Fail there and you will fail, period. -- Tristan Louis
Don't make me think. -- Steve Krug, usability expert

#3 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 17 February 2012 - 02:02 PM

I found this implementation of Reed-Solomon coding. It is GPL, and it's not very much code. Anybody see any reason that we shouldn't try it?

#4 XXL-Wing

XXL-Wing

    Lead Developer

  • Members
  • PipPipPip
  • 200 posts
  • LocationNiederglatt
  • Country: flag of Switzerland Switzerland

Posted 17 February 2012 - 05:27 PM

we should, definitely.

anyone having any idea if there is already some usable pipX code or we should start from scratch?

i would suggest to build a fully time-triggered system which is perfect for such transmission things.
No matter how good your backend systems are, the users will only remember your front end. Fail there and you will fail, period. -- Tristan Louis
Don't make me think. -- Steve Krug, usability expert

#5 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 17 February 2012 - 05:53 PM

anyone having any idea if there is already some usable pipX code or we should start from scratch?


If you mean an existing firmware base then the answer is yes. I compiled the bootloader and firmware that is in the next branch last night and installed it onto a prototype STM32 board, and it appears to have worked. The only major additional hardware in addition to the MCU is the radio unit, so I thought I could start testing by hooking 2 proto boards together via a serial port.

There were a couple of little glitches that I'll probably check into a branch this weekend.

i would suggest to build a fully time-triggered system which is perfect for such transmission things.


I'm not sure what you mean by that.

#6 XXL-Wing

XXL-Wing

    Lead Developer

  • Members
  • PipPipPip
  • 200 posts
  • LocationNiederglatt
  • Country: flag of Switzerland Switzerland

Posted 17 February 2012 - 06:22 PM

http://www.le.ac.uk/...mjp9/pttes.html
as a start in time triggered systems

another thing to read could be this:
http://www.vmars.tuwien.ac.at/ttpa/
and
http://www.hartcomm.org/
No matter how good your backend systems are, the users will only remember your front end. Fail there and you will fail, period. -- Tristan Louis
Don't make me think. -- Steve Krug, usability expert

#7 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 17 February 2012 - 06:37 PM

http://www.le.ac.uk/...mjp9/pttes.html
as a start in time triggered systems

another thing to read could be this:
http://www.vmars.tuwien.ac.at/ttpa/
and
http://www.hartcomm.org/


I'm not sure we need that. The underlying architecture is already built on an RTOS (I'm assuming it's using FreeRTOS), and I don't think there's anything so time critical that we need to do fixed time slicing. The STM32 is a pretty powerful MCU, and if it can keep up with everything that is going on in the CC, it can keep up with message processing.

The only issue in my mind is the encryption and/or error correction algorithms. Those have to be able to keep up or they will not work.

#8 PT_Dreamer

PT_Dreamer

    GCS Core Developer

  • Members
  • PipPipPip
  • 1979 posts
  • LocationCaparica, Portugal
  • Country: flag of Portugal Portugal


Posted 17 February 2012 - 06:40 PM

There is fully functional code already, and AFAIK without any glitches, I have used it many times without a problem.
Before you jump to the coding I would advise you guys to spend some time looking at the structure of the CC code and the effort put on the HAL. This is without a doubt one of the most difficult things to do (maintain a proper separation between logic and hw) and one of the not so good things about the existing PipX fw.
Life is just a game, but atleast the graphics are awesome!

#9 PT_Dreamer

PT_Dreamer

    GCS Core Developer

  • Members
  • PipPipPip
  • 1979 posts
  • LocationCaparica, Portugal
  • Country: flag of Portugal Portugal


Posted 17 February 2012 - 06:41 PM

I'm not sure we need that. The underlying architecture is already built on an RTOS (I'm assuming it's using FreeRTOS), and I don't think there's anything so time critical that we need to do fixed time slicing. The STM32 is a pretty powerful MCU, and if it can keep up with everything that is going on in the CC, it can keep up with message processing.

The only issue in my mind is the encryption and/or error correction algorithms. Those have to be able to keep up or they will not work.

Last time I looked the PipX fw didn't use a RTOS.
Life is just a game, but atleast the graphics are awesome!

#10 XXL-Wing

XXL-Wing

    Lead Developer

  • Members
  • PipPipPip
  • 200 posts
  • LocationNiederglatt
  • Country: flag of Switzerland Switzerland

Posted 17 February 2012 - 06:56 PM

this piece of HW is a very specific one, so any type of OS or any abstrction layer is not needed i think. it's all about serial in -> wireless -> serial out. the rest is wireless hacking and protocol stuff.
a fixed time slice implementation helps a lot in communications....
but anyway as there is a working firmware the first step should be to check this and then decide what to do.
No matter how good your backend systems are, the users will only remember your front end. Fail there and you will fail, period. -- Tristan Louis
Don't make me think. -- Steve Krug, usability expert

#11 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 17 February 2012 - 07:09 PM

The 2 (minor) "glitches" that I found were I think just caused by recent changes that didn't get fully propagated to the pipxtream code.
  • The MCU config file is not passed to the openocd command. I had to add it to some config file that I can't remember now.
  • The LED initialization code was not called in the boot loader, which caused an assert.
I would rather not think of this as necessarily a single use item. Of course, I like to tinker, so who knows what I'll do with it.

The MCU really does have plenty of power, so I don't think we should unnecessarily limit it. I would really like to make it more like the CC, with a modular architecture. The CC (and I assume Revo) architecture is really quite modular and easy to use. It would also make it easier to share code between the devices (e.g. adding the receiver code, etc).

#12 Guest_dankers_*

Guest_dankers_*
  • Guests

Posted 17 February 2012 - 07:14 PM

It also was not completely using PiOS yet either and I do think modular with the RTOS is the right architecture, look at CC, we can drop in new modules to replace old ones (and have done this a few times). It's very powerful and will enable experimentation easier such as dropping in new transmission schemes.

Another area that would be helpful is Stac's & Jose's work with VCP and HID as a composite USB device, VCP for data and the HID channel for monitoring and configuration.

The closer this is kept to CC, the more solid and well tested code can be shared.

Just my 2c

#13 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 18 February 2012 - 02:24 AM

In order to "re-factor" the PipX code to be more inline with the CC code, I think we need to do the following:
  • Create a stripped down version of CC bootloader and firmware that is configured for the hardware that's on the PipX.
    • I did this for my transmitter, and it shouldn't be hard to do it again. We could start from that, but I would rather start over since things have changed and been cleaned up somewhat since I did that.
  • Integrate the rfm22b driver back in that codebase.
  • Create a message passing module that relays the messages between the serial port and the radio.
    • This is probably going to be similar to what I did in the transmitter branch.
  • Integrate the encryption and write/integrate the error correction code.
  • Figure out how to do configuration and save settings.
    • I don't know how the settings work now, but it would be good to use settings UAVObjects like in the CC. There's no flash chip, so settings must be being saved to the internal flash.
  • Create a module to read the receiver and inject messages onto the radio channel.
It's a fair amount of work, but I think it's doable. If we wanted to get something up ASAP, we could probably start with my transmitter branch, but it's a bit of a hack in places.

#14 Guest_dankers_*

Guest_dankers_*
  • Guests

Posted 18 February 2012 - 03:00 AM

There is quite a lot of that complete, there is a bootloader already for example, AES encryption works and basically it does function as a modem right now.

However, most of the areas I believe you are right about but I view it as more of a heavy refactoring job and use PiOS a lot more (which will make life simpler).

You are right about the flash chip, there is already EEPROM emulation for the internal flash on the STM32 in the code base.

Anyway, you have the reigns. I hope to find time to do some stuff in the GCS for it but Pip even did the basics of that as well.

#15 XXL-Wing

XXL-Wing

    Lead Developer

  • Members
  • PipPipPip
  • 200 posts
  • LocationNiederglatt
  • Country: flag of Switzerland Switzerland

Posted 18 February 2012 - 06:04 AM

one more thing to add to the nearly complete list above:

we have to check that the usage of the 433 MHz band is in line with regulations.
i do not know how it is in other parts of the world but in europe you are not allowed to permanently use the 433 MHz band, so the modem has to go on and off the transmit mode, so this is another reason for me to have something that is controlling the slots used.

and: we definitely need some check of the medium, as especially in 433 MHz a lot of other stuff happens, and i would not like it if my neighbour opens his car door and the connection is completely lost.
No matter how good your backend systems are, the users will only remember your front end. Fail there and you will fail, period. -- Tristan Louis
Don't make me think. -- Steve Krug, usability expert

#16 psupine

psupine

    Learner

  • Members
  • PipPipPip
  • 172 posts
  • LocationBrisbane
  • Country: flag of Australia Australia


Posted 18 February 2012 - 07:09 AM

I've just been re-reading the regulations for the 70cm band in Australia (see www.wia.org.au).


Am I right in thinking that PipX will implement the packet radio protocol, with the higher level ECC over the top?

I'm happy to help with coding and testing, and have a Ham ticket if we need that for testing too. (So do lots of people on this forum - it's good to see it's alive and kicking.)


Edit: Here is what might be a useful link, rather than reinventing the wheel.

Edited by psupine, 18 February 2012 - 07:24 AM.


#17 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 27 February 2012 - 06:49 PM

I now have a "core" firmware that is essentially a stripped down CC firmware that (I believe) correctly presents itself as a PipXtreme. That means that it's using the PipX USB signature and hardware configuration profile. It even shows a PipXtreme graphic on the hardware page when you plug it in!

Currently all it does is configure a USB HID device, which will be used for communicating the the GCS for configuration and stats, etc., and a Virtual Com Port (VCP) device that will be used for the actual message traffic. I'm having some trouble getting a driver loaded on my Win7-64 machine, but I believe the firmware should work. It also configures the UARTS and I wrote a module based on the ComUsbBridge module, called RadioComBridge that I expect to be the basis for relaying data from the USB port/com port to the radio link. It currently just relays from the USB HID device to a com port.

Next we need to get the packetization and radio link working. These parts don't use FreeRTOS, so there's going to take a considerable amount of reworking from expecting a big for loop to being individual threads that wait for data/events and act appropriately. This brings up the question of how much we want to change.

My preference is to add functionality in gradually, and save the bells and whistles for later. This might mean not putting the encryption and/or error correction in at first, which I think is reasonable. A bigger question is if we want to maintain compatibility with the existing PipX protocol, or if we want to rethink it.

This is the current message format:

// encrypted packet format
// 16-byte  CBC .. 1st byte must not be zero
//  4-byte  source id
//  4-byte  destination id
//  1-byte  packet type
//  1-byte  tx sequence value
//  1-byte  rx sequence value
//  1-byte  data size
//  4-byte  crc of entire packet not including CBC bytes


// unencrypted packet format
//  1-byte  null byte .. set to zero to indicate packet is not encrypted
//  4-byte  source id
//  4-byte  destination id
//  1-byte  packet type
//  1-byte  tx sequence value
//  1-byte  rx sequence value
//  1-byte  data size
//  4-byte  crc of entire packet not including the null byte

Given that each packet has a source and destination ID, it looks like the intent is to support multiple modems on the same frequency. I haven't seen that in the code yet, so I don't know if it's supported. It would be nice to support multiple modems on slightly different frequencies, but I don't know if that's possible.

I also see references in the code to connecting and disconnecting. I think that a connection less protocol is the way to go. Maintaining connections adds a great deal of complication that I'm not sure we care about.

My thinking on how to implement it is to mimic the USART driver as much as possible, which means that the radio link could look just like another com port externally, and would use a similar fifo buffer. A packet would be sent either when the fifo filled to a certain point, or when a timer expires, which would cause a partially filled message to be sent. There would also have to be an API for inserting priority messages for e.g. PPM packets, etc.

Any thoughts / suggestions?

#18 Bani Greyling

Bani Greyling

    GCS Core Developer

  • Members
  • PipPipPip
  • 190 posts
  • Country: flag of South Africa South Africa


Posted 27 February 2012 - 08:15 PM

I'm having some trouble getting a driver loaded on my Win7-64 machine


try starting Win 7-64 in "Test Mode" to load the driver (bcdedit /set TESTSIGNING ON). Not a long term solution but can give you a perspective on the actual cause.
What I cannot create, I do not understand - Richard Feynman

#19 LeeS

LeeS

    Developer

  • Members
  • PipPipPip
  • 343 posts
  • LocationLake City, Florida
  • Country: flag of United States United States


Posted 27 February 2012 - 08:24 PM

What is the benefit of having a totally different packet format when you are sending encrypted data packets? Seems like this is more an application layer issue than a link layer one.

#20 Brian

Brian

    RF Lead Developer

  • Members
  • PipPipPip
  • 1337 posts
  • LocationTucson, AZ
  • Country: flag of United States United States


Posted 27 February 2012 - 08:31 PM

What is the benefit of having a totally different packet format when you are sending encrypted data packets? Seems like this is more an application layer issue than a link layer one.


I agree with that. On the other hand, the packet format is not totally different. The body is the same, but the first byte or two are different.

I'm a little more concerned that there isn't a sync byte (or two). I haven't looked at the radio code to see if it really is packet oriented. If it isn't, I don't know how one tells where the start of the packet is.