35 replies to this topic

[Corvus Corax]

Wellenbrett, on 11 September 2011 - 05:52 PM, said:

What about the the "Raspberry Pi" ? It will cost $ 25.- !
It is a Broadcom BCM2835 SoC with an ARM1176 Processor.
Here are some links concerning this board:
http://www.raspberrypi.org/
http://mobile.osnews...ing_1080p_Video
http://www.broadcom....roducts/BCM2835
http://www.arm.com/p...m11/arm1176.php

Looks not bad. It might just have the necessary "juice" - though it heavily depends if the GPU can be used to do arbitrary image filtering on an "incoming" video stream. HD MPEG en and decoding - while useful for mobile video devices - are useless if one needs to do arbitrary video stream filtering as needed for feature extraction and video analysis - even though the algorithms running in low level are more or less the same.

So it basically depends how much "custom tweaking" that GPU allows and how easily that is accessible using standard libraries like OpenCV without spending months in driver development.

[cwabbott]

I think I've mentioned this before, but IMHO it's useless to use sim_posix on something like this. Your messing around with very low-level scheduling code in order to fool the code into thinking it's on a microcontroller, instead of directly using what the OS gives you. My experience is that this leads to a lot of instability, no matter how much "hardening" you do. Usually, it crashes after a few hours; honestly, it's a miracle that it lasts this long.

I get what your saying about how to use UAVTalk - however, I would just replace the INS. It's the route I'm going down with the Beagleboard, so we could share code that way. Furthermore, you're using stuff that's already there instead of creating new stuff. Also, it seems to me like it would be better to just ditch the INS filter (perhaps getting sensor data from it) and fuse the data directly to your camera stuff in a special-designed EKF. That way, your algorithms get a better estimate of the pose of the aircraft (something which Dr. Schinstock mentioned in the 2nd podcast) and theoretically, a more optimal filter. Perhaps someone more qualified than me could chime in here.

[peabody124]

FWIW I would be doing most of this design and testing not on an embedded system first.  Then you actually know your requirements instead of a picking a system and justifying your requirements.  I know for me I'm much more comfortably keeping the hardware running as it is and inserting additional information into it (what Dale did in previous SLAM demo).  That way if something goes wrong on linux (which it will while developing code) your OP can be manually overridden.  However, if you find out the raw information needs to be fused in a more complex EKF then you can justify doing it that way.

For example you want all of this in one EKF if the uncertainties from the vision estimate interact in a meaningful way with the uncertainties in the INS.  However in a lot of cases the vision algorithm won't even produce any uncertainty over position so that would be useless.

[cwabbott]

peabody124, on 14 September 2011 - 09:07 PM, said:

FWIW I would be doing most of this design and testing not on an embedded system first.  Then you actually know your requirements instead of a picking a system and justifying your requirements.  I know for me I'm much more comfortably keeping the hardware running as it is and inserting additional information into it (what Dale did in previous SLAM demo).  That way if something goes wrong on linux (which it will while developing code) your OP can be manually overridden.  However, if you find out the raw information needs to be fused in a more complex EKF then you can justify doing it that way.

For example you want all of this in one EKF if the uncertainties from the vision estimate interact in a meaningful way with the uncertainties in the INS.  However in a lot of cases the vision algorithm won't even produce any uncertainty over position so that would be useless.

Well, I don't know how I could develop on a non-embedded system (ie a laptop) and still get useful inertial data - i think lag would become an issue. Anyways, AFAIK the Beagleboard is a pretty good general development board anyways - it was designed partially to do just that. Of course, I'm not the first to do this anyways - see for example this paper. Furthermore, you can't always assume that the SLAM filter is going to give a good estimate of position & attitude - especially when the camera moves fast. Any sudden maneuvers, and you're screwed. For example, when Dale tried to do a flip with what he had, the copter crashed. With directly intergrating the sensors, you can just not do an update phase until things settle down, and then you have a pretty good estimate of where you are so you can re-initialize pretty quickly. Of course, you get scale information too (with the accels). That's why Dale said (during the podcast) that directly fusing the inertial & vision estimates on-board would be a better idea.

[peabody124]

cwabbott, on 14 September 2011 - 09:44 PM, said:

Well, I don't know how I could develop on a non-embedded system (ie a laptop) and still get useful inertial data

That's easy - get a few streams of known test data (using beagle board for example).  For example video + inertia sitting still, moving at a fixed rate, etc.  Then try processing it through different examples.

Quote

That's why Dale said (during the podcast) that directly fusing the inertial & vision estimates on-board would be a better idea.

Heh.  I remember, I was the one having the discussion with him about it :-p.  But if you go back to that you'll see it's because of running it in a more complete model.  For example if your model only says "this is my position and uncertainty around it" that extension to EKF can easily run on INS.  It's only if you develop a complete EKF that, for example, utilizes the uncertainty in the state vector for the visual processing that you'd need it integrated into the BB.  I completely agree if you write something like that it will work better (for example where sensor data drives the vision solution) it will work better and need to run on BB.  However that is a complex enough project you really do need to work on it you need some good test data sets to work offline with.

For me if I can ever finish the other stuff to play with this I'll start by using the BB as another data source into the INS.

[cwabbott]

peabody124, on 14 September 2011 - 09:57 PM, said:

That's easy - get a few streams of known test data (using beagle board for example).  For example video + inertia sitting still, moving at a fixed rate, etc.  Then try processing it through different examples.

Good point. Most of that stuff is pretty device-independant anyways.

peabody124, on 14 September 2011 - 09:57 PM, said:

Heh.  I remember, I was the one having the discussion with him about it :-p.  But if you go back to that you'll see it's because of running it in a more complete model.  For example if your model only says "this is my position and uncertainty around it" that extension to EKF can easily run on INS.  It's only if you develop a complete EKF that, for example, utilizes the uncertainty in the state vector for the visual processing that you'd need it integrated into the BB.  I completely agree if you write something like that it will work better (for example where sensor data drives the vision solution) it will work better and need to run on BB.  However that is a complex enough project you really do need to work on it you need some good test data sets to work offline with.

For me if I can ever finish the other stuff to play with this I'll start by using the BB as another data source into the INS.

Hey, I never said you had forgotten . Anyways, I'm not going for something right away anyways. I'm ok if it takes a while - I'm busy enough that it'll take a while anyways. It's also a lot of stuff for me to learn, so it's as much about learning the theory as actually getting it done.

[peabody124]

cwabbott, on 14 September 2011 - 10:13 PM, said:

Hey, I never said you had forgotten . Anyways, I'm not going for something right away anyways. I'm ok if it takes a while - I'm busy enough that it'll take a while anyways. It's also a lot of stuff for me to learn, so it's as much about learning the theory as actually getting it done.

Always a great goal!  And I bet once the infrastructure is in place to process it some mathematically inclined people/researchers would be really interested in the the algorithms too.  SLAM does so well, just imagine if it were augmented by gyros/accels

[cwabbott]

peabody124, on 14 September 2011 - 10:23 PM, said:

Always a great goal!  And I bet once the infrastructure is in place to process it some mathematically inclined people/researchers would be really interested in the the algorithms too.  SLAM does so well, just imagine if it were augmented by gyros/accels

Exactly! it's been done before (see the paper I posted above), but I don't know if it's been done yet using a keyframe-based approach. Then there's also the challenge of fitting everything on the quad - all of the SLAM-on-a-quad stuff I've seen so far has been using some sort of offline computation.

[Brian]

I think I'm missing a portion of this conversation, but I believe you're talking about using video to somehow aid in navigation/stabilization.

We have implemented a couple of image stabilization algorithms on a single core of a Pandaboard and have them running at "a few" frames a second on 640x480 video.  I believe it could run faster with some optimizations.  This, or course, gives you linear shifts between frames, which we had thought could be fed back into at least the camera stabilization algorithm.  It could also be used to provide a velocity estimate.  Is this what your goal is?  If so, I think it's doable.

[cwabbott]

Brian, on 15 September 2011 - 12:53 AM, said:

I think I'm missing a portion of this conversation, but I believe you're talking about using video to somehow aid in navigation/stabilization.

We have implemented a couple of image stabilization algorithms on a single core of a Pandaboard and have them running at "a few" frames a second on 640x480 video.  I believe it could run faster with some optimizations.  This, or course, gives you linear shifts between frames, which we had thought could be fed back into at least the camera stabilization algorithm.  It could also be used to provide a velocity estimate.  Is this what your goal is?  If so, I think it's doable.

It's more than just optic flow, which is what you've described. It's called SLAM - Simultaneous Localization and Mapping. It's like the difference between just having gyros, and having gyros and accels - you can correct for drift in position. Basically, you have to use the camera to simultaneously build up a map of your surroundings, and locate yourself within that map (as the name implies). So, for example, you could use it to automatically have your robot fly around a building and create a map of it. The tricky part is that errors can easily accumulate upon themselves, causing your solution to blow up if your not careful. Also, it can get derailed very easily if you shake the camera too much - gyros and accels help here. Also, you have no idea of orientation (which way is up) and scale (how big things really are) - again, accels can help here.

[ThomasB]

Corvus Corax, on 07 September 2011 - 07:18 PM, said:

boards with a high end CPU are hard to design due to the high clock rates involved, even in a system-on-a-chip solution (unless even the memory is on chip) I'd like to see that mysterious board

Hi Corvus Corax,

Today Chris open's the curtain, The new secret board of DIY Drones is based on a .... tataa!   Atmel SAM3U    
HA!  Really unimpressive !
The new future starts with an old style cpu (compared to the F2xx)  
So I've to offer an excuse, my former comment was really misplaced in this topic

best

   Thomas

[StarRider]

Hi,

I was not going to mention these processors/ controllers but since you are talking about SLAM, a couple of year ago ESA developed a Embedded Multi-Camera System for Simultaneous Localization and Mapping, at the time the cpu used was a SPARC V8 core (LEON) that was developed under contract for ESA by Aeroflex (the usual ESA subcontractor for critical systems mcu cores).

The LEONs are suitable for SOC, ASIC and FPGAs. They are designed for VERY TOUGH environments, critical applications, can support high levels of radiation, temperatures, etc,  …  the LEONs can have dual-cores, are capable of 1.7 DMIPS/MHz, have fully pipelined double-precision IEEE-754 floating point unit, CCSDS / ECSS Telecommand decoder, CCSDS / ECSS Telemetry encoder,  etc, etc. ... it really doesn't get any better than these.

Linux, Nucleus, RTEMS, eCos, WxWorks, GNU C/C++, eCos real-time O/S for LEON, LEON C/C++ IDE for Eclipse, etc, etc.

They are expensive, and they came in strange packages, and the devboards are not very small … take a look at: http://www.gaisler.com/cms

I can get a couple of this if anyone wants to give them a try.

Regards,
PA

Edited by StarRider, 18 September 2011 - 02:42 AM.

[Maniac]

VIA EPIA Mobile-ITX and Pico-ITX boards

[Corvus Corax]

Thanks for all the nice suggestions.

My current state is - I have an ARM11 running linux on my plane but currently its only logging data to sdcard - lots of data - gyros, accels, GPS, baro, mags, .... At the same time I am getting an mpeg stream 640x480 30fps from a camera - also to sdcard. With enough of that it should be possible to evaluate slam algorithms on a PC on that test data, and then optimize them until I have something fast enough to run embedded.
Hopefully.

It has been done before. I found some papers dealing with onboard slam on microcopters. They use POI detection followed by a kalman filter to get the coordinates of each interest point right.  It works with relatively few points and doesn't create a complex map, just enough points to locate the copter in 3d-space. Not enough for navigation (as in obstacle detection and avoidance) I fear. But I will be able to tell more next year when I tested a few actual implementations on real data.

[cwabbott]

Corvus Corax, on 23 November 2011 - 10:44 PM, said:

Thanks for all the nice suggestions.

My current state is - I have an ARM11 running linux on my plane but currently its only logging data to sdcard - lots of data - gyros, accels, GPS, baro, mags, .... At the same time I am getting an mpeg stream 640x480 30fps from a camera - also to sdcard. With enough of that it should be possible to evaluate slam algorithms on a PC on that test data, and then optimize them until I have something fast enough to run embedded.
Hopefully.

It has been done before. I found some papers dealing with onboard slam on microcopters. They use POI detection followed by a kalman filter to get the coordinates of each interest point right.  It works with relatively few points and doesn't create a complex map, just enough points to locate the copter in 3d-space. Not enough for navigation (as in obstacle detection and avoidance) I fear. But I will be able to tell more next year when I tested a few actual implementations on real data.

Can you share that data eventually? I'm trying to modify PTAM to record & playback gyro & accel data, but I've been unable to get the new oe-core stuff running on my Beagleboard-xm; there were a number of configuration problems with the X server, and it seems I've only fixed some of them so far.

[Corvus Corax]

I do put my logs on-line as far as possible. The log format is the same as used by the GCS when logging a telemetry stream but much higher throughput. It is not synchronized with the video stream yet unfortunately since that is logged by a standalone camera system (turnigy keychain crap right now)

You can see the result in my latest video. But I will upload the raw on-board video stream somewhere if you like.

The on-board flight data logs are here: http://wiki.openpilo...x 1km-milestone