Showing posts with label geppetto. Show all posts
Showing posts with label geppetto. Show all posts

Tuesday, September 5, 2017

Updates: Intel Joule, LoRa, Arduino and Protocase

Blog Break

It has been a while since I posted a blog entry.  I have been knee-deep in a challenging internal project and haven't had much time to come up for air.  Today, I've got a chance to reflect on some of the things that have been happening here at Gumstix and share my perspective with you.

First, there's the EOL announcement from Intel that blindsided the x86-focused IoT community:
 the Joule, Edison, and Curie modules are soon to be no more than a footnote in the history of embedded computing.

Also, Just recently, hardware support for LoRaWAN was added to the Geppetto module library and 3 LoRa boards were released.  I got to play with that quite a bit.  It also brought with it an ATmega32U4 Geppetto module, supplanting the Curie module as our primary Arduino-compatible MCU.

Finally, I've been talking a lot with Protocase.  These guys are cool.  They provide design and production services for custom small-run enclosures, rackmounts, brackets, and consolets. I'm very excited to see what they're making for me.

Intel EOL Announcements and Me

I'll admit it, I took the Intel Joule news harder than maybe I should have.  I spent a lot of time working with it and it's carrier boards.  I was looking forward to putting the GadgetDrone in the air with the AeroCore 2 for Joule, the Caspa HD and one of the RealSense point cloud cameras we have at the office.  I liked the idea of setting up my Workstation board in a 3D-printed enclosure as a Yocto build slave.  Oh and I still hope to test the Caspa 4K's "tone-mapping (Er, I mean HDR) Video" mode.

I was also sad to see the Curie go.  Working with our Radium 96Boards IE board is a lot of fun.  It had been a while since I'd worked at the MCU level.  Bare-wire programming on an 8051 and using a dual-Arduino Uno plus ZigBee robot controller were highlights of my academic career, but I haven't done anything of the sort since.  The Radium's nice and small, IE compliant and has all the cool features of the Curie, like Bluetooth, 6-axis IMU and Neuron pattern recognition nodes.

For whatever reason, Intel decided to terminate their IoT-targeted endeavors.  Maybe it was the slow - and sometimes negative - response from the community.  It's also possible that the challenges in providing software support for their hardware were more monolithic than anticipated.  Either way, the Joule, Curie and Edison are gone.

For all of you who jumped on board with Intel's IoT hardware just over a year ago, I empathize with your plight.


If you're into IoT, you may have heard of LoRa, LoRaWAN and the LoRa Alliance.  It's a communication protocol for sub-GHz long range LPWANs, and it's sweeping Europe and North America's IIoT industry.  It works like this:  

You set up a Gateway. This is the equivalent of a WiFi router in your home, but the difference is these things can have a range of up to 15 km, depending on the quality of your antenna.

You deploy nodes.  These are your data acquisition points - temperature, presence detection, air quality, etc.  Whatever you need to know.  Put them where they need to be and hook them up to a battery, solar panel or hamster wheel (No hamsters were harmed in the writing of this blog post).  The idea is that they require very little power to run and can last anywhere from a week to several months on a single charge, or indefinitely with solar.  These tend to have a range of 2-5 km.

You monitor the data and use it as you see fit.

Gumstix released a gateway/concentrator and a transciever module in Geppetto, as well as a gateway dev board for both the Overo and the Raspberry Pi Compute Modules (Overo Conduit and Gumstix Pi Conduit boards), and a weather station sensor board (Strata Node).  They're in the store and available in both North American and European frequency bands.

Once I had my Gumstix Overo Conduit gateway and an RHF0M301 gateway/concentrator module in hand, I was impressed with how quick and easy it was to set up on  The Strata node I recieved was pre-release and hadn't had the bootloader flashed yet (they come pre-flashed now), so it took a little longer, but writing a sketch and setting up a project on TTN and went super-smoothly.  It just so happens that I made a bit of a quick-start video:

Arduino Stuff

Arduino is a great thing.  For artists, makers, inventors, amateur developers, and teachers, it's a great way to avoid the challenges of bare-wire programming and get physical objects doing what you want them to do.  For professionals, it's a good prototyping tool, delivering your proof of concept to the project manager in hours or days instead of weeks (or worse).

Adding the ATmega32U4 to the Geppetto library means I'll get a lot more time to play with Arduino hardware, projects, board support, and the IDE.  It also means that there will likely be more Arduino boards coming to the store and hardware modules coming to the Geppetto library.

I'm also going to have to find a quick and easy way to set up my 'arduino_pins.h' file.

Discovering Protocase

If you've seen my previous posts, chances are you've seen my low-tech enclosures, mounting brackets and test environments.  My indoor quadcopter test flight had a paracord tether tied to the rafters so that I didn't give my co-workers a hair cut.  I like to think of it as ISRU (In Situ Resource Utilization).  However, in some cases, a solid, well-made case is more than just a good idea.  When I went shopping for enclosures for my Overo Conduit board so that I could deploy it outside, my boss pointed me to  I think he just wanted me to stop asking for a 3D printer for the office.

These guys are awesome.  They're working on something for me and I can't wait to show it off.  They have a huge variety of custom products: L-shape, U-shape 5-sided, milled aluminum, and more!  They'll build from your CAD drawings and have free templates to help get started.  They even have their own design software for you to use.  If all else fails, they will work with you and design a fully customized enclosure for your device.

If you've got a prototype, an invention or a first-run for a kickstarter campaign, Protocase might be for you.  Just check out their page and see for yourself.

To Summarize:

I've been busy.  From grinding away at that internal project to working on LoRa and Arduino board support to designing enclosures to recovering from the Intel IoT fallout, I've hardly had enough time to catch my breath.  Now that things are settling down a bit, I am looking forward to spending more time telling you all about the cool stuff I'm working on.

Friday, July 14, 2017

AutoDoc: We Automated Product Documentation

Geppetto's AutoDoc

Gumstix has announced this week that Geppetto users can now download documentation for their designs that is generated from the current state of their project.  The addition of the AutoDoc button marks the public release of a tool we've been using internally for some time.  As one of the content curators of this application, I'm excited to share it with everyone because it's a bigger deal than it sounds.

Hardware Documentation

I'm one of those people who find it satisfying to produce usable technical documentation - to describe the operation and setup of a platform in a legible, accurate and eloquent way.  I have always felt that written tech materials either skimp on details or skimp on readability, and I strive to find a balance of both.

For hardware, one of the documents engineers find most helpful is the schematic - a block diagram of the ICs and passive elements that are interconnected in a design.  Firmware programmers, on the other hand, are less reliant on these drawings.  Instead, they need to know the bus names, addresses, GPIO indexes, and features  of the devices connected to the SoC they are programming.
Both need access to technical reference manuals, a layout overview, and connection maps.

AutoDoc cover page for one of my designs
Geppetto's purpose is to provide an avenue for board development that does not require in-house electrical engineers, providing a piece of hardware that could pass directly into a developer's hands.  In my opinion, this is the epitome of the startup use-case.

Why AutoDoc?

Because Geppetto completes the layout, BOM, and fabrication steps for your design, you can have a board in your hands really fast.  That doesn't leave a lot of time for someone like me to sit down and write out a detailed description of your design and its components, so we developed a method of generating written design specs solely from the data later used to generate your board.  We have been able to provide this kind of documentation for some time and it has helped many clients get their applications up and running fast.

With the data in these files, developers can configure drivers, write interface software, and begin testing code before the board has even shipped.  By exposing access to this resource to you before you complete your design, you don't even need to order your boards before you get started. Clicking on the AutoDoc button gives you the pin-outs and signals as they are currently saved.  I feel like it's empowering you with ownership of your design at the earliest possible time.

...And I don't have to sit there and write it all out.

Board layout diagram and a module description

Why it Works

If you don't know what LaTeX is, you should.  It is essentially a programming language for documents.  Look at this way:  When you write a document in MS Word or Google Docs, you have to struggle with the complexity of its "What You See is What You Get" (WYSIWYG) interface.  This is nice for letters, memos, and most resumees, but when it comes to technical documentation, it becomes a fight.  Also, it's not particularly portable.  When you create a document in Google and download it as a .docx file, it definitely doesn't look the same.  Margins change, images move or disappear entirely, fonts are lost or mutilated...  A mess.

Tables, captions, title pages, pagination and indenting:  All these things can be hard-coded into a document with LaTeX.  It will look right every time you compile it.  It will look right when someone else compiles it.

So if we treat documentation as source, then we can script it.  We can apply templates to it. We can make the result deterministic and incrementally improve its quality and content.  And this is what we did with AutoDoc.  There is now no temporal or financial expense in creating a useful reference manual for custom designed Geppetto boards  Enjoy.

What's Next?

I like how Geppetto automates things.  Seeing what it delivers always makes me smile.  So what else can we automate.  Lots!  What else do you need, as firmware developers?  As project managers?  As startup companies?  You can look forward to more of it from Geppetto.

Friday, March 17, 2017

Gumstix Pi Compute Boards are CM3-Ready

If you follow me on twitter (@gstixguru), you might know that I recently ordered an RPi CM3.  Lots of people have been contacting us to find out how well our Pi Compute boards support the new, faster module, so I found a bit of time to play around with it.  I'd worked with the original CM on our dev board for my GPS and RTK project a year ago with great success, and was looking forward to getting back to the Pi Compute boards.

First Steps

As always, my first step was to flash a brand new image onto the CM's eMMC.  I downloaded the latest Raspbian Jessie Lite ISO and mounted my CM on a Gumstix Pi FastFlash.  Next, I ran rpiboot, plugged the board into my USB hub and CROSSED MY FINGERS!

RPi CM3 on a FastFlash getting flashed. Pardon the clutter.
So what happened next?  Exactly what should:  the eMMC was mounted to my file system like any unpartitioned flash drive would be.  So I dd'ed the image, moved the module over to the Gumstix Pi Compute Dev Board and got ready to Pi.

First Boot

At first, all I wanted was proof of life.  That and I was sure the default wpa supplicant and network interfaces config would not get me on the WiFi network.  So I screen'ed in and powered up the board.  And yes, the console came to life, spewing forth those familiar Linux startup messages.  No kernel panic, no errors, no problem.  So far so good. Raspbian Lite was up and running.  Oh, all the things I should test: GPIOs, I2C, SPI....  BORING!

Let's start with USB (Oh, and get the WiFi up and running while we're at it; screen is not my friend and SSH makes me smile:).  The WiFi dongle goes into the port and lsusb shows a list of devices.  And there it is.

Bus 001 Device 002: ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter

Beautiful.  I fix up /etc/network/interfaces and add the office WiFi network to wpa_supplicant.config and shut it down.  Time to set this asside and get back to my other tasks.

Day 2

Before ditching the USB console connection, I have to go into raspi-config and enable the SSH host, and reconfigure the daemon:

sudo rm /etc/ssh/ssh_host_*
sudo dpkg-reconfigure openssh_server
After a restart, ssh works fine.

Let's got straight up the food chain to the camera!  That's what I want to see!  I want to get that Sony IMX219 taking stills and recording videos.  I want to see those LVDS signals in action.  The CSI-2 camera connector is by far my favorite feature of the dev board.  So while I was in raspi-config, I made sure to enable the camera as well.

Here's my Frankenberry Pi camera rig, ready to go, I hope.
So I hooked it up, fired up the module and... nothing.  Did I forget something?  Of course I did! I needed the device tree overlay blob for the camera.  Oops.  OK, so I grab the binary, -- I get the one for both camera and display, just because I can -- copy it to the boot partition and restart.

And did it work?  See for yourself:
Me and my clipboard.
Edit: Here's me trying to pretend I'm not being recorded by the Pi Camera:


I also took a few minutes and got the USB-Ethernet board fired up, and yes, everything works great.
I am very happy.  Stay tuned!  I have a Raspberry Pi DSI display around here somewhere and I want to get that up and running too.

Wednesday, February 15, 2017

There is Nodana...


For those of you who don't know about the 96Boards open-specification hardware platform, it's a design spec for single-board computers (SBCs) that enables SoC vendors to provide their hardware in a standard form factor for increased compatibility.  It's also an engaged community working together to develop applications, software, and mezzanine cards for this ecosystem.

96Boards now has 3 different specifications for 3 classes of application.  There's Consumer Edition (CE), with standardized breakouts for both high-speed and low-speed signals, USB ports, HDMI, and so on.  There's also the Enterprise Edition (EE), which is more for server and networking applications.  It's a larger and more free-form design, with a low-speed header, USB and Ethernet, minimum 1 GB DRAM or expandable SODIMM slots, and optional 1 - 16 x PICe.  Finally there's the brand new IoT Edition (IE) spec.  It's designed to be tiny in order to fit anywhere.

All of these specifications have variants that allow hardware developers to add extra bits to their boards, making this a very flexible way of standardizing the important parts of SBCs.

The big benefit is that you can unite developer communities accross platforms.  The mezzanine card or maker project developed for board A will be compatible with board B, and vice versa.  With support from Linaro, providing a common Linux ecosystem for these boards, not even software compatibility should get in your way.

My honest opinion is that this open specification is very cool.

Gumstix is a 96Boards Partner

Yep, we're in cahoots with the folks at 96Boards and Linaro to bring you compliant hardware.  The release of the AeroCore 2 for Dragonboard 410C was only the start.  At the same time, we added the 96Boards Mezzanine Connector module to Geppetto D2O's library so that users can design their own mezzos for other applications.  If you don't know what Geppetto is, you can learn more by going to the Meet Geppetto page, read my earlier posts, or go straight to and give it a try.

I did a demo for 96Boards OpenHours, hosted by Mr. Robert Wolff (@sdrobertw) and actually flew my MAV, using a Dragonboard and the AeroCore 2 live in my office -- complete with a visit from the "demo demon".  The whole thing's on YouTube.

...Only Joule

So for those of you who don't know, a little compute module was released last year with quite a lot of juice hidden under its heat dissipator. The Intel® Joule™ module delivers unprecedented compute power in a tiny package.  From its two 100-pin Hirose connectors pour USB 3.0, MIPI LVDS,  PCI Express, HDMI, and a lot of what you already expect from COMs and SoCs.  It also houses its own WiFi and Bluetooth hardware.  All with the power of a quad-core processor akin to the Core-I7s you find in your desktop PCs.

Surprise, surprise, Geppetto's got that too!  You can go in and build your own host board using the Intel module and harness most of what it has to offer.

So a Square Peg and a Round Hole Walk Into a Bar...

On one hand you have this fantastic open spec hardware platform [round hole].  In the other, this epic compute module [square peg].  "those will never fit together," you might say (in fact, one 96Boards community member did).  Well, we gumstixians are very resourceful.  And the spec doesn't restrict the SoC's architecture to ARM, that's just the expectation.  So what did we do?  We took all of the components that make the 96Boards Consumer Edition spec great, we wired it up to the Joule connectors, (tested it), gave it a name, and unleashed it on the unsuspecting masses.

And that is how the Nodana 96Boards Consumer Edition (96BCE) for the Intel Joule module came to be.  Here it is:

Gumstix Nodana Features

The Black Sheep

That's right, all you doubters.  Now you can test your 96Boards projects on a powerful 64-bit multi-core Intel chip.  It's the first of its kind -- the first non-ARM 96Boards device.  Take it for a spin and tell me about what you do with it.  You can order it at

x86 IoT Fun

Psst!  We are also taking the IE spec to this dimension.  Our Radium 96BIE board complies with the 96Boards IoT Edition specification and runs the Intel® Curie™ module.  A 32-bit Quark processor  in bed with an ARCv2 MCU, a 6-axis internal measurement unit (IMU) and an independently programmable Bluetooth controller. Check it out at

Tuesday, February 14, 2017

Jacob Marks: Demystifying Computer Architecture for High School Students with Yale's Ventures in Science... And a Little Help From Gumstix

Interview with Jacob Marks:  
It’s a Processor!
Yale Undergraduates Bring Computer Architecture and Design into the High School Classroom with Geppetto D2O
By Keith Lee

Computer systems and embedded technology are playing a larger and larger role in our day-to-day lives.  From smartphones to smart homes, we’re putting computers everywhere.  As these systems become more and more complex, training the next generation of innovators gets increasingly difficult.

The first step, of course, is to convince High School students that degrees and diplomas in computer science are attainable goals.  Today, kids can usually pick up any device and start using it right a way.  But to them, the inner workings of computers may be a complete mystery.  So how do we break that barrier and demystify computer hardware?

Well, a handful of Yale undergraduates may be onto something, and they’re using Gumstix boards and Geppetto to do it.  Jacob Marks, a Yale senior and president of the Ventures in Science student organization, along with his colleagues designed a lesson plan to help young students engage, both intellectually and manually, with the inner workings of computer hardware and architecture. I asked Jacob some questions about it and he told me all about Ventures in Science, ‘It’s a Processor’, and his experiences.

Jacob and ‘Ventures in Science’ Take on STEM Education

“Ventures in Science (ViS) is an undergraduate organization at Yale, started five years ago by a bunch of students who wanted to connect undergraduates with scientists in industry,” Jacob says. “[...] In the past few years, ViS has refocused its efforts on educating the leaders of tomorrow about science and its role in law, technology, policy, and culture. When I became president of ViS, I realized we could further advance our objective by increasing scientific literacy and exciting children about science.”

To this end, he and his team started teaching middle school kids about the scientific method with paper airplanes. “My team led students as they experimented with different designs and materials, trying to throw their airplane the farthest. In just an hour, students learned about hypotheses, trials, controls, and variables.”

From there, their involvement in science education continued in the form of demonstrations at Science on Saturdays, a popular lecture series for kids at Yale, reaching out to students from dozens of New Haven schools.

IMG_5829.JPGJacob is a student of physics, mathematics and philosophy with a keen interest in quantum computing technologies.  But he confesses that his exposure to computers and technology was very limited in grade school. “I remember spending elementary and middle school computer classes learning to type with Mavis Beacon, and (more frequently) an “educational” computer game called Oregon Trail,” he related to me,  “When I started college I had never written a line of code, and I couldn’t tell ROM from RAM.”

This continues to be a typical sentiment in public schools.  For whatever reason, computer science and technology tends to get only the most cursory treatment.  Kids are encouraged to use computers in and out of the classroom for word processing, research, and edutainment but most often everything beyond the keyboard and screen is left a mysterious, black box of parts to many kids.

After being exposed to subjects like cryptography, and a software engineering internship at Reservoir Labs, and realizing how many other people’s pre-university computer science background was near zero he decided to make it his goal to “expose - and excite - as many students as possible about the subject.”

In order to tackle that challenge, he and his team developed a lesson plan called “It’s a Processor” for High School students. “ ‘It’s a Processor’ was the beta-test of a new interactive curriculum to teach kids about digital logic and computer architecture. Analogies and demonstrations help illustrate how all of the hardware components in a computer come together to perform computational tasks.”  

The title refers to more than just computer processor.  “The name ‘It’s a Processor’ points to the process of learning itself, self-consciously acknowledging that this one event is only one small part of a complete education in computer science,” He told me, “We wanted students to walk away with a basic understanding of computer architecture, but more importantly we wanted them to leave feeling inspired to dig further into the theory and practice of computer science.”  This certainly a noble aspiration for inspiration, which deserves a stronger foothold in our children's’ education.

IMG_2426.JPGThe Workshop

The ‘It’s a Processor’ drew in students from 7 different public schools, none of whom had ever seen the inside of a PC.  “For many, this was their first time seeing logic gates, and holding a processor, but they all caught on quickly. It was truly a pleasure working with these students!” The computers built during the event are all being donated to Wilbur Cross High School, “the largest public school in New Haven, and one of the most lacking in resources.”

IMG_5831.JPG1 - Getting Warmed Up

The workshop starts with a quick, 20-slide high-level presentation.  First it reviews the nature of electricity, describes digital logic, and then moves on to deconstruct computers into its various hardware components and architecture.  Explaining these more detailed features to the uninitiated can be a real challenge.

Often, when it comes time to describing the inner functions of the computer, educators will use the brain and central nervous system as an analogy to compute systems, but is this the right approach when you’re talking to high school students?  

“It’s true that the computer is often anthropomorphized - and the human-computer analogy is very strong,” Jacob agrees, but “[...] While children understand the roles that the ‘brain’ and the ‘nervous system’ play in a human, the connections between these various body parts are somewhat complicated and can be hard to visualize. To really appreciate the way that they work together, one would need a basis in biology and physiology.”

His team opted to use a beautifully hand-rendered “Computer as a City” example.  It relates memory to storage facilities, external ports to shipping docks, and buses to... well, buses.  Admittedly, this sounds much easier to assimilate than the organic CNS example. So where did this come from?

Computer as a City color with labels.png
IMG_2484.JPG“I first came up with the idea when I heard data buses described as ‘information highways’, and thought it would be interesting to map the parts of the computer onto different urban features. Then our graphic designer Dana Smooke ran with it and turned it into both a teaching tool and a work of art. The finished product was even better than we had hoped - and the kids loved it!”

2 - It’s a Computer

The kids had an opportunity to check out, up close and personal, what goes into the workstations at their school and at home.  Able to touch things like the motherboard, RAM modules, SSDs and power supply, they got to assemble their own “city”.  “[...] the tactile act of holding the internal components in their hands helped make computers more tangible. ‘It’s a Processor’ turned a ‘black box’ into hardware that they recognize and appreciate.”  After all the demystification of computer hardware and architecture is the primary goal of the workshop.


IMG_2437.JPGBut desktop computers weren’t the only things these kids got their hands on.  Gumstix partnered with ViS at Yale, and donated ‘Pepper DVI-D’ single-board computers (SBCs) for the students to investigate.  The Pepper DVI-D is a 10x7 cm SBC with Ethernet, HDMI, USB, audio and a bootable SD card slot -- everything a barebones system needs -- connected to an ARM Cortex-A9 SoC.  Each group was given a Pepper to connect, boot, and play with, adding to the haptics of the experience.  “ Working with the sleek and light-weight Pepper board took away the potentially intimidating scale of a massive HP-like motherboard.” With peripherals connected and SD card loaded, the students were able to observe a working Linux environment on a tiny, unobtrusive device.


3 - Grand Finale: Geppetto

IMG_2489.JPGGeppetto is a web application from Gumstix that allows users to build their own embedded computers from the ground up in minutes without using complex tools.  While this tool is ideal for small to medium size tech companies and startups looking for a fast path to market, its intuitive design proved helpful in driving home the day’s experiences.  “All of the students loved using Geppetto!” shared Marks, “I walked them through a brief tutorial on the projector screen and then they each had a chance to play around with it on their own. The kids were so drawn into Geppetto that we actually had to get more computers so that they could all explore the framework at once.”

IMG_2494.JPG“The most valuable feature of Geppetto [for the lesson] was definitely the “show-price” capability. The students got really into seeing who could design the cheapest motherboard that had specific components,” he shared, “The blueprint view also helped them as they attempted to fit as much as possible onto a board without having to extend it. [...] the students really loved the opportunity to design custom motherboards, moving beyond the limitations of pre-made hardware.”

The team wrapped up the session with Geppetto’s 3D preview, a feature that shows you a close approximation of your board’s appearance before you even order it.  It’s rendered in real time in your browser so you can rotate, zoom, and even export the model as an .STL file.  “Geppetto was the perfect finale to our event. Its 3D viewing capabilities allowed the kids to concretely identify the abstractions of Computer as a City with the physical components of the computer they had just constructed.”

Bringing It Home

When asked about how the whole session went, he told me about how the students felt about the experience:

“After the event, we asked the students to take a short survey about our interactive activities and demonstrations. Their responses were mostly positive, but some of the students suggested that we make the activities EVEN MORE hands-on. We definitely hope to do so in future iterations.

IMG_5840.JPG“‘It’s a Processor’ was a huge success in that it helped get kids interested in computer architecture. But there is still so much work to be done refining our curriculum and learning to express concepts as effectively as possible. This is absolutely an ongoing process. The next step is to bring this to a wider audience by working with both middle and high schools all over connecticut.”

“It brought me great joy to see how the team was able to use our Geppetto system in the curriculum to bring the students right into the process of device creation,” said Gordon Kruberg, M.D., C.E.O. of Gumstix, Inc.  “We at Gumstix are proud to support the It’s a Processor curriculum and look forward to seeing more smiling faces from student inventors as they begin to understand that they can control their own futures.”

For more information or the Ventures in Science curriculum contact Jacob Marks at

IMG_2470.JPGRelated Links:

Photos from the ‘It’s a Processor’ workshop provided by Jacob Marks and the Ventures in Science Student Society with written permission from participants.  

The ‘Computer As a City’ artwork contributed by Dana Smooke.

Keith Lee is the Gadget Guru at Gumstix.  He has a Master’s degree in Computer Engineering From the University of British  Columbia and enjoys making, tinkering with, and designing gadgets of all kinds whenever possible.
Twitter: @gstixguru

Wednesday, September 21, 2016

Improving Safety in Surface Mining with Geppetto

Image: Wikipedia
Surface mining is a modern economic necessity and, all controversy aside, it's not going anywhere soon. One day, the process of removing minerals and ore from the earth's crust may be performed by drones and remote operated machines but until that day, men and women will be working in the high-risk, hazard-wrought environment that is open-pit mining.

The industry does a great many things to mitigate the potential dangers these brave individuals face everyday.  For example, I once saw a really cool piece of surveying tech that automates some of these tasks. Two jumped out at me in particular: slope fracturing measurement and equipment wear detection.

Slope fracturing describes the stability of a slope based on the arrangement and size of the particles in it.  Mines must monitor the degree of slope fracturing in order to prevent dangerous rock slides.  What I saw was a hand-held device that would process, on-site, images of the slope using computer vision techniques and special analysis software to estimate, at a glance, some metric of stability.

Image: Wikipedia
Another important task that this device tackled was measuring the degree to which wear components of excavating machinery had deteriorated.  Operating heavy machinery is dangerous enough in ideal circumstances, but as the teeth on the scoop of an excavator begin to wear, it has to work harder to do its job.  A weak everyday metaphor of the ensuing situation is using a knife in a kitchen.  The duller your blade becomes, the more you compensate to get the job done.  Suddeny, you're pressing down with all your strength to cut through a potato, for example, when the knife slips and cuts your hand badly.  Damage to your knife blade, like chips and burrs, can cause their own problems as well.

Similarly, as the teeth of an excavator scoop wear down and its work begins to require more force, the operator will inadvertently begin to compensate.  Eventually, like the slipping knife, the machinery may experience a severe and life-threatening mechanical failure. 

Now imagine an industrial kitchen with hundreds of knives to maintain.  It would be good to be able to look at whole batches of these knives and immediately be able to tell which ones need to be sharpened or replaced.  With the replaceable teeth on mining equipment, there is a standard level of degradation permitted before they are replaced.  Without assistive technology, each tooth of each machine would have to be inspected.  The device I saw allowed an individual to audit an entire array of parts at once and tag those in need of replacement, again using an on-board camera and computer vision software.

Both of these processes involve the physical presence of an expert to manipulate the device and interpret the data.  To put this in the context of the industrial kitchen, let's equate slope fracturing to the state of decay of food. Under this system, the chef must go through every fridge, freezer and storage room to ensure every steak is fresh, every leek is green and every expiry date hasn't passed.  Then, he has to go around testing every knife to make sure it is in good repair.  This protects the health and safety of his employees and customers.  How do we make his job easier?  What if each cook was knowledgeable enough to make the micromanagement decisions of which head of lettuce was rotten and which blade was dull?

Okay, so this is sort of already the case in kitchens, but maybe we can use technology to make this happen in the mining industry.

Here's my caveat:  I know very little about open-pit mining.  What I know is that mining engineers need to collect slope fracturing data and measure the state of degradation of wear components on mining machinery.  And I understand that this data can be used to improve safety and reduce operating costs.

Designing a Mezzanine Board

So how do we automate an early-warning system for landslides and machine wear in the context of surface mining?  So we have two computer vision tasks, one of which takes proximity as an input.  Other potentially important variables are geographic location and heading.  We also need some powerful brains to process this data.  Oh yeah, and we need to get the data from the field to an off-site computer for analysis.  Oh and let's keep costs and power consumption to a minimum, shall we?

The Big Idea

What if we could equip every excavator, bulldozer and dump truck with the means to relay slope fracturing and mechanical wear data off-site in real time, providing analysts with the means to monitor the safety of the equipment and terrain? Then they could dispatch crews to deal with these issues before they become problems.  Let's slap a little device in or on the cab of the machines at the mine, attach a few devices and antennas to it and give it the ability to perform the two tasks I described above.

It'll need two views, one facing forward and one in view of the wear components of the machine when it's at rest.  Since these are unlikely to be the same, we'll need two cameras.  It needs some method of measuring distance.  IR and ultrasonic methods do not have the range needed for the given application so something like LIDAR would be useful.  Positional and directional data are also important for both tasks so some kind of GPS and digital compass will be needed.

Beyond that, we just need a way of processing and transmitting the data for analysis.

So here's most of our hardware requirements:

  • 8MP+ camera  x2
  • LIDAR rangefinder
  • GPS
  • magnetometer
  • LTE modem
  • compute device
Hey no problem.  We're going to use my favourite tool: GEPPETTO D2O.

Compute Device

Geppetto has an ever-expanding list of connectors for COMs.  Recently, we've added 96Boards, TechNexion PICO SOMs, and the new Intel® Joule™ compute module to that list.  What we need is a CM that supports 2 HD cameras.  Coincidentally, the 96Boards mezzanine header does. Specifically, the Dragonboard 410c and HiKey support 2 MIPI CSI-2 cameras (one 4-lane and one 2-lane).  Also, 96Boards are SBCs so they come equipped with WiFi, USB ports, HDMI, and USB-OTG, and include a bunch of low-speed communication buses, such as I2C.  These specs satisfy all of our hardware needs.



Garmin sells a module called the LIDAR-Lite for a reasonable price (~$150 USD) and I'm sure there are similar products from other vendors out there.  It can communicate either over I2C or PWM and is good up to 40 m (+/- 2.5cm).  This would do nicely for our system.


Geppetto has a convenient 5-pin header for Gumstix's Pre-GO and Pre-GO PPP GPS modules, which I have discussed at length in previous posts and is easy to implement in software.  That settles that.


We could use a compass to provide information as to which slope the device is analysing.  If we know where we are and we can tell which way we are facing, we should be able to discern what feature of the terrain we are looking at. Add Geppetto's 9-axis IMU and you get  a gyro and accelerometer as well.

LTE Modem

Geppetto provides a connector for NimbeLink® Skywire™ 4G LTE modems.  50 Mbps upstream is more than enough to transmit the analytical data from the board, even if you want to transmit a live video feed.

My Board: FracJaw

Well that's it.    all that remains is to slap that on a board and click "Order".  I did some playing around and this is what I came up with:

This took me about 30 minutes to come up with. It has my two CSI-2 camera headers, an I2C header for the LIDAR, an LTE modem connector, GPS, and an RTC.  And it all fits on a 13x5cm board., so about the size of your smartphone.

I've saved my board design and made it public in the community tab.  Or you can jump directly to it here.

Monday, September 19, 2016

A New Board and New COM Connector in Geppetto: TechNexion PICO-IMX6

Some of you may be wondering why I haven't posted any updates with respect to my RTK project.  Well, truth be told, it's been pretty busy here at Gumstix.  The release of Intel's new 64-bit IoT compute module at IDF, and our recent induction as manufacturing partner with 96Boards, gave me a steady flow of work.  And now we've released a new development board for the TechNexion PICO-IMX6 COM.

NXP's i.MX6 SoC has a fantastic selection of features - from 1080p HDMI to Gigabit ethernet, PCI express to image processing - and TechNexion has done a fantastic job of breaking out these features in a compact, low-profile compute module, complete with on-board WiFi and Bluetooth, an Edison-compatible low-speed header and two high-speed expansion headers.

Gumstix has put together a board with a long list of features to help you get going with TechNexion's PICO-IMX6 COMs.  Here's a list of its key features:

  • HDMI connector
  • Dual USB 2.0
  • microSD
  • Gigabyte Ethernet
  • MIPI DSI and CSI2 connectors
  • Audio in/out
  • NewHaven 4.3" cap-touch LCD connector

These and several more features, packed onto an 11x8cm PCB, make this board developer-ready for all kinds of projects such as handhelds, home automation control, tiny workstations or home theatre applications.  It's available now in the Gumstix Store

If you like the Gumstix PICO-IMX6 expansion board but it's missing something, or you just don't need this header or that display for your application, Its Geppetto design is available on the "Designed by Gumstix" tab in GeppettoD2O.  You can re-position, remove and add board modules to match up with your needs.