Driving every corner
of our global economy is code. In this series, we look under the hood
at today’s most dynamic open source software
with the people behind it. Matt: The emergence of AI
in machine learning, this idea that we can have machines
that could actually help people. Robotics in AI are transforming
the factory floor. John: I’m John Shewchuk,
Technical Fellow at Microsoft. In this episode,
we’ll meet up with Matt Robinson, a thought leader
in the field of robotics, and then we’ll head to Munich, Germany to see how open source
is driving innovation at BMW. Matt: We’re really at a tipping point. As we think about the capabilities
we’re developing and how we’re applying them,
what we can do is trust that we have conscientious people
making those decisions. This is Decoded. John: Welcome to San Antonio, Texas. Scientists and engineers
think of San Antonio as a high tech hub where advanced technologies
like robotics are being developed. So let’s talk
with a local robotics expert, Matt Robinson. So what do you do, Matt? Matt: I’m the Program Manager
for ROS-Industrial at Southwest Research Institute. And ROS-Industrial is a program
managed by the institute that takes the power
of the Robot Operating System, ROS, and extends it
to manufacturing industrial use cases. John: Nice.
So how do they get started? Matt: About 10 years ago,
an outfit called Willow Garage in Silicon Valley,
brought forward this, basically this framework
or middleware called ROS, basically leverage to develop
service robotics at the time. About five years later after that the institute embedded a staff engineer
Shawn Edwards into Willow Garage, they created this first bridge
from ROS to a manufacturing robot and that was the birth
of ROS-Industrial. John: What’s the benefit? Why would an organization want
an open source robot operating system? Matt: That’s a great question. The idea is taking something
that already exists, a great foundation, and getting right to solving the client
or application problem. Grab these pieces we need,
build out that application, relatively short order and get right to the heart
of solving application. John: And it’ll work across a whole bunch
of different kinds of robot. Matt: Exactly. Interoperability
becomes sort of baked in. The other nearly nice exciting benefit is obviously
this vibrant open source community. They’re always like innovating,
solving problems, providing feedback. In fact, we’ve got a number
of great examples up the road. You want to go, check it out? John: That would be great.
Matt: Let’s go. John: Let’s do it. Wow.
Very impressive space. Matt: Thanks. Welcome to Southwest Research Institute. You know, we like to say
from deep sea to deep space with all of the different things we do here. John: And what is your team do?
How do you fit into all this? Matt: We actually seek out
and work with our customers to identify opportunities that difficult to solve challenges
around automation. And we leverage basically
an extensive software suite of tools, ROS, ROS-Industrial,
to help solve those challenges. And this is actually a great example
of that behind us. John: Wow! Matt: This application here,
we’re looking at just getting people out of dangerous conditions
in Air Force depots. John: Okay, so for a traditional robot
where everything is sort of lined up and you know where everything is,
this would be a hard problem. Matt: What we’re looking at
here is we leverage that ROS and ROS-Industrials software stack. In this particular case,
we leverage the perception pipeline and the path planning tools
to basically perceive that dynamic environment
and respond to it. John: Now you have some control software
we can take a look at it? Matt: Yeah, actually
the visualization environment is right over here. And what you can see
here is the live feed from the lidar scanners, both on top
and on the front of the system. John: Well, so this works
with these big commercial robots, but you can apply this
to many different kinds of robots. Matt: Exactly. Aerospace, automotive,
your hobby application, you name it. John: Wow, now you have
some of these in a lab somewhere here. Matt: So happens, we do.
Let’s go, check it out. You can see here we have a number
of different pieces of hardware. You know robots by
Yaskawa, ABB, Universal, FANUC, the gamut. John: Now the software that
that you’ve been working with runs across all of these. How much modification do you need
to do to be on a given device? Matt: Well, relatively minimal. And the whole benefit of ROS
and ROS-Industrial is is we have these reusable components and the interface is how we operate… So it’s actually to get up and running, and mockup and approve the concepts in
relatively short order. Here we’re working on this project
with NASA to do basically intelligent
teleoperation for systems that are very far away,
say the moon or Mars. John: An operator is able
to manipulate something remotely, but the robot has local sensing… Matt: Correct. Right. It’s got that layer of intelligence
to basically make teleoperation more intuitive. It’s extensible in the fact that like, you know, we can go from space
or work with automotive like, outfits like BMW, or any other number
of applications that come down the road. John: Well, thanks for showing me
around today. Matt: My pleasure! John: Let’s step outside
the research lab to a very different part of the world and explore how open robotics
meets the open road. Welcome to Old Town Munich
in the heart of Bavaria. Germany is known for distinctive design
and precise engineering. And there is no better example
of that than BMW. The philosophy
of form following function comes through not only in their cars,
but also in how they’re made. We’ll sit down
with some of the minds behind BMW to learn more about
the role robotics plays in manufacturing
the ultimate driving machine. But first, let’s go for a drive. The outside hand
is all the time responsible for what you do
with this steering wheel. Walter: Look at this. It’s great. John: I’m getting the hang of it. Walter: We here have the radar
which follows the car in front of you. The steering camera,
which follows the lines. John: Let’s give it a try. Now what do I need to do
to get this into autonomous mode? Walter: Oh, you activate
the autonomous system. Press the button. John: Now look at that.
No hands. Walter: Look at this.
No hands. It worked. John: Very nice. Wow! Martin: Hi, John.
Welcome to BMW. John: Thank you.
Martin: Do you like your ride? John: It was great.
The drive in this thing, the drifting. It was awesome. Martin: It’s brilliant that you like it. Today, we want to talk about
what happens to make such a ride possible. Let us go inside
and talk a bit more, okay. John: Sounds great. Tell me a little about
your role here at BMW. Martin: I’m responsible
for setting up the central transportation
steering system with my team. This involves combining robot solutions,
actually setting up robots ourselves, and steering them within the plant. John: We’ve got a centralized system,
coordinating robots, what do they do? Martin: At the moment,
we’re building 9,500 cars and we are handling
over 30 million parts per day. And that’s where our logistics come in. What we wanted to do,
we wanted to ensure that we bring the right parts
at the right time and the right quantity. We bring automation in like
a smart transport robots and combine
and share them within logistics. John: How do you put
something like that together? Martin: It’s a two-fold approach. You have the robot
which is based on ROS, it’s an open operating system, and attend
Central Services orchestrating all that. We’re not only centrally working
on orchestration, we’re also working on the robots
to understand what openness means. On the robot side, we actually deploy
what you have experienced autonomously in you’re driving out there,
so we take components, navigation, batteries,
and all these sort of things. John: One of the key things
about this open source approach is that you could design a solution once and then they could work across
many different kinds of robots. Is that something
you’re taking advantage of? Martin: The market situation
at the moment is that everyone brings his own solution
and there is no openness. So, what ROS helps us to do,
it helps us to define the interfaces
to create an open communication with various robots. John: I was at the track,
got to go on the skid pad with the M3 and the i8, and the autonomous vehicle
was really incredible. When I think about
kind of what was involved in making those cars,
how does that all come together and what’s your role in this? Marcus: I’m responsible
for the operations and physical logistics. In the past in logistics,
we’ve really haven’t had a lot. And it was quite
a boring business, and that’s going to change. Things logistics has never seen before, all that is coming. Now that’s what my colleague here, Dirk, he is from the planning
part responsible and he actually
then comes up with the project. Dirk: We constructed
an autonomous vehicle. We called it Smart Transport Robot. And we started with a cool idea
really to use our battery, the electric module from our i3 because we can use this as a second use,
for example, from used cars. And we needed a platform
as a communication layer between the different vehicles we use and we have to create
open source techniques and open interfaces
with other vehicle manufacturers for the material flow. Alexander:
To build that out as open source is very important to attract other companies to move into the system, then we have the opportunity
to buy products there and combinate that with ours. Marcus: And we came from the point
that we looked and the entire process for logistics and to developed ideas
and visions for every step. Some of these steps
might not be available in the next five years, but others, maybe tomorrow,
or even today. This platform should last
for the next 10 to 20 years in the basic structure. Otherwise, we would’ve come up
with a lot of individual solutions. Dirk: The first transport robot drove
after six weeks… John: Wow. Dirk: So it was really fast
and it was very important to bring it down to the shop floor. And that’s the possibility
we have in Plant Alpha, we call it, it’s very important
to go fast and quick in the plant. And therefore, we went to Regensburg,
spoke to Marcus and really integrated it in the shop floor, and made these
very important experiences working with IT and developers as integrated teams
right on the shop floor. Marcus: At the moment,
we run nine different cars on the same assembly line,
and we’re having in the shop, more than 50,000 part numbers. Alexander:
But the small transport systems give us a completely different way
how we can now do the line, supply to line, because and the passage
was always take to tugger trains that just have to be filled in advance,
and now we can do it more independently, more flexible. John: Do you think it’s harder
to build an autonomous vehicle for the factory environment
doing logistics, or one that has to work out
in the real world? Marcus: I think the question
is very easy to answer. We have no snow in the plant,
we have… There is no fog in there,
and the speed of the vehicles is completely different. Dirk: It’s 1.5 meters per second, so… Alexander: So, there’s only one,
I think, challenge we have to face that the others do
not have to face in this way. It has to be extremely cheap
what we are building. Dirk: And we need
the same safety issues. We are talking about
collaborative robots in the plant. So we have a lot of people there,
a lot of associates walking around. John: It’s almost like you’re building
your own little mini city inside the plants
and taking those ideas and being able to apply them
broadly seems like it’d be a pretty interesting thing. Now my understanding
is you’re even taking those algorithms. For example,
for traffic, and building them out as open source capabilities. Marcus: The logistics process at
the moment is nothing where the customer
is happily paying for. John: Mm-hm. Marcus: We are open now for solutions. And if this is an open,
I would say interface where, everybody on the world
can actually develop new solutions which might help us. John: I know we’ve got people
working together from BMW, from other companies, from Microsoft,
all together over a Plant Alpha. In fact, I’ve got Matt Robinson,
a special guest, that it will be fun to introduce
sort of rep of Plant Alpha. So I’m excited to check it out. Martin: Here we’re in Plant Alpha,
that’s our testing and developing center for all the robots before we actually
bring them out into the plant. Matt: What example do you have here? Christian:
We are palletizing empty boxes, so when we get the boxes
back from the assembly line, we have to palletize them before we can send them
back to our suppliers. This is only one type of boxes. We have four hundred of them
in each plant. For a robot, it’s really hard
to detect all the boxes. This is now working already
in Leipzig, a lot of progress going on there and without some kind of modularization
it would have been impossible. Hey, John, what are you doing? We’re doing some great stuff with BMW with the autonomous robots
that we’re building here. Let me introduce you
to my friend Christina over here. She’s actually working
on the UI part for us. – Christina.
– Hi, Christina. Christina: As you know
we’re working on services for autonomous transport systems. To enable our users,
which are the workers and operators at the plant. We provide them
with the great user experience with our web application. John: And what kinds of things
can they do? Christina: For example,
they can manage and edit maps, like the robot needs maps to navigate. So they can do some configuration stuff
in the user interface. But, of course, there’s also a big part
for monitoring the robots. John: So you can see where they are
and what their health is… Christina: Exactly.
John: What do you connect to? Christina: Our backend services. Like, for example, the map manager and also the chop manager
are connected via ROS APIs. John: And you cache that locally
and then build it up… Christina: Exactly,
so for sending down chops, but also for receiving
robot information, but we also use messaging
via SignalIR. John: Oh, nice.
Christina: So to get all the real time information. John: So that’s how…
Yeah. Great. So who do you work with on that? Christina: One of my colleagues, Lucas.
He’s sitting over there. John: Hi, Lucas. Lucas: Hi, John. Nice to see you. John: So what are you working on? Lucas: I’m working on the integration, the robot runs ROS components
and they are written in Python and C++, and they’re connected to the cloud
using a rosbridge that connects through IoT hub
and IoT hub manages like hundreds or thousands of robots. John: Wow.
And, is this also all open source? Lucas: It’s all open source. And there’s long term support
by the Windows IoT core team. John: Wow, that’s really cool. Matt: Hey, Carlos. Carlos: Hi, Matt. Matt: What have we got here? Carlos: So this is our STR,
Smart Transport Robot, that we have been developing
over the last year. And this robot, this brings flexibility
to our production field, moving different transport goods
into specific places in the production. Matt: Great.
Show me what’s under the hood here. Carlos: So open source, and usability
is quite important at BMW. An example
will be the battery of our i3 model, which we’re using in this robot. And it’s quite powerful,
moving at 8 kilometer per hour and moving more than 500 kilograms
in our production fields. Matt: Over here, it looks like
we’ve got a ROS box, maybe. Carlos: Yup, and this will enable
all the communication between the sensors
and the navigation stack. When the robot is moving, we need to ensure
the security of our workers. Matt: Well you touched on sensor fusion.
Carlos: Yeah, that’s right. I mean, that’s the most important
part of ROS. There’s communication
with topics and notes will give me a lot of… Ease of communication
between these sensors. Matt: Hardware in the loop
and being able to see what the sensors see, really enable
robust application development. Marcus: Now we can actually visit
our innovation area in the shop floor where the rubber hits the ground. John: There are so many people
working here, including all these engineers
working together. How do you think about the impact? Marcus: There are a lot of people here
just transporting material from A to B. John: Mm-hm. Marcus:
And that is really not considered in our business as a value-add. Martin: Here in the plant,
we have 9,200 tugger trains. And you see everything is manual. Driving is manual,
the exchange is manual. Our vision
is the human-robot collaboration because we will have always
associate in the line. Matt: Oh, one of the exciting benefits
or opportunities of these like, sort of, agile deployments
and developments is that we can introduce them
into legacy factories, right, you don’t have to put
in a whole new infrastructure. John: Yeah, that’s a big deal. Marcus: People really are waiting
for innovations. We talked a lot about industry 4.0
and innovation digitalization. People here in the plant are getting
the first impression what the future really needs. So we can take that part of the process
and we can automate it, then we will have impact
on our bottom line. Matt: It’s looking good, guys. Carlos: Hey, guys.
Yeah. John: Yeah. Mission accomplished. Matt: Yup.
John: Nicely done. Matt: Great job, guys. John: The thing I love about robotics
and what we’re doing with logistics and so on is it’s fun
like our developers seem to be having a great time, working together
to make these things happen, and the impact is enormous. That combo of really interesting
hard technical problems that we get to work together to solve
and that the impact that it can have on our businesses,
it’s amazing. Martin: I mean, you saw that when you saw
the robot going around the curve, that it slows down
and we have to actually introduce the different safety software. So we have to sync them
so it would be wonderful if you have it open source
and connected so that you don’t have
that many interfaces because collaboration is in our opinion
one of the most crucial points. The better, the more, free flowing it is,
the more stable it gets. Matt: When your team was down
at our place at Southwest Research Institute, it was really great
to bring all those minds together, you know, my team members
who had experience obviously with Gazebo
and the ROS stack and then doing
all these benchmarking exercises. We tried to spin up to scale
of a BMW type application. It was really eye-opening. And it was great to see like those minds with the passion coming together
to really sync their teeth in the meaty application like
what BMW has got to go on. Martin: And we saw this week
on the hack where we actually went
from Gazebo to ROS to actually be able to simulate. John: Yeah. Martin: Hundreds of robots
to see what’s happening. And you had the robot guys,
you had your guys and our guys, it was great
what happened in this one week. John: All right. Cheers. Martin: Cheers.
Matt: Cheers. Good job, guys. What a really exciting time
relative to robotics. We see the future with tools
like ROS and ROS-Industrial and open source tools, in general,
the real opportunity to bring human
and machine closer together. Closer together as in like,
physically side by side, but also how they maybe operate
more intelligently with each other even in remote ways. That’s really exciting time.
I’m very excited to be a part of it. We’re really at a tipping point,
relative to capability, between the emergence
of advanced robotics capabilities and the leverage and developments
around AI machine learning. And we want to make sure
as we think about the capabilities we’re developing
and how we’re applying them that we give really great thought
to what is their end use, and what is the extensible use case. We have to be cognizant
that there is a lot of power that that could manifest over time. And so at least being aware
of that could help us prevent any kind of a Skynet scenario,
if you will. The benefit of what we’ve been seeing going on in the open source
development community is the idea that like anybody
with an internet connection and a decent machine, they can pull down
a lot of this capability and get to work doing
really interesting things pretty quick. The tutorials are immense
and really powerful and easy to use. Give it a try, share what you’re doing, participate in the community,
and if something sticks, run with it.