Flying Cars to go With Our 140 Characters

March 6, 2018 Pivotal Software


Raj Kapoor, Chief Strategy Officer at Lyft, talks with Michal Lev-Ram, Senior Writer, Fortune, about what’s to come for mobility and autonomous vehicles.

“The car is utilized four percent of the time. So it’s utilized less than some of those servers that you have sitting in that data center when you move to the cloud.” —Raj Kapoor


Michal Lev-Ram: What’s your take on—your prediction for autonomous vehicles?

Raj Kapoor: It’s here.

Michal Lev-Ram: Yeah.

Raj Kapoor: It’s happening.

Michal Lev-Ram: I had a feeling you would say that.

Raj Kapoor: Yeah.

Michal Lev-Ram: Not 10 years out.

Raj Kapoor: If you go to Phoenix right now, our partner Waymo is live, driverless. And it’s not from one specific route. It’s around a very large area.

Michal Lev-Ram: How many of you have been in an autonomous vehicle? Raise your hand. Okay, you need to invite a few people out to Phoenix apparently.

Raj Kapoor: They’ll be plenty of opportunities in 2018.

Michal Lev-Ram: Let’s start a little bit of a higher level overview of what your mandate is at Lyft.

Raj Kapoor: Sure. First of all, Lyft is a company. Mission is very clear, which is to improve people’s lives through the best transportation. Notice, that’s not just about ride sharing. I think the long term vision of Lyft is that people want to get from point A to point B. And we’re passionate that they can get there without owning a car. So if you think about cars, it’s probably the biggest waste of money that we’re facing right now as a generation. It is the second largest expense, after food, for a household.

The car is utilized four percent of the time. So it’s utilized less than some of those servers that you have sitting in that data center when you move to the cloud. There’s something fundamentally wrong about that. And, also, in addition to that, we’re seeing that it’s causing some societal problems too, which is that the congestion that I’m sure a lot of your experience in cities is at an all time high. Admissions and what we’re doing about climate change is also a big consideration there. So we think that the solution is really about moving about more efficiently, but making the experience and making the price point something that consumers are excited about.

Lyft started back at Zimride. My history there is a little bit different because I was actually an entrepreneur, then I was a VC, then I was an entrepreneur. And while I was a VC, I invested in Zimride. So I was on the board when it was about a 15 person company and then recently joined as chief strategy officer full time because I got so excited about what they’re doing. But really Lyft started out with looking at this transportation problem and seeing we could be more efficient.

Our founder, Logan Green, while everyone else was drinking a lot of beer at college, he was on the transportation board at UCSP. And he was trying to figure out and solve solutions around how can we get buses to be more full, why aren’t people sharing rides more. So he created what’s called a ride board, just like there’s a job board, and that was Zimride. And that was the company that I originally invested in. But like most great companies that are created, you never end up doing the thing that you started out doing. We pivoted.

The issue with Zimride was that people were using it, but how often do you need to go from San Francisco to LA? What we found was there was a much bigger problem intracity to go across the street and to go to work every day, etc. That use case was a lot more frequent. We saw that this little company called Uber was doing this thing with black cars. We thought, “Why not let this person over there that has a car that has space in there, why not let them drive? Why not make everyone a driver in doing it?”

So we launched it, fully illegally by the way. This was a big moment at the board. “This seems like a good idea, but it’s illegal. Should we do it?” And the answer was yes. We got it out the door, and the demand was just through the roof. Since there, the history’s been exciting. We’ve had a great year this year. We’re up to probably around triple the market share that we had. We’re doing over a million and a half rides a day. So things are moving in the right direction.

To bring it back to what you’re saying, it’s about improving people’s lives. And the next generation, which is really exciting is in transportation, is around autonomous. This is an interesting challenge for a company like Lyft. If you think about some of the large technology companies, they were disrupting another company that was not in their industry. For example, Facebook was thinking about how could I change the publishing world or Google in that case as well.

In our case, we have over 800,000 drivers that are making a living. So making this transition to something that is inevitable, that consumers want, that’s safer, that we can do more efficiently is something that’s important for us to do. But I think we have to do it with a lot of care both from a safety perspective, but also in terms of managing that workforce that is going to be needed for a very, very long time. The growth in our industry is such that for the next decade, we think we’ll have more drivers. It’s not that drivers are going away any time soon.

Michal Lev-Ram: I know you guys have also talked about the role of the workforce once autonomous is becoming more and more ubiquitous and that there might be other roles that you could provide for a workforce, so we can get to that later.

Raj Kapoor: Sure.

Photo by Wesley Verhoeve.

“When you’re operating a fleet the size that we are, the amount of data that we can bring into improve the software is massive.” —Raj Kapoor

Michal Lev-Ram: But talk about … you explained some of the earlier iterations of the company. It’s very recent that you guys have really invested in coming out with developing your own autonomous, your own AV technology. I know you oversee partnerships, but talk about the thought process behind that because you have a partnership with Waymo. You’ve got several partnerships on the automaker/manufacturer side. Why build out some of the underlining technology to power AV yourselves?

Raj Kapoor: Yeah, there’s a couple of reasons. First of all, the future is very uncertain in terms of who is going to be the winner. Is this going to turn out to be a winner take all scenario where there’s one company that has the best software? And as you all know, it’s not just the software, it’s the data that’s gonna be powering and improving that software. Is that gonna be the case? Is it gonna be multiple companies. So, one was that there’s an uncertainty in the future, and this is our core business. We can’t just leave it up to a potential partnership.

Two is that we bring a lot to the table. The Lyft today in 2017 has an amazing brand, an amazing culture where we are. People want to come and work for us. So as a result, when we started these efforts to build our own system as well … we call it Level Five Engineering Center in Palo Alto. We’re at about 120 percent above recruiting in probably the most difficult area to recruit right now. I estimate there’s probably about five or 6,000 autonomous engineering jobs that need to be filled by probably 200 qualified people in doing it, that have experience in making that happen.

And so, I think getting the talent in the right place is important. The second piece that we bring to bear is the data that’s there, as I mentioned. We have 800,000 drivers. We’re doing over a million rides a day. What’s going to matter is how that system is able to perceive the world, and make decisions, and work through scenarios, and make sure that it’s handling those scenarios correctly, which means how much real world data can you get? When you’re operating a fleet the size that we are, the amount of data that we can bring into improve the software is massive. And it’s not really something that a lot have. We feel like we can bring that to the industry.

The other piece is that we have a lot of partners, and we kind of bifurcate it. There’s the open platform, which is the Waymo’s of the world, the Ford that we announced, NuTonomy, Drive AI, and others that are going to be announced. And then, we have our Level Five Engineering Center. Our belief is, we’re not trying to create something entirely proprietary that we’re going to take over the world. We just want to make sure that the building blocks are going to happen. That they’re going to be something that everyone can use. That the data’s available.

One example I’ll give you is that, today we all use Google Maps, and it can get you around. It’s great for your feed. It’s great if you’re driving. It doesn’t work in an autonomous vehicle. We need a map down to the centimeter to know where you are. So that’s one of the challenges that a vehicle has.

So now the entire industry has to recreate maps in the high definition level in doing it. So we’re in a position where we have all these drivers in now 370 cities around the country. We can be generating that map while those drivers are earning an income as well at the same time. So it gives us that advantage in doing it.

Michal Lev-Ram: So, just real quick, walk us through some of the other building blocks because there’s a lot that goes into the recipe of now developing this on your own.

Raj Kapoor: Yeah. And I think there’s a simplistic way to think about an autonomous car. First of all, this would not have happened if it’s not right now at this moment in time where there’s so many technologies that have intersected, whether it’s machine learning in neural networks, and computer vision, and compute power going down.

So what does a car need to do? The first thing it needs to do … it asks itself several questions. First of all, where am I? And this is a concept called localization. It need to know where it is down to a centimeter. And, usually, it relies on an HD map. GPS only gets so far. It uses multiple sensors in it’s map to lock in and say, “Okay, this is exactly where I am.”

Now, it asks the next question is what do I see around me? Now, it’s using usually computer vision, but usually, you can also use radar. And you can LIDAR is a very big new technology. It’s basically a laser that’s pointing, that’s bopping around and creating a 3D map, a point cloud of what’s there. And we fuse all of those technologies, and then overlay it with some machine learning to say, “Oh, I can do object detection. Okay, that’s a person. That’s a static object, which is a stop sign. This is a road marking that’s there.” So it makes those determinations.

Then it has to think about what’s static and what’s dynamic. And now it does planning. It says, “Where is that dynamic object going to move in the next millisecond or the next second?” And it has to predict and make a prediction of where that object is moving. Then it has to ask the question of what do I do now based upon my environment, based upon a global objective in my map that I want to get to my destination. But I have to think about what I do for the next centimeter or two. Do I slightly turn the left? Do I go fast? Do I slow down? Where am I?

And you can imagine that this problem becomes harder and harder the faster that the car is going because its reaction time has to be very, very fast in doing it. So once it figures out what it’s going to do … and usually that is also an application that you can have of neural networks that can help you solve that problem. Then it actually has to execute the instruction. And this has to translate back into actuation. And a lot of cars now are being built with drive by wire systems. In fact, we got one delivered where I literally used a Nintendo controller to drive the car around because everything could be done by wire.

So that’s what we need, that kind of level of reliability in doing it. And then it goes and plays that over and over again. And then it takes all that data back and pushes it into the cloud. We analyze it. We run it through 10 times the amount of simulations and permutations on it, make the system better, flash the new software into the car. And that car is collecting potentially like a half a terabyte of data in a couple hours. So the scale at which we’re collecting data, processing data, and trying to make decisions has never really been done before, which makes it such a grand challenge in doing it.

So those are the technologies that are there. It involves the cloud. It involves compute in the car. One of the challenges that Detroit has is that … to put this full stack of technologies today cost about … my estimate is about $250,000. Now there’s no way you’re going to go to a Mercedes dealership and buy an add-on for $250,000. They estimate that the consumer even in the luxury segment is willing to pay about 10 or $15,000. So we need to see the cost of all of that compute, the cost of all of that hardware and software come down so that it can work.

The good news for ride sharing is that we’re not utilizing a car 4% like a consumer. We’re utilizing a car 80 percent. So my cost of what we can have in a vehicle can be a lot cheaper. So that poses a challenge for our industry because we need to develop this technology and throw it on top of existing cars because if we wait around for the passenger market, the consumer market, we’re going to be waiting for years. So that’s why you see all of us in this industry, including Waymo and ourselves, building stuff on top of existing cars because not for a long time will cars be coming off with the autonomous technology.

“Right now, 40,000 deaths a year in the United States, 1.3 million worldwide. 94 percent are due to human error. 94 percent. And with vehicle to vehicle communication and autonomy that’s there, you could reduce that down to practically nothing. And what happens then to insurance?” —Raj Kapoor

Michal Lev-Ram: You mention that there’s a lot of uncertainty in the industry and a lot that needs to be figured out still. Where do you think this whole huge nebulous ecosystem could … what do you think it could look like a few years out? What’s the role of … GM is trying to do on its own a lot of what you’re describing regarding the stack, and you’re also partnering with them. So talk about the role of the auto manufacturers, the role of the ride sharing services, the role of insurance companies. How should they be thinking about this and their part in the ecosystem?

Raj Kapoor: So first of all, we are at a state in our industry … and this is a massive, massive opportunity. $2 trillion is the domestic opportunity for consumer transportation. So as a result, every player in the space is almost … it’s like their strategy is thrown up in the air right now. They’re trying to figure out what they should be doing. Should they be a mobility service provider? Should we be making cars? Should be vertically integrating into providing insurance? Everything is really up for grabs right now.

And that’s why you’re seeing lots of partnerships, and the next day you’re seeing partnerships with people that they though they weren’t going to partner with. And the next day you’re seeing different business models that are coming out. So none of this has been really settled. The other challenge that’s there in the industry is that nothing is really standard space today. So if I take one particular proprietary LIDAR, and I’m developing my perception software and all the way down the stack, it’s very specific to that.

Now if a new piece of LIDAR comes in, I have to retrain all those neural nets that are there. I have to also change some of the software, and I have to run it through a massive amount of testing and certification. Because the difference here is this is not just sitting in a data center. This is on wheels, and it could kill someone. So we have to have the redundancy of an airplane in our system.

So we’ve got usually two compute systems and a fail over system that can just take it to the curb in terms of making that happen. So my point there is the complexity and the lack of standards are causing a lot of vertical integration and partnerships of maybe I could use this technology. So some of that I think will play out over time in making that happen.

Now when you think about the larger ecosystem, there’s the vehicle, there’s the technology in the vehicle, but the impacts that autonomous are going to make happen like you’re alluding to are far reaching. So insurance is one of them certainly. Right now, 40,000 deaths a year in the United States, 1.3 million worldwide. 94 percent are due to human error. 94 percent. And with vehicle to vehicle communication and autonomy that’s there, you could reduce that down to practically nothing. And what happens then to insurance?

But there are other second order effects. So the city of Los Angeles estimates that they’re making about $250 million a year in revenue from parking tickets, speeding citations. That goes away. And the problem is that funds police, that funds schools. How are they going to now make up for that that’s there. The other effect is on cities. In Los Angeles, roughly 15% of all the real estate is parking. If you don’t need to park your car or you don’t own a car, what happens to all of that real estate? How do you redesign that? What happens to car dealerships?

So there’s a lot of open questions around where it is, but there’s going to be an opportunity for a lot of people in this ecosystem that’s going to be there. But we are going to go through a gut wrenching decade or so where everyone is jostling for a position.

Michal Lev-Ram: Okay. Questions for Raj.

Michal Lev-Ram: Go ahead.

Audience #1: How will present thinking will be mended because that’s going to be a massive challenge? Something fatal happens and the government says no. Then you have to stop the business. So how do you think that you deal with that?

Raj Kapoor: Yeah, so first of all, so far the United States is moving in the right direction with having a federal safety standard where they’re setting out guidelines, which are still be developed. And then it is upon the industry to adhere to those guidelines. And when there are issues, they can go in and inspect and shutdown programs if necessary. So that’s how safety standards work.

Now from an indemnity perspective, I do believe it’s going to come down to the owner of the network, who is responsible at the end of the day that has that consumer relationship that’s there. And like in many products, there’s multiple companies in the supply chain and you do a root cause analysis to figure out and apportion out liability. So I don’t think there’s that much different than how complex products work today.

Michal Lev-Ram: Does that answer your question?

Audience #1: Yeah. if you come from a healthcare industry… If that company pays a billion dollars for one death, then how do these companies are going to pay a billion dollars cash back to that person?

Raj Kapoor: And that’s going to be an important question that consumers are going to ask before they step into an autonomous vehicle.

Audience #1: Yes.

Michal Lev-Ram: Yeah?

Audience #2: What do you think about the new comment on Tesla’s strategy and where do you think that will end up?

Raj Kapoor: Which particular part of Tesla’s strategy?

Audience #2: It’s that they’re outfitting all the cars with the gear to capture the data regardless of whether or not you’ve subscribed. And then can they, with their quality of radar and vision, can they achieve a level of performance that you think you could achieve with a different order of magnitude of investment in a vehicle?

Raj Kapoor: So there are kind of two separate questions there. One is Tesla has a strategy of collecting data and also potentially then enabling their owners to go into the ride sharing business. You’re sitting at home, and your car’s making money for you and coming back home. That’s a big cultural change, but regardless I think one of the challenges that people have to understand is one of the critical things in our business is wait time. Three minutes is the magic number. Got to be below three minutes. In order to hit that wait time, you need to have a lot of cars that are available in lots of sections of town.

Autonomous technology is not zero to one. It’s not what Google is doing right now in Phoenix even is perfect. It works in certain conditions. Every year we’re going to expand the operating envelope with that. So for a decade or longer, you’re going to have to have human drivers. And this is where I think we come in with 800,000 of them, which is going to make a difference. Second question was more technology, which is they’re trying to be zero LIDAR, much cheaper by the way. Because LIDAR today is very expensive to do it, 25 to $50,000 even $75,000.

There’s an open question as to whether that can work because vision is not good under all conditions that are there. And it’s certainly is not the safest approach today. If it evolves in that direction, maybe.

“Autonomous technology is not zero to one.”—Raj Kapoor

Audience #3: How do you think about autonomous vehicles in the context of last month?

Raj Kapoor: So I think it’s absolutely an opportunity. And there’s some interesting startups that I’ve seen that are taking a totally different form factor for the robots. They could have a much smaller form factor. It is a much simpler problem because you don’t have to worry about a human being inside. You just have to worry about the last 50 feet and how you handle that. You could have a drone come out or you can have the person always come in there in doing that. But there’s a significant opportunity.

And then the other piece of it is that our business is driven by peaks just like most utilization businesses. And so the question is what do our vehicles do off peak? And delivery is certainly an option that’s there. It’s less of an issue in my opinion for what we have today because our drivers can be 100% utilized. And they’re coming on when they’re business versus us having a fixed asset, but that changes then in the world of autonomous.

Audience #3: How do autonomous vehicles deal with all the non-autonomous vehicles that are out there?

Raj Kapoor: Very carefully.

Audience #3: It’s a challenge, right?

Raj Kapoor: It’s a challenge, but it’s also been proven … there’s a study at MIT that even with a couple percentage points on the road of autonomous vehicles, you significantly alter the congestion and the safety because of the butterfly effect of it as well. So it doesn’t take a full turnover of the fleet for it to work.

Audience #4: Decision making. Are you doing machine learning models on the edge or just setting the data back to the cloud for the current process?

Raj Kapoor: Right now, this is what’s exciting. There’s totally different approaches. I’ve seen a company that takes in raw data from its sensors outputs accelerator and driving instructions. Then there others that break down into the four different things like for planning, prediction and have different models that are there. One of the biggest challenges you all know around machine learning and neural nets is that how do you diagnose when there’s a problem? How can you go back and troubleshoot?

So the bigger that envelope is that it’s covering, it can be challenging especially with liability issues or safety issues that happen. So it’s happening. Everything is happening at the edge centrally. There also has to be enough compute in the car because you can’t rely on network connectivity.

Audience #5: So looking at the future, what do you see as the role for autonomous personal air vehicles?

Raj Kapoor: To me, there’s just a whole other order and layer of challenges that are there from regulatory perspective, from a noise perspective, from an energy consumption perspective. I think it’ll happen at some point, but we are far from it. We have so many exciting challenges in land based autonomous vehicles that we’re focusing our energies on that.

Michal Lev-Ram: Other questions? So let me ask you one.

Raj Kapoor: Sure.

Michal Lev-Ram: You talked at length about the technology stack and one thing I think is interesting about your approach at Lyft is that you guys are also paying attention to the in car experience. And the company … Raj told some of the origin story about one of the founders, Logan. There’s another co-founder who came along a little bit later, John Zimmer. And it wasn’t Zimride because of John Zimmer by the way. It was Zimbabwe, right?

Raj Kapoor: Zimbabwe, yeah. Just coincidence.

Michal Lev-Ram: But they met eventually. Yes, very coincidental. But, anyways, but John Zimmer really came from the hospitality industry and that’s something that has been part of the DNA. If you’ve been in a Lyft from early on, the experience was different and you guys tried to create this very friendly environment. So how do you take those routes and advance that and expand it into this new technological era that you guys are going into?

Raj Kapoor: Yeah, that’s a really good question. So backdrop again what we have today. In ride sharing today, the real source of differentiation that we can do because we don’t really control the vehicle. It’s the driver’s vehicle. It is the quality of the driver and that personality of the driver. And that’s what we focused on at Lyft, and the culture, and the community around drivers. I almost guarantee if you walk into a Lyft today and ask them who do you like driving for, what do you enjoy more, you’ll get 8 out of 10 times that they’ll say Lyft because of that reason in doing that.

And it goes back and forth between the passengers and the drivers. So in the autonomous world, there isn’t a driver. There could be other forms of people in there. Either could be concierge. There could people that are helping. Let’s assume that there’s no one in there. The experience still matters. What matters is reliability, ETA, price, and experience. We think we have a really exciting opportunity to differentiate on experience, to bring some of that culture in that ethos.

One of the things that comes in is that Lyft from day one has been around shared. Shared results in about 25 to 30 percent cheaper price of ride if you share it with someone that you don’t know. People thought we were crazy. Over 50 percent of all rides in San Francisco are now shared. In every market where we offer it, it’s not about 40 percent of all rides. People are doing it the first time because of price. The second time because they like the experience of actually talking to a human being rather than looking at their phone. Amazing.

But I think we’re back. And so we see this opportunity to bring that human element in it, but it’s not just that. We’re going to now consider not cars as vehicles, but cars as cabins like you think about air travel. What’s going to matter for you is what are the amenities in there. I need to sleep on this ride. I want to play poker with three of my friends on this ride. I want the entertainment experience with 7.1 channel surround sound on this ride. I need the most optimal office with the best connectivity on this particular ride.

I need to have a six person soccer team that’s on this ride and entertain them while they’re going. So the opportunities because we freed up the time now in the vehicle and what you can do in there are going to be many. So the ability to differentiate on what you do during the ride becomes even more significant.

Michal Lev-Ram: All right. I think we’re all looking forward to that and especially to not having to drive. I think we all love driving. We don’t like commuting. That’s the problem. But thank you so much, Raj. We are out of time.

Raj Kapoor: Thank you.

Michal Lev-Ram: Appreciate you being here.

This video was filmed at the Built to Adapt conference in Sausalito, California. The transcript was edited for clarity.

Flying Cars to go With Our 140 Characters was originally published in Built to Adapt on Medium, where people are continuing the conversation by highlighting and responding to this story.


Letter to a Junior Product Manager
Letter to a Junior Product Manager

Some advice for those who’ve chosen product management as a career.Therese Stowell (left) and Emmanouil Kia...

Can Blockchain Help Feed the Hungry?
Can Blockchain Help Feed the Hungry?

Why the World Food Program wants to build a blockchain aid ecosystem.Photo from a woman refugree in Jordan,...

SpringOne 2021

Register Now