Awesome Self-Driving Cars

We wanted flying cars, instead we got 140 characters. - Peter Thiel

We want self-driving cars, and we want them with an awesome list. - Not Peter Thiel

Introduction

A curated list of all awesome things related to self-driving car.

Contributing

Contributions are highly appreciated. Please send me pull request. Make sure the stuff you add is actually awesome. We are not trying to include everything here, only awesome stuff.

Table of Contents

• Datasets
• Simulators
• Courses
• Papers & Blogs
• Big Players
• Legislation

Datasets

• KITTI Vision Benchmark Suite [Images] - Large vision benchmark dataset with objection detection evaluation training/testing images and leaderboard on cars and pedestrians.
• Cityscapes - Semantic, instance-wise, dense pixel annotations of 30 classes.
• comma.ai's Driving Dataset [Videos] - Seven and a quarter hours (~ 80 GB) of largely highway driving. With this dataset, comma.ai's founder George Hotz trained a self-driving car all by himself.
• Udacity's Driving Dataset [Videos] - Eight hours (over 280 GB) of driving data collected for their open source self-driving car challenges. Udacity also provides convenient scripts to port the data.
• German Traffic Sign [Images] - More than 50,000 images and 40 classes of traffic signs. Excellent resource to benchmark your traffic sign classifier.
• Washington DC's Lidar Data - Lidar point cloud of the entire Washington DC area is made available by the District of Columbia’s Office of the Chief Technology Officer (OCTO).
• Apolloscape - Apolloscape provides images with 10x higher resolution and pixel-level annotation. And also Provides multiple levels of scene complexity.

Simulators

• Udacity's Self-Driving Car Simulator - This simulator is built for Udacity's Self-Driving Car Nanodegree to teach students how to train cars how to navigate road courses using deep learning. It is used for the project of Behavioral Cloning.
• Microsoft's AirSim - An open-source and cross platform simulator built for drones and other vehicles. AirSim is designed as a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles.
• MIT's Moral Machine - Moral machine provides a "platform for 1) building a crowd-sourced picture of human opinion on how machines should make decisions when faced with moral dilemmas, and 2) crowd-sourcing assembly and discussion of potential scenarios of moral consequence". If you are a fan of the trolley problem, you can't miss this.
• MIT's Google Self-Driving Car Simulator - Self-driving car simulated completely by visual programming language Scratch.
• BARK - An open-source, semantic simulator built to develop and benchmark novel behavior planners. Runs on multiple platforms and can easily be installed via PyPi.

Courses

• Udacity Self-Driving Car Nanodegree - Udacity's flagship program is sponsored by many self-driving car hiring partners. The nanodegree program includes 3 terms: 1) Term 1: Computer Vision and Deep Learning, 2) Term 2: Sensor Fusion, Localization, and Control, and 3) Term 3: Path Planning, Elective, and Systems. Each term costs $800.
• MIT 6.S094: Deep Learning for Self-Driving Cars - This class is an introduction to the practice of deep learning through the applied theme of building a self-driving car. It is open to beginners and is designed for those who are new to machine learning, but it can also benefit advanced researchers in the field looking for a practical overview of deep learning methods and their application. By the way, it's free!
• BitTiger Build Your Own Autonomous Vehicle Mastery Program - Two weeks of live classes in Bay Area taught by engineers from Vector.ai on building a self-driving mini car from ground up. Topics include deep learning, ROS, sensors, computer vision, localization, mapping and control. The program costs $7,000.

Papers & Blogs

• Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art - State-of-the-art survey on computer vision-related problems datasets and methods for self-driving cars.
• End to End Learning for Self-Driving Cars - Nvidia's ground breaking paper on using end to end learning (i.e., raw camera images as the input and steering commands as the output) with a Convolutional Neural Network (CNN) for behavioral cloning.
• Learning a Driving Simulator - comma.ai's approach for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road. This paper investigates variational autoencoders with classical and learned cost functions using generative adversarial networks for embedding road frames. A transition model is learned in the embedded space using action conditioned Recurrent Neural Networks (RNNs).
• The Third Transportation Revolution - Awesome blog post by John Zimmer on Lyft's vision for self-driving cars. Spoiler alert, John predicts self-driving cars will account for the majority of Lyft rides within 5 years. And by 2025, private car ownership will all-but end in major U.S. cities.
• 16 Questions About Self-Driving Cars - a16z's Frank Chen goes over the 16 most commonly asked questions, technical and non-technical, about self-driving cars.
• Ways to think about cars - Awesome blog post by a16z's Benedict Evans on electric cars, on-demand car services, and self-driving cars.
• Cars and second order consequences - Benedict Evans on the impact of electric and autonomy on cars and beyond.
• An Introduction to LIDAR - Awesome introduction by Voyage about the key sensor of self-driving cars.

Big Players

• comma.ai - The Android of self-driving cars (with the iPhone being Tesla) claimed by their founder George Hotz. They open sourced their dataset and software, driving agent, and research platform after some disagreement with the NHTSA.
• Cruise Automation - San Francisco-based startup was acquired by GM for $1B. They regularly post their self-driving videos on their YouTube channel.
• drive.ai - Silicon Valley startup founded by former lab mates out of Stanford University’s Artificial Intelligence Lab. Working on creating AI software for autonomous vehicles with deep learning. See their impressive drive demo here.
• Ford - Invested $1B in an artificial intelligence startup Argo AI, Ford plans to have self-driving cars in commercial operation for a ride-hailing service by 2021.
• GM - Invested $500M in Lyft and bought Cruise Automation for $1B, GM is expanding its R&D to Silicon Valley and targeting to soon test self-driving taxis on public roads.
• nuTonomy - Startup spun out from MIT launched world's first self-driving taxi service in Singapore. nuTonomy is currently expanding its self-driving car road test to Boston.
• NIO - electric car startup formerly known as NextEV, demonstrated world's fastest (top speed of 160 mph) autonomous lap with their EP9.
• NVIDIA - NVIDIA is in a market dominant position for the use of their GPU for deep learning. Naturally self-driving car is a huge opportunity for them. NVIDIA already demonstrated their own self-driving car BB8. They also build their open AI car computing platform and software development kit.
• Otto - Self-driving truck startup acquired by Uber for $680M. Made to the awesome list by delivering beer from Denver to Colorado Springs with 120 miles of highway self-driving. Since the acquisition, Uber has reportedly terminated the testing on self-driving turck and directed the talent of Otto to the development of their self-driving ride-sharing service.
• Tesla - Currently the only automobile manufacturer sells hardware-ready vehicles. With thousands of self-driving capable vehicles already on the road and millions of miles driven, Tesla is using fleet learning to make their autopilot smarter and smarter.
• Uber - Since self-driving cars are existential crisis to Uber, they have been actively building and testing their self-driving cars in Pittsburgh, PA and Tempe, Arizona. They also bought Otto and teamed up with Volvo and Daimler on deployment of self-driving cars.
• Udacity - Founded by self-driving car pioneer Sebastian Thrun, Udacity is democratizing education and providing the best self-driving car education program through their nanodegree.
• Waymo - Google's self-driving car company has driven way more miles than any other companies also with a much smaller disengagement rate.

Legislation

• Arizona - Executive order 2015-09 was signed to direct all state agencies to undertake any necessary steps to support the testing and operation of self-driving vehicles on public roads within Arizona. The executive order also specifies that in Arizona the operator of a self-driving vehicle does not have to be physically inside the vehicle. The vehicle can be directed remotely in self-driving mode.
• Colorado - Colorado passes first law to regulate self-driving cars. Companies who plan to test self-driving cars in Colorado need to first check in with the state Department of Transportation and State Patrol.
• California - Application required for testing self-driving cars. Manufacturers are required to provide accident reports and disengagement reports. The regulations for post-testing deployment of self-driving cars have been drafted and will establish the requirements for manufacturers to meet prior to operation on California’s on public roads.
• New York - It is announced on May 10, 2017 that the state is accepting application for self-driving car testing. The pilot testing program is included in FY 2018 and can be further extended.
• Texas - Bill SB 2205 is proposed to implement minimum safety requirements and accelerate testing of self-driving cars on public roads. The bill passed the Texas Senate Transportation Committee and it is currently pending Senate debate and referral to the House.
• Virginia - Arguably the most friendly state to self-driving cars with no application or permit required and $25M per year fund set to facilitate self-driving cars. Virginia Automated Corridors is announced to offer self-driving car developers the opportunity to test their technologies on Virginia roads covering more than 70 miles of interstates and arterials in the Northern Virginia region.