Welcome to the documentation for Stanford AA 241x: Spring 2018. Here we will describe the architecture for our Automated Drone Bidding System which goal is to simulate Uber Elevate’s eVTOL service using DJI drones.
During a simulation, we will have 2 types of servers running: MainServer (MainServer.py) and TeamServer (MainServer.py). There will be one MainServer running (by the class admins) and one TeamServer running for every team in the simulation. The MainServer has a two-way communication with the TeamServers, while TeamServers cannot communicate with each other.
Each team has multiple drones flying (only one physical one). The drone_id of the physical drone must be 0. Each physical drone is a DJI Spark and is controlled by a RemoteController which has an Android device plugged in to it. The Android device runs our own version of the DJI This app receives information from a server running on the Team’s computer, which is used to control the drone.
In the next sections we will provide the necessary documentation for the teams to communicate with the MainServer through their respective TeamServers and for teams to control their drones.
TheMainServer is connected with the TeamServers using the Twisted protocol. To start communication, the admins will first launch the MainServer and be open for communication. IP address and Port number will be provided to the teams. To initiate communication teams should simply run their TeamServer.py program, which has everything set up for communication. Make sure to run TeamServer.py <team_id> <password> to be able to login. Also make sure to install all the requirements in requirements.txt using pip install -r requirements.txt and use python3.
Teams should make sure that they have all the logic implemented once they run their server. To send a json message to the server, call self.sendToServer(<json_string>), which is in the TeamClientSideProtocol class. When teams send messages to MainServer, they must include a 'type' field. The types will be defined below.
Teams MUST get the TeamServer.py and other relevant files from us to be able and complete all the TODOs listed. Please ask us questions if there is any confusion.
The communication should follow the structure below:
Team sends message to Main to be authenticated by the system. If a team does not do this, any message they send will be ignored.
Auth request: Team sends ‘auth’ message to login. Teams must let us know what their team_id and password is beforehand.
# TeamServer -> MainServer
{
'type': 'auth',
'team_id': (int), # ID of team.
'password': (string) # Password used for login.
}
Auth response: Server responds back with auth result.
# MainServer -> TeamServer
{
'type': 'response',
'result': (text), # 'error' OR 'success'.
'msg': (text) # Describes error in case there is one. None if no error.
}
Now that we have established communication between the TeamServers and the MainServer, there are multiple things that can happen. The team can log out, send drone state information, can submit a bid, or can accept/reject a bid that is offered to them. Lets define some of the protocols that will be going on at the same time.
The MainServer will first load all the requests from a csv file written by the admins. Every request has a certain timestamp, which denotes at what time the request will be broadcasted. At that time, MainServer will send a flight request to all TeamServers, which must respond back whether they want to bid or not. If they do, they must specify estimated times for the flight and their bid price. Once a confirms/starts a task, pickups up passengers (from port from_port) drone finishes its task, or if something goes unexpected, the team must send a task update.
Bid Broadcast:
MainServersends a message with the following format to the teams.
# MainServer -> TeamServer
{
'type': 'request',
'request': {
'request_id': (int), # Used to identify the request.
'k_passengers': (int), # Number of passengers for request. This might affect prices.
'expected_price': (float), # Expected price. Teams should bid close to this price, but there is no restriction.
'from_port': (int), # Port for pickup.
'to_port': (int) # Port for delivery.
}
}
Bidding: The TeamServers respond back with their bid decision. IMPORTANT NOTE: teams must write function that determines if they want to bid, calculate their ETA and the price they want to set. We will set a timeout of 5~10 seconds to let teams respond. After we receive all responses, we decide who wins the bid, and we send back the results to all teams that submitted bids. The API for those results are described in the following section. PLEASE DO NOT SUBMIT TWO DIFFERENT BIDS WITH THE SAME DRONE IF YOU HAVE NOT YET RECEIVED THE FIRST BID RESULTS
# TeamServer -> MainServer
{ 'type': 'bid',
'bid': {
'request_id': (int), # Request that team wants to fulfill.
'accepted': (bool) . # True if submitting bid. Next attributes must be filled if True, leave None if False.
'drone_id': (int), # Drone that will fulfill the request.
'seconds_expected': (int),# Number of seconds that team expect trip to take. Time starts when the team confirms the task.
# Teams will be penalized if they do not fulfill the request on time. Helps determine if team will
# win the bid.
'price': (float) # bid price. This helps determine if teams will win the bid.
}
}
Task: Now that teams have submitted their bids,
MainServerdecides which team ‘wins’ the request and send that information back to the teams.
# TeamServer -> MainServer
{
'type': 'bid_result',
'result': (text), # 'win' or 'lose'
'request_id': (int), # Request which team must fulfill,
'task': { # Task is ONLY sent to the team who wins the bid. Must be fulfilled with the drone which they specified, listed on the task.
'k_passengers': (int), # The next properties are the same ones from request, and they will be left as None if result is 'lose'.
'time_expected': (datetime), # Time we expect you to finish task.
'price': (float), # Price we are paying.
'from_port': (int), #
'to_port': (int), #
'drone_id': (int) #
}
}
Task Update: The team must send an update of the task once its done or if something happens. Send ‘confirm’ to confirm that you will perform the task that we sent. Send ‘deny’ if you will not perform the task . ‘pickup’ when you pick up a passengers. ‘success’ if you deliver the passenger, and ‘failure’ if you fail to deliver the passengers. When you send a ‘pickup’ or ‘success’ message, make sure that you have sent the state of the drone at hand very recently (before sending the message), since we will check your location and make sure that the location of the from_port/to_port is close in distance from your drones last reported location.
# TeamServer -> MainServer
{
'type': 'task_update'
'request_id': (int)
'status': (text) # 'confirm','deny', 'pickup', 'success' or 'failure'
'msg': (option) # Describes what happened if there is a failure. None if success.
}
Every X seconds, TeamServers must send the MainServer their drone state information, so we can keep track of the state of the world. The message must have the following format. The state ‘type’ must be either ‘physical’ (for physical drone) or ‘simulation’ (for simulation drone). ‘pax’ must be an int representing the number of passengers. ‘battery_left’ must be 0 <= x <= 100. ‘fulfilling’ must be the request_id of the request the drone is fulfilling, or None. ‘state’ must be ‘working’ or ‘not_working’.
# TeamServer -> MainServer
{
'type': 'drone_state'
'drone_state': {
'drone_id': (int), # ID of drone being updated.
'longitude': (text), #
'latitude': (text), #
'altitude': (text), #
'velocity': (list size 3), # [vx, vy, vz] Velocity vector.
'k_passengers': (int), #
'battery_left': (float), # 0 <= x <= 100
'state': (text), # 'working' or 'not_working'
'fulfilling': (bool), # True if drone is fulfilling a request.
'next_port': (int), # Port where drone is headed to if fulfilling is True.
# i.e. if drone is looking for passenger, then next_port = from_port.
# if drone has passengers, then next_port = to_port. None if fulfilling is false.
}
}
Every time the TeamServer sends a message to the MainServer, the MainServer will send a response back acknowledging the message. Note that for 'type': 'auth', this response message differs. The team can ignore this message, but make sure to be aware if they receive an error response. Make sure to be receiving the correct responses.
# MainServer -> TeamServer
{
'type': 'response',
'result': (text), # 'error' OR 'thanks'.
'msg': (text) # Describes error in case there is one. None if no error.
}
Running / Compiling:
-
To download the app onto an Android device, you will first need to download Android Studio (from here: https://developer.android.com/studio/). Then, open Android Studio and select “Open an existing Android Studio project”, opening the ENTIRE DroneController folder.
-
Enable USB debugging on your Android device. This can be done in the settings. Plug your device into your laptop via USB and run the application from Android studio (using the green play button). Default settings should be fine, but if not, then use build tool “28.0.0 rc1” with SDK 26 to compile this app. The app should be able run on any device with SDK version ≥ 19 (i.e. ≥ Android 4.4).
IMPORTANT: When you open the app for the first time, you must be connected to the internet. The app will need to download necessary tools used for connecting with the drone. Only after you see the “SDK Registration Succeeded” message may you connect to the drone’s WiFi network.
- Now we’re ready to test the communication. We are using python3 for this code (python2 will give you weird error messages!). On your laptop, install the packages python-socketio, eventlet, and flask. You can do this with the command
pip install package_name
where package_name is the name of the desired package.
-
Run the attached socketio-server.py program in your terminal.
-
In your terminal, run the command
ABS_PATH/platform-tools/adb reverse tcp:9001 tcp:9090
where ABS_PATH is the absolute path to the folder containing your SDK Tools. adb is a command line utility provided in the Android SDK tools that allows for port forwarding. You can find the correct location of the SDK folder by going to Tools --> SDK Manager in Android Studio. For example, the command that I run is
/Users/andrewchang/Library/Android/sdk/platform-tools/adb
Otherwise, you can download the standalone SDK tools here: https://developer.android.com/studio/releases/platform-tools
Make sure that the second port number is consistent with that which you used in the socketio-server program!
IMPORTANT: THIS MUST BE DONE EVERY TIME YOU RECONNECT YOUR DEVICE TO YOUR LAPTOP!
-
In the app, make sure “http://127.0.0.1:9001” is in the URL input box and click “connect”. You should be able to see “connect xxxxxx (a long hexadecimal id)” in the python output and see an Android Toast saying “Connect!”
-
To connect with the Spark, connect with the drone’s WiFi network and click the “Open” button in the home page of the app. In the new page, click “Get Drone State.” You will be able to see the updated drone state in your python terminal. In the Android Studio log, you will able to see the commands sent from the laptop to the Android app.
-
The implementation of the controls is left up to the individual groups. To start, look at the FlyTask in DroneControlActivity.
Extra Information:
Network Connection:
The Android app uses Socket.io (somewhat like a WebSocket client/server) to send “events” with associated JSON data for communication.
Get Drone State:
After connecting to the Socket.io server and connecting with the drone, you can click “Get Drone State” to poll drone state every second (you can change the frequency yourself in DroneControlActivity.java). The drone state (in JSON Format) will be shown on the screen and will be sent to the Socket.io server, after which the server may respond to the state information.
Control Drone:
After the drone takes off and we finish all the drone configuration, we would like to set the target position (longitude, latitude and altitude) and velocity (vx, vy, vz) and then click “Fly to” to let it fly to a specific position. The current code has some basic code, which should be completed by students. [PENDING UPDATES: ADDITIONAL CODE MAY BE PROVIDED]