Skip to content

GCE Feeder Box Tutorial

Jump to bottom Edit New page
PhysK edited this page Jan 22, 2020 · 2 revisions

About GCEFB

  • Special Thanks : John Doe for his work and ingenuity and Beta Testing SuperTransfer2.
  • CleanUp Author : Admin9705 - Yes, I was blue in the face scripting more and cleaning :D
  • Original Poster: xxx
  • Community : Thank you for BETA Testing & Schooling Us to get this right for everyone!

Compute Engine

Basically fill out here what it is

Establishing Billing & Basic Parameters

  1. Ensure to start off by selecting the right category.

  1. When the page first loads, start off by selecting the Compute Engine.

  1. If you have not enabled your billing, you will be prompted to do so. Be careful with your usage and understand that you can be liable for any money owed.

  1. Create a Compute Engine instance:

  1. Recommend Specifications are as Followed!
  • Zone: (pick one closest to your usenet server):
    • us-central1-a (Iowa)
    • europe-west1-b (Belgium)
    • asia-east1-a (Taiwan)
    • we are picking these locations because they have 2.6ghz cores instead of 2.2ghz
  • 8-12 Cores
    • If you go above 8 cores, you will have to uPTSrade your account
    • This means you will have to remember to cancel your server when credits run out
  • 7-11G Memory
  • Ubuntu 16.04 LTS
    • Boot disk: 10G
    • If google added NVMe drivers to 18.04 by the time you are reading this, choose 18.04 instead.
  1. Select Local SSD Scratch Disk. To start, the option is somewhat hidden; but luckily we have a picture here for you where to start.

  1. From here, you going to create x2 Local SSD Scratch Disks. Make sure that you CREATE 2 and make sure they are NVMe versions!

Optional Security

Enter in your public ssh key so you can access the box through your own ssh client.

When you are finished with your establishing your Compute Engine, visit the front page and you should see the something of the following!

Installing PlexGuide GCE Feeder Edition

  1. First install PlexGuide by visiting here: Install PlexGuide

  2. Go through the Install Process and Select the GCE Feeder Edition. If no errors, the program will deploy most of your programs by default!

WARNING - BONEHEAD ALERT - WARNING:

If you get this error below afterwards, it means there is a 99.9 percent chance that you FAILED to create the 2 additional drives that are required.

Configuring Your Static IP & Firewall

First, you must configure your network. Please view the picture below on what to click.

Next, you have to setup your external IP address. If you fail to do this, you will not be able to access the programs!

From there, you will click a top to reserve a static ip address.

Next, you will click firewall rules and add a new one.

Ensure to setup you information as shown below. In regards to the ports for better security, you can copy this line: tcp:9000; tcp:7878; tcp:8989; tcp:6789; tcp:8080; tcp:8000; tcp:7999

If you are having problems, just select the all option; but just keep in mind that this results in poor security.

Now test by going to your programs. IPv4:9000 should work by default due to it being Portainer. If you just applied the rules, wait up-to 1 minute for the modified Google Firewall rules to kick in.

Configuring Your Compute Engine

Make a Teamdrive at drive.google.com and name it media. [Instructions]

Note: If you cannot create a teamdrive, do not see it, or have errors; [Follow This]

Configure Initial Team Drive

In it, create your directories: movies and tv Configure rclone with the teamdrive: rclone config

  1. n/s/q> n

  2. name> teamdrive

  3. storage> type --> drive

  1. client_id> [PRESS ENTER]

  2. client_secret> [PRESS ENTER]

  3. scope> 1

  4. root_folder_id> [PRESS ENTER]

  5. service_account_file> [PRESS ENTER]

  6. headless browser question y/n > n

  7. enter verification code> [PASTE FROM BROWSER]

  8. configure as a Team Drive y/n > y

  9. enter a Team Drive ID> Usually 1, but check to make sure it lines up with media

  1. y to the config if all looks right to you!

  2. q to quit and exit from the rclone config.

Verify That It Works!

Verify that your teamdrive is configured properly with rclone touch teamdrive:/test123 && echo "rclone OK" You should see a file called test123 in your Media teamdrive at drive.google.com/

RClone Cache

Configure rclone cache with your teamdrive with rclone config

name> rcache
Storage> cache
remote> teamdrive:
[ skip plex related questions ]
chunk_size> 10M
info_age> 1h
chunk_total_size> 500M

Install rclone cache & unionfs service daemons: ansible-playbook /opt/plexguide/ansible/plexguide.yml --tags rcache

Verify rclone cache is mounted with systemctl status rcache. If it isn't, check the error message and restart it with:

systemctl enable rcache systemctl restart rcache

Verify that there are teamdrive directories you made eariler in /mnt/rcache

Verify unionfs is mounted with systemctl status unionfsc. If it isn't, check the error message and restart it with systemctl restart unionfsc.

Utilizing SuperTransfer2

Before you start this process, ensure that you utilize CloudCMD. By default, this program will deploy and will be installed for you! CloudCMD allows you to upload your files to your server just by dragging and dropping your files! Visit http://your-ipv4:7999. The default username and password is plex | guide

NOTE: You can change the username and password if you wish once your inside the program. For more information, visit the [CloudCMD Wiki Page]

Load PlexGuide and Goto PTS SuperTransfer2. Now press any key and enter y to start the configuration process.

Load up http://yourip:8000 and you will see where you will upload your json files. Follow the information below to create your JSON Files

Create JSON Files & Upload using CLOUDCMD

Visit: https://github.com/MHA-Team/PTS-Team/wiki/ST2-JSON-Files-&-CloudCMD-Upload

Follow the instructions properly!

Conclusion

If you followed everything.... (fill later)

Troubleshooting

Troubleshooting:

  1. Make sure user 1000 can read and write into /root/.config/rclone/rclone.conf
  2. Make sure there is a proper config in /root/.config/rclone/rclone.conf (sometimes, the config ends up in ~/.config/rclone/rclone.conf)
  3. Make sure the line user_allow_other is uncommented in /etc/fuse.conf

PTS Forum Support: GCEFB

https://pgblitz.com/forums/PTS-3rd-party-programs.148/

Visit our Website By Click the Picture Below! N|Demo

N|Solid N|Solid

Supertransfer2

Pre-requisite

**Google Drive Service Accounts ** Go to http://console.developers.google.com and navigate to the credentials area to create new credentials - the quickest way is through the 'Manage Service Accounts' page.

Service Account Name: name account (e.g. GSA1, GSA2 etc) Role: leave blank Check 'Furnish New Private Key' Check 'JSON' under key type Check 'Enable G Suite Domain-wide Delegation'

And save - this will download the JSON to your HDD. Repeat this 10-50 times (that's plenty really)

Configure SuperTransfer2

  • Optional - Install BBR Kernel Mod from the Plexguide menu under Settings and Tools and select Kernel Mods then its Option 1.

---- OLD NOTES

SSH into your box and switch to root with sudo su

Update your server with apt-get update && apt-get uPTSrade -y

Mount your first NVMe drive to /mnt, and the other to /nvme2. We will be using /mnt as the directory to download files to, and /nvme2 as the directory that files are unpacked to and uploaded to gdrive. (note: we will be symlinking directories in /nvme2, not writing to it directly)

Make sure you have both NVMe disks on your server with lsblk | grep nvme. You should see two unmounted NVMEe disks named nvme0n1 and nvme0n2.

Mount your NVMe disks.

mkfs.ext4 -F /dev/nvme0n1
mount -o discard,defaults,nobarrier /dev/nvme0n1 /mnt
chmod a+w /mnt
echo UUID=`blkid | grep nvme0n1 | cut -f2 -d'"'` /mnt ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

mkdir -p /nvme2
mkfs.ext4 -F /dev/nvme0n2
mount -o discard,defaults,nobarrier /dev/nvme0n2 /nvme2
chmod a+w /nvme2
echo UUID=`blkid | grep nvme0n2 | cut -f2 -d'"'` /nvme2 ext4 discard,defaults,nobarrier,nofail 0 2 | tee -a /etc/fstab

Verify if they mounted properly.

ls /mnt | grep -q 'lost+found' && echo "nvme1 mount OK!" || echo "nvme1 mount FAIL"
ls /nvme2 | grep -q 'lost+found' && echo "nvme2 mount OK!" || echo "nvme2 mount FAIL"

Install Plexguide with wget -qO- PTS/install.sh | bash

- Settings:
  - No second HDD

- Install:
  - Sonarr
  - Radarr
  - Sabnzbd

Verify the ports are open: for port in 7878 8989 8080 9000; do nc -w1 -zv $(curl -s icanhazip.com) $port; done

Create passwords for each app and restart the containers: for container in portainer radarr sonarr sabnzbd; do docker restart $container; done

Make a symlink for /mnt/move and /mnt/sab/complete to /nvme/... This will allow usenet articles to unpack and upload a little faster, since it's on a separate disk. You can repeat this process with nzbget, if you prefer it over sabnzbd.

mv /mnt/move /nvme2/move
ln -s /nvme2/move /mnt

mkdir -p /nvme2/sab/complete
rm -r /mnt/sab/complete
ln -s /nvme2/sab/complete /mnt/sab/

chown -R 1000:1000 /mnt
chown -R 1000:1000 /nvme2

NOTE: no menu items for these yet. Please install manually.

Install rclone beta: curl -s https://rclone.org/install.sh | bash -s beta

chown -R 1000:1000 /root chmod 775 /root/.config/rclone/rclone.conf

Wiki - Wiki

Useful Starter Links

  1. Introduction
  2. G-Suite Signup
  3. Recommended NewsHosts
  4. Change Log

Prior Planning 101

  1. Recommended Pre-Reading
  2. Server - Storage Planning
  3. Usenet or BitTorrent
  4. PTS Editions
  5. PTS Folder Structure
  6. PTS-Repos
  7. Common Issues

Deploy & Config

PTS-Team
  1. SSH Server Access
  2. Install PTS
  3. WatchTower
  4. Remote Path Mappings

Data Transport
  1. PTS Clone
    1. Google OAuth Keys
    2. PTS Move
    3. PTS Blitz
    4. 2nd HD Option

Key Components

  1. Traefik
  2. Port Guard
  3. PTS Shield
  4. PTS Press
  5. Google Cloud - GCE
    1. Automated setup
  6. Hetzner Cloud
  7. PTS Fork
  8. Extra PTS Commands

Blitz App Info

  1. Accessing PTS Apps
  2. App Port Scheme

Tools & Services

Core Apps [Expand]
  1. Emby
  2. Jackett
  3. Jellyfin
  4. LazyLibrarian
  5. Lidarr
  6. Netdata
  7. NZBGet
  8. Ombi
  9. Plex
  10. Portainer
  11. qBittorrent
  12. Radarr
  13. Radarr4k
  14. RadarrHDR
  15. RuTorrent
  16. SABnzbd
  17. Sonarr
  18. Sonarr4k
  19. SonarrHDR
  20. Tautulli
  21. JDownloader2
Community Apps [Expand]
  1. AirSonic
  2. Bazarr
  3. Bitwarden
  4. BookSonic
  5. cAdvisor
  6. Cloud Commander
  7. Deluge
  8. DelugeVPN
  9. Dozzle
  10. Duplicati
  11. Filebrowser
  12. FlexGet
  13. Gazee
  14. Headphones
  15. Heimdall
  16. HomeAssistant
  17. jd2-openvpn
  18. Kitana
  19. Logarr
  20. MakeMKV
  21. MariaDB
  22. McMyadmin
  23. MEDUSA
  24. Mellow
  25. Miniflux
  26. Monitorr
  27. Muximux
  28. Mylar
  29. NextCloud
  30. NowShowing
  31. NZBHydra2
  32. Organizr
  33. plpp
  34. pyLoad
  35. qBittorrent
  36. QBittorrent VPN
  37. radarrsyncarr
  38. Resilio
  39. rflood-openvpn
  40. rutorrent-openvpn
  41. Shoko Anime Server
  42. SpeedTest
  43. SyncLounge
  44. Teamspeak3
  45. The Lounge
  46. Transmission
  47. Trakt.or
  48. Ubooquity
  49. Varken
  50. vnc-xfce
  51. vnstat
  52. ZeroTier

Misc Info & Articles

  1. Change Server Time
  2. Fail2Ban
  3. Custom Containers
  4. Deploying Varken, Influx, Telegraf, Grafana
  5. cloneclean commands
  6. Understanding htop
Scripts
  1. Scripting Area
    1. Auto Upload for Sonarr and Lidarr
    2. Auto Delete Log
    3. Plex Plugin DB Backup
    4. Plex fast BACKUP
    5. Delete All Unmonitored from Radarr
Clone this wiki locally