When deploying T-Pot 22.04 on AWS using Terraform, it appears that apt-fast is trying to use an empty $myINSTALLPACKAGES.

[tadashi-oya]

Discussed in #1279

^{Originally posted by tadashi-oya March 16, 2023}
When deploying T-Pot 22.04 on AWS using Terraform, it appears that apt-fast is trying to use an empty $myINSTALLPACKAGES.
As a result, the installation proceeds without docker, docker-compose, etc., and ends up incomplete.
...it seems so.

tpotce/iso/installer/install.sh

Lines 307 to 309 in 668a4d9

	echo "### Installing T-Pot dependencies."
	echo
	apt-fast -y install $myINSTALLPACKAGES

$myINSTALLPACKAGES is created from line 25 to line 30, but if install.sh is launched from cloud-init, the file $myINSTALLPACKAGESFILE does not seem to be accessible at this point.

tpotce/iso/installer/install.sh

Lines 25 to 30 in 668a4d9

	if [ -f "../../packages.txt" ];
	then myINSTALLPACKAGESFILE="../../packages.txt"
	elif [ -f "/opt/tpot/packages.txt" ];
	then myINSTALLPACKAGESFILE="/opt/tpot/packages.txt"
	fi
	myINSTALLPACKAGES=$(cat $myINSTALLPACKAGESFILE)

It seems that /opt/tpot will become available after line 707, which is the git clone.

tpotce/iso/installer/install.sh

Lines 702 to 708 in 668a4d9

	# Cloning T-Pot from GitHub
	if ! [ "$myTPOT_DEPLOYMENT_TYPE" == "iso" ];
	then
	fuBANNER "Cloning T-Pot"
	### DEV
	git clone https://github.com/telekom-security/tpotce /opt/tpot
	fi

⚠️ Basic support information (commands are expected to run as root)

• What version of the OS are you currently using lsb_release -a and uname -a?

I am trying to deploy to Debian 11.

[root@smoothbrake:~]# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye
[root@smoothbrake:~]# uname -a
Linux smoothbrake 5.10.0-21-cloud-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux

• What T-Pot version are you currently using?

[root@smoothbrake:~]# cat /opt/tpot/version
22.04.0

• What edition (Standard, Nextgen, etc.) of T-Pot are you running?

admin@ip-172-31-28-200:~/work/terraform/aws$ grep -A 5 -e "tpot_flavor" variables.tf
variable "tpot_flavor" {
  default     = "STANDARD"
  description = "Specify your tpot flavor [STANDARD, HIVE, HIVE_SENSOR, INDUSTRIAL, LOG4J, MEDICAL, MINI, SENSOR]"
}

variable "web_user" {

• What architecture are you running on (i.e. hardware, cloud, VM, etc.)?

[root@smoothbrake:~]# curl http://169.254.169.254/latest/meta-data/ami-id
ami-08dbbf1c0485a4aa8
[root@smoothbrake:~]# curl http://169.254.169.254/latest/meta-data/instance-type
t3.large

• Did you have any problems during the install? If yes, please attach /install.log /install.err.

tadashi@S1056419:~$ grep -A 10 -e "### Installing T-Pot dependencies." install.log
### Installing T-Pot dependencies.

[apt-fast 16:18:26]
[apt-fast 16:18:26]Working... this may take a while.
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
### Removing and holding back problematic packages ...
Reading package lists...
Building dependency tree...

install.log
install.err.log

• How long has your installation been running?

The installation has not been executed perfectly.

• Did you install upgrades, packages or use the update script?

cloud-init is executing install.sh with runcmd.

admin@ip-172-31-28-200:~/work/terraform$ grep -A 5 -e "runcmd" cloud-init.yaml
runcmd:
  - curl -sS --retry 5 https://github.com
  - git clone https://github.com/telekom-security/tpotce /root/tpot
  - /root/tpot/iso/installer/install.sh --type=auto --conf=/root/tpot.conf
  - rm /root/tpot.conf
  - /sbin/shutdown -r now

• Did you modify any scripts or configs? If yes, please attach the changes.

The security group specification has been changed to use an existing security group.

admin@ip-172-31-28-200:~/work/terraform-test/aws$ cat main.tf
provider "aws" {
  access_key = "${jsondecode(file("credentials.json")).AccessKeyId}"
  secret_key = "${jsondecode(file("credentials.json")).SecretAccessKey}"
  token = "${jsondecode(file("credentials.json")).Token}"
  region = var.ec2_region
}

resource "aws_instance" "tpot" {
  ami           = var.ec2_ami[var.ec2_region]
  instance_type = var.ec2_instance_type
  key_name      = var.ec2_ssh_key_name
  subnet_id     = var.ec2_subnet_id
  tags = {
    Name = "T-Pot Honeypot"
  }
  root_block_device {
    volume_type           = "gp2"
    volume_size           = 128
    delete_on_termination = true
  }
  user_data                   = templatefile("../cloud-init.yaml", { timezone = var.timezone, password = var.linux_password, tpot_flavor = var.tpot_flavor, web_user = var.web_user, web_password = var.web_password })
  vpc_security_group_ids      = ["sg-ExistingSecurityGroup"]
  associate_public_ip_address = true
}

• Please provide a screenshot of glances and htop.

The installation has not been executed perfectly.

• How much free disk space is available (df -h)?

[root@smoothbrake:~]# df -h
Filesystem       Size  Used Avail Use% Mounted on
udev             3.9G     0  3.9G   0% /dev
tmpfs            786M  372K  786M   1% /run
/dev/nvme0n1p1   126G  1.9G  119G   2% /
tmpfs            3.9G     0  3.9G   0% /dev/shm
tmpfs            5.0M     0  5.0M   0% /run/lock
/dev/nvme0n1p15  124M   11M  114M   9% /boot/efi
tmpfs            786M     0  786M   0% /run/user/1000

• What is the current container status (dps.sh)?

docker and docker-compose have not been installed.

[root@smoothbrake:~]# dps.sh
grc: docker: No such file or directory
[ ========| System |======== ]
     DATE:  Thu Mar 16 18:36:18 JST 2023
   UPTIME:   18:36:18 up  1:17,  1 user,  load average: 0.00, 0.00, 0.00
    T-POT:  INACTIVE
BLACKHOLE:  DISABLED

NAME                  STATUS                       PORTS
adbhoney              DOWN
ciscoasa              DOWN
citrixhoneypot        DOWN
conpot_guardian_ast   DOWN
conpot_iec104         DOWN
conpot_ipmi           DOWN
conpot_kamstrup_382   DOWN
cowrie                DOWN
ddospot               DOWN
dicompot              DOWN
dionaea               DOWN
elasticpot            DOWN
elasticsearch         DOWN
ewsposter             DOWN
fatt                  DOWN
heralding             DOWN
honeytrap             DOWN
ipphoney              DOWN
kibana                DOWN
logstash              DOWN
mailoney              DOWN
map_data              DOWN
map_redis             DOWN
map_web               DOWN
medpot                DOWN
nginx                 DOWN
p0f                   DOWN
redishoneypot         DOWN
sentrypeer            DOWN
snare                 DOWN
spiderfoot            DOWN
suricata              DOWN
tanner                DOWN
tanner_api            DOWN
tanner_phpox          DOWN
tanner_redis          DOWN

• What is the status of the T-Pot service (systemctl status tpot)?

[root@smoothbrake:~]# systemctl status tpot.service
● tpot.service - tpot
     Loaded: loaded (/etc/systemd/system/tpot.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Thu 2023-03-16 17:19:08 JST; 1h 18min ago
    Process: 576 ExecStartPre=/opt/tpot/bin/updateip.sh (code=exited, status=0/SUCCESS)
    Process: 643 ExecStartPre=/bin/bash -c /opt/tpot/bin/clean.sh on (code=exited, status=0/SUCCESS)
    Process: 693 ExecStartPre=/opt/tpot/bin/tpdclean.sh -y (code=exited, status=0/SUCCESS)
    Process: 712 ExecStartPre=/bin/bash -c /sbin/ethtool --offload $(/sbin/ip address | grep "^2: " | awk '{ print $2 }>
    Process: 718 ExecStartPre=/bin/bash -c /sbin/ethtool -K $(/sbin/ip address | grep "^2: " | awk '{ print $2 }' | tr >
    Process: 724 ExecStartPre=/bin/bash -c /sbin/ip link set $(/sbin/ip address | grep "^2: " | awk '{ print $2 }' | tr>
    Process: 730 ExecStartPre=/opt/tpot/bin/rules.sh /opt/tpot/etc/tpot.yml set (code=exited, status=0/SUCCESS)
    Process: 758 ExecStart=/usr/bin/docker-compose -f /opt/tpot/etc/tpot.yml up --no-color (code=exited, status=203/EXE>
    Process: 759 ExecStartPost=/bin/bash -c /usr/bin/sleep 30 && /usr/sbin/conntrack -D -p udp (code=exited, status=127)
    Process: 790 ExecStopPost=/opt/tpot/bin/rules.sh /opt/tpot/etc/tpot.yml unset (code=exited, status=0/SUCCESS)
   Main PID: 758 (code=exited, status=203/EXEC)
        CPU: 280ms

Mar 16 17:19:03 smoothbrake systemd[1]: Failed to start tpot.
Mar 16 17:19:08 smoothbrake systemd[1]: tpot.service: Failed to schedule restart job: Unit docker.service not found.
Mar 16 17:19:08 smoothbrake systemd[1]: tpot.service: Failed with result 'exit-code'.

• What ports are being occupied? Stop T-Pot systemctl stop tpot and run netstat -tulpen

[root@smoothbrake:~]# netstat -tulpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name

tcp        0      0 0.0.0.0:64295           0.0.0.0:*               LISTEN      0          11562      600/sshd: /usr/sbin
tcp6       0      0 :::64295                :::*                    LISTEN      0          11564      600/sshd: /usr/sbin
udp        0      0 0.0.0.0:68              0.0.0.0:*                           0          2043       412/dhclient

udp        0      0 127.0.0.1:323           0.0.0.0:*                           0          11533      587/chronyd

udp6       0      0 ::1:323                 :::*                                0          11534      587/chronyd

udp6       0      0 fe80::c64:abff:fe4f:546 :::*                                0          11313      483/dhclient

• If a single container shows as DOWN you can run docker logs <container-name> for the latest log entries

docker and docker-compose have not been installed.

[tadashi-oya]

And an adhoc workaround. >>>#1279 (comment)