Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ubuntu 20.04 LTS w/Remnux - update fails #67

Closed
Duocast opened this issue Mar 27, 2022 · 27 comments
Closed

Ubuntu 20.04 LTS w/Remnux - update fails #67

Duocast opened this issue Mar 27, 2022 · 27 comments

Comments

@Duocast
Copy link

Duocast commented Mar 27, 2022

Error occures after running remnux update ("remnux upgrade" works verifying I am on v2022.11.1).
The terminal error shows as
Update returned exit code not zero
Error: Update returned exit code not zero
at ChildProcess. (/snapshot/remnux-cli/remnux-cli.js:569:23)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

saltstack.log results:
Traceback (most recent call last):
File "/usr/bin/salt-call", line 11, in
load_entry_point('salt==3001.8', 'console_scripts', 'salt-call')()
File "/usr/lib/python3/dist-packages/salt/scripts.py", line 439, in salt_call
import salt.cli.call
File "/usr/lib/python3/dist-packages/salt/call.py", line 6, in
import salt.cli.caller
File "/usr/lib/python3/dist-packages/salt/caller.py", line 19, in
import salt.minion
File "/usr/lib/python3/dist-packages/salt/minion.py", line 42, in
import salt.pillar
File "/usr/lib/python3/dist-packages/salt/pillar/init.py", line 19, in
import salt.fileclient
File "/usr/lib/python3/dist-packages/salt/fileclient.py", line 32, in
import salt.utils.templates
File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 27, in
import salt.utils.jinja
File "/usr/lib/python3/dist-packages/salt/utils/jinja.py", line 31, in
from jinja2 import BaseLoader, Markup, TemplateNotFound, nodes
ImportError: cannot import name 'Markup' from 'jinja2' (/usr/local/lib/python3.8/dist-packages/jinja2/init.py)

Looks to be related to python? Any guidance would be helpful. Thanks!
(This is on a fresh installation of Ubuntu 20.04 LTS and a fresh Remnux install following the guidance of the Remnux documentation).

Note: The post doesn't allow <>. Everything following an "in" on a line is "module" surrounded in arrow brackets.

@digitalsleuth
Copy link

Hi @Duocast , did you happen to attempt to install or uninstall SaltStack prior to running the remnux CLI? This looks curious as it seems to suggest part of the required salt components are missing.

Also, with the SaltStack log results, is that the entirety of the log file, or just the error? If it's just a part of the log, can you upload the whole thing? There might be more hints in there.

Cheers!

@Duocast
Copy link
Author

Duocast commented Mar 27, 2022

No attempt to uninstall or reinstall Saltstack.
This is the entire saltstack.log file.
This is from a fresh install.

@lennyzeltser
Copy link
Contributor

Looks like a recently reported SaltStack issue. @digitalsleuth, any chance you'd be able to look into it for a workaround?

@digitalsleuth
Copy link

Sure thing, I'm on it!

@digitalsleuth
Copy link

It looks like Saltstack has updated their version about an hour ago to 3004.1 which includes the pinning of pre-reqs for jinja to 3.0.3. I'm going to test against the newest version to confirm everything works, then I would probably recommend we upgrade the CLI to 3004.1. More to follow.

@digitalsleuth
Copy link

Looks like it didn't work, I'll keep working on a workaround :)

@phos2lampsei
Copy link

phos2lampsei commented Apr 1, 2022

Hello everyone. I am new here but I ran into this same issue while trying to do a remnux-cli install in add-on mode for a recently built SIFT on Ubuntu 20.04 VM. I just figured out a workaround a few minutes ago and was trying to figure out how to reach out to share what I found in case that helps the broader effort.

From what I have found the issue was apparently caused by the changes in Jinja 2 3.1.x builds (my VM had been on Jinja 2 version 3.1.1). I found some suggestions to back rev Jinja 2 to the older 3.0.x version (I used 3.0.3). I was able to use PIP to remove Jinja 2 version 3.1.1 and instead install Jinja 2 version 3.0.3. After I did this, the remnux-cli installer started working for me.

Here are some of the URL's I found in my research if they help at all:

sphinx-doc/sphinx#10291
https://stackoverflow.com/questions/71645272/importerror-cannot-import-name-markup-from-jinja2/71647006#71647006
holoviz/panel#3260

Hope this helps!

@digitalsleuth
Copy link

Hi @phos2lampsei , thank you for providing your feedback! We actually just released a new version 2 days ago which addresses the issue going forward, but existing systems having the issue should do just what you mentioned; install jinja2==3.0.3 and start from there.
Cheers!

@phos2lampsei
Copy link

phos2lampsei commented Apr 1, 2022

Wow...that was fast! Thanks! Yeah I saw the other note but in my case I think either the SIFT install or standard Apt-Get updates had already broken me by upgrading me to Jinja 2 3.1.1 so I was stuck until I did the manual forced downgrade. Been a learning experience for me since this broke on me last night trying to get this done. Not sure if there is a way to make the installer do a downgrade for unfortunate souls like me who got upgraded to the bum build? I just built the VM yesterday with a fresh download of the Remnux-CLI installer so I think in my case because I was in add-on mode I was broken by the existing presence of the newer jinja2 that my VM already had.

@phos2lampsei
Copy link

phos2lampsei commented Apr 1, 2022

It looks like Saltstack has updated their version about an hour ago to 3004.1 which includes the pinning of pre-reqs for jinja to 3.0.3. I'm going to test against the newest version to confirm everything works, then I would probably recommend we upgrade the CLI to 3004.1. More to follow.

So I think I spoke too soon. After fixing the Jinja 2 issues by the forced downgrade to Jina2 3.0.3 things were humming along with my Remnux addon mode install but it still failed on me. I ran into several more package failures. While trying to debug those I noticed apt-get update was referring to a non-existent https://repo.saltproject.io/py3/ubuntu/20.04/amd64/3001/ directory. I was able to fix this manually in my apt-get config but pointing the path to a newer version and get Apt-Get update happy again. However I downloaded a fresh copy of remnux-cli (per the instructions at https://docs.remnux.org/install-distro/add-to-existing-system) to be sure and retried my install but it seems it is hardcoded to look for the non-existent https://repo.saltproject.io/py3/ubuntu/20.04/amd64/3001/ directory. So it appears that not only has salt project been upgraded several times in the last few days but that the old 3001 folder referenced by the REMnux installer no longer even exists.

I think I am stuck at this point but would be happy to help test any changes when they are ready.

@digitalsleuth
Copy link

@phos2lampsei We were just made aware that the saltstack repo has removed support for 3001, so we're preparing a new CLI tool to support 3004. Currently running tests so we can confirm no breakage, then the changes will be applied.

We'll keep you, and everyone, posted!

@phos2lampsei
Copy link

Sounds like you all are already on top of it! Thanks a lot!!

@lennyzeltser
Copy link
Contributor

@phos2lampsei, this should be fixed now. Please give it a a try!

@Y8765
Copy link

Y8765 commented Apr 4, 2022

With all that have been written here , still encoutering the problem :(
at the moment im using the OVA , but preffering a fresh install
b.t.w. installation with the latest binary dosent works even as addon in sift workstation
hope that you guys will find a solution
thank you for your time and efforts!

@digitalsleuth
Copy link

Hi @Y8765 Can you possibly paste us your error, or upload the saltstack.log file, if applicable?
This will help us sort out the issue.

@phos2lampsei
Copy link

Just had a chance to circle back on this. I pulled down a fresh copy of the remnux-cli, verified the hash and confirmed it was the newer version. While attempting to run it, there was an immediate "unable to locate package salt-minion" failure trying to remove salt-common and salt-minion. I dropped out and confirmed that these packages were not installed. Re-ran another apt-get update and noticed the failure again for the non-existent saltproject /3001 subdirectory (even though I had manually fixed that before). I went back in and manually fixed the /etc/apt/sources.list.d/saltstack.list file again to point it to the /3004 directory again and then made sure apt-get update was happy again. After doing this I was then able to get the new remnux-cli installer to run.

I don't know if my system had just been missed up by the prior testing and workaround attempts but I wonder if for those who tried the install before (either successfully when the /3001 saltproject directory still existed or during the failure period after that /3001 directory went away) that there may also be inconsistent machine artifacts that require a bit of cleanup before things will be happy again.

Personally I think I may be finally fixed but raising this for awareness in case artifacts from the old /3001 saltstack repo cause other users install issues that may warrant a bit of automated cleanup in the install script.

Thanks again for your speedy attention in updating the installer!

@Y8765
Copy link

Y8765 commented Apr 5, 2022

HI !
Tried again today on fresh install , seems to work smoothly.
i think for the best of everyone i might will upload my ready machine for everyone for easy use.
thank you

@Y8765
Copy link

Y8765 commented Apr 5, 2022

Hi @Y8765 Can you possibly paste us your error, or upload the saltstack.log file, if applicable? This will help us sort out the issue.

i've deleted that machine so log source isnt available :(

@Y8765
Copy link

Y8765 commented Apr 5, 2022

now encountering this issue - Fresh installation
Running: echo "wireshark-common wireshark-common/install-setuid select True"| debconf-set-selections; dpkg-reconfigure -f noninteractive wireshark-common

Running: wireshark
Running: ssh
Update returned exit code not zero
Error: Update returned exit code not zero
at ChildProcess. (/snapshot/remnux-cli/remnux-cli.js:569:23)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)

saltstack.log

@digitalsleuth
Copy link

Hi @Y8765 It looks like Network Miner has been updated and the hash will have to be updated as well. We'll get that done today, but as for your install, everything except for Network Miner installed properly.

@Y8765
Copy link

Y8765 commented Apr 5, 2022

Hi @Y8765 It looks like Network Miner has been updated and the hash will have to be updated as well. We'll get that done today, but as for your install, everything except for Network Miner installed properly.

Yeh, i saw that the system backround has been changed to remnux etc
Thank you guys 👌

@digitalsleuth
Copy link

Hi @muawyah , the log file shows that everything except NetworkMiner installed correctly, and the background should change once you reboot.

Please give that a shot and let us know how it goes!

@digitalsleuth
Copy link

You can try running the following script to see if that will work for you:
/usr/local/share/remnux/gnome-config.sh

@digitalsleuth
Copy link

digitalsleuth commented Apr 12, 2022 via email

@digitalsleuth
Copy link

@muawyah I think I see the issue. Try running the command as sudo remnux upgrade and sudo ./usr/local/share/remnux/gnome-config.sh. This should fix your problem.

@digitalsleuth
Copy link

I'm not sure what change you might be looking for, but the background I can see in the screenshot you provided is the actual background that's supposed to be configured. Are you expecting a different background?

@digitalsleuth
Copy link

@muawyah I'm afraid I still don't quite understand what you mean, however the end result you have from the image above is the correct output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants