Message in allsky log

[maserowik]

I just did a new pull request and reconfigured my system. Looking at the log file i see the following

Oct 2 17:57:17 Pi-AllSky allsky.sh[2310]: FILE_TO_UPLOAD='/home/pi/allsky/tmp/image.jpg'
Oct 2 17:57:17 Pi-AllSky allsky.sh[2310]: REMOTE_DIR='current'
Oct 2 17:57:17 Pi-AllSky allsky.sh[2310]: TEMP_NAME='NI-16525'

I then noticed the file name NI -16525 uploaded to my webhosting company.

I tried to view it and it is i get random images on the screen

any idea what this is?

also when i stop and start the allsky service i get a different file but still starts with NI

Mike

[linuxkidd]

Comparing the EXIF data Adobe Photoshop 22.4 (Windows) and the Modified date/time 2021:06:24 18:07, that matches the NotRunning.jpg file.

If I were to guess..

• It got uploaded by mistake
• The upload couldn't be moved to the final location, so the TEMP named file didn't get renamed on your web host.

[maserowik]

So is this something i need to look at on my side or a "non error"

[linuxkidd]

Is there anything else in the log after your snippet above? If so, providing it may give some additional insight.

Otherwise, you may need to check permissions in the directory or for the final file name you're intending the upload to write to.

Note: If you paste log snippets, please be sure to highlight them, then click the <> button ( code ) in the tool bar above the comment box so that it formats in a way that's easier to work with.

Thanks!

[EricClaeys]

The new "upload.sh" script lftp's a file using a temporary name, then removes the prior file and renames the temporary name to the final name.
For example, if you are uploading "allsky-20211002.mp4" and want the final name to be "allsky-mp4", lftp does this:

upload allsky-20211002.mp4 to a file called TL-12345    # TL for Time Lapse
rm allsky.mp4  # final file name
mv TL-12345 allsky.mp4

The temporary files are named XX-RANDOM_NUMBER; each of the scripts that call upload.sh use a different value for "XX". "NI" is for Notification Images, "TL" for Time Lapse, etc. So when you saw NI-16525 it was one of the Notification Images (NotRunning.jpg). I used Photoshop to create all the Notification Images which is why you see "Photoshop" in the file. And you will get a different NI-* file every time because of the random number.

Uploads used to simply upload allsky-20211002 to allsky.mp4, but since the upload could take a long time on a slow network, it was possible for the web server to try and access allsky.mp4 as it was being uploaded, which would produce an error. Or the web server would be reading allsky.mp4 and lftp would try to update it, which would also produce an error.

The fact that you are seeing the NI-* files after lftp finishes (you'll always see them while the file is being uploaded), implies an error occurred during lftp. When an error happens, you'll see FILE_TO_UPLOAD=xxx, REMOTE_DIR=yyy, and other lines used by lftp in the log file. If you look after those entries in the log file you should see an lftp error that tells you what the problem is.
If uploads were working a couple days ago and you didn't change anything except for installing the new upload changes, then there's a bug somewhere.
Are uploads of your images working?
Would you please include more of the log file?

[EricClaeys]

BTW, I also notice some temp files on my remote server - in fact right now I have 2 from image.jpg files from a couple days ago, so there's clearly a bug somewhere.

[maserowik]

so i did review my log file and found the following. the temp file appears to be created when I stop or start the allsky service with the following command sudo service allsky stop/start

i have attached a error in the log file that is looking for a non existent file call setting_auto.json i check the location /etc/raspap and that file is not there.

attached a screen capture of the error in the log file.

my etc/raspap directory on pi

Mike

[EricClaeys]

@maserowik Check your IMG_DIR in scripts/ftp_settings.sh. It looks like it's set to "current" which likely doesn't exist on your hosting server. If your startrails, keogram and timelapse files ARE uploading, then your MP4DIR, KEOGRAM_DIR, and STARTRAILS_DIR are set ok.
I'll look into the setting_auto.json error, but in the meantime, please set CAMERA=ZWO in config.sh.
Let me know how things go.

Thanks for being patient with us as we make changes that will improve the software over time.

[maserowik]

@EricClaeys
I have set ZWO in the config.sh

here is my ftp settings

FTP/SFTP/S3 settings

PROTOCOL='ftp'
USER=' '
PASSWORD=' '
HOST=' '
IMGDIR='/domains/n3bsq.com/public_html/allsky/image/'
MP4DIR='/domains/n3bsq.com/public_html/allsky/videos/'
KEOGRAM_DIR='/domains/n3bsq.com/public_html/allsky/keograms/'
STARTRAILS_DIR='/domains/n3bsq.com/public_html/allsky/startrails/'

hosting site file structure

inside the image folder

not following the "current" comment.

Thanks for all you assistance.

Mike

[maserowik]

@EricClaeys

Hardcoding the ZWO in the config.sh fixed the setting_auto.json message.

[EricClaeys]

@maserowik Thanks Mike. I know what the setting_auto.json bug is just not sure the best way to fix it. We'll eventually implement the ability to have multiple cameras and I'm not sure CAMERA=auto would be appropriate; we may want people to always hard-code the camera or pick it from a drop-down list. 99.9% of the people will only need to do this one.

I just issued a bug fix for upload.sh failure with Notification Images (NI12345). I was using $IMG_DIR which is 'current' and doesn't exist on your hosting server, rather than $IMGDIR which for you is '/domains/n3bsq.com/public_html/allsky/image/'

[maserowik]

thanks for all the work you have done on this project!

[linuxkidd]

Just submitted PR #548 to resolve the jq trying to read settings_auto.json file. Once it's merged, that message should be gone.

[EricClaeys]

You are welcome Mike. I appreciate the thanks; it's what keeps me going.

[maserowik]

@EricClaeys

BTW, I also notice some temp files on my remote server - in fact right now I have 2 from image.jpg files from a couple days ago, so there's clearly a bug somewhere.

just to confirm with the new git pull the files are still appearing on the root of the hosting company I am using.

[EricClaeys]

@maserowik Please check in the log file around the times the files were created on the hosting company server. There should be error messages from upload.sh's lftp as to why it couldn't rename the NI-* and PD-* files (PD is PostData from postData.sh which uploads the small data.json file).

Would you also please download the two NI-* files to your PC/Pi and append ".jpg" to the names, then open them and tell me what they say? That might help me in determining where the bug is.

[maserowik]

@EricClaeys
I have been trying to reproduce the error but it is not playing nice with me. looking at the log files I think I have a idea.
Could the software be trying to upload the image before the FTP_settings.sh is configured?

For the NI files I deleted them off my system already.

Mike

[EricClaeys]

@maserowik I just submitted a bug fix that should fix the PD-* files.

ftp-settings.sh is configured once by the user when they install the software, then isn't changed unless they want to change locations. The upload.sh script calls the ftp-settings.sh script at the top, before lftp is called. Assuming you configured ftp-settings.sh before you started Allsky, there shouldn't be a problem, but obviously there still is.
The best way to debug is to look in the log for error messages.

Near the end of your scripts/copy_notification_image.sh file (possibly line 71), do you see this?
"${ALLSKY_SCRIPTS}/upload.sh" --silent "${IMAGE_TO_USE}" "${IMGDIR}" "${FULL_FILENAME}" "NI"
or is it "${IMG_DIR}" instead of "${IMGDIR}"?

[maserowik]

that may be the issue. I start allsky and then use the gui to modify the settings.

here is the listing of the scritps/copy_notification_image.sh
<
66
67 # We're actually uploading $IMAGE_TO_USE, but show $NOTIFICATIONFILE
68 # in the message since it's more descriptive.
69 echo -e "${ME}: Uploading $(basename "${NOTIFICATIONFILE}")\n"
70 # NI == Notification Image
71 "${ALLSKY_SCRIPTS}/upload.sh" --silent "${IMAGE_TO_USE}" "${IMGDIR}" "${FULL_FILENAME}" "NI"
72 exit $?
73 fi
74 exit 0

[EricClaeys]

@maserowik you have the correct scripts/copy_notification_images.sh. I'll need to see the upload.sh error messages to debug, starting a few lines above the FILE_TO_UPLOAD=xxx line, and going until the last error line.

[maserowik]

@EricClaeys

I been running my allsky all day and have not seen the files show up on the hosting site.
Lets close this for now.

Mike

[linuxkidd]

thx @maserowik .. we'll close this for now. Should it reappear, let us know.