nc.files.download2stream - corupted file bytes

[PatrickPromitzer]

Describe the bug

I looked into the to_gif example and tried to make a an own app for nextcloud.
After some tests I noticed that "nc.files.download2stream(input_file, tmp_in)" is not working as intended.

I made an own version with downloading JSON files, and noticed the following.
Every file with more than 68B is not working, but if the file is smaller, it works.

68B file:
68b_file.json

68B file with one " " character at the end:
68b_file_with_one_space_at_the_end.json

"nc.files.download(nc_file)" is working as intented

Steps/Code to Reproduce

1.) prepare and start a REST API server for debuging
https://github.com/ynsrc/python-simple-rest-api

I added an function do save bytes in a file.
Download the server.py file and add this functions

import pathlib

@api.post("/log_string")
def log_string(args):
    print("------------------------")
    print(time.strftime("%Y-%m-%d %H:%M:%S", time.localtime()))
    if "error" in args:
        print("Error: ", args["error"])
    print(args)
    name = ""
    if "name" in args:
        name = args["name"]
    if "bytes" in args:
        write_bytes_to_file(args["bytes"], name)
    return "OK"

def write_bytes_to_file(data, name):
    if name == "":
        name = "temp.raw"
    temp_folder_parth = pathlib.Path("temp")
    output_file = temp_folder_parth.joinpath(name)
    bytes_data = bytes.fromhex(data)
    with open(output_file, "wb") as f:
        f.write(bytes_data)

    print("Bytes written to file: ", output_file)

2.) download the to_gif example

3.) add "requests" to the requirements.txt

4.) simplify code to only download the file + add debug message

def convert_video_to_gif(input_file: FsNode, nc: NextcloudApp):
    save_path = path.splitext(input_file.user_path)[0] + ".gif"
    nc.log(LogLvl.WARNING, f"Processing:{input_file.user_path} -> {save_path}")
    try:
        nc.files.download2stream(input_file, save_path)
    except Exception as e:
        nc.log(LogLvl.ERROR, str(e))
        nc.notifications.create("Error occurred", "Error information was written to log file")
    with open(save_path, "rb") as f:
        file_bytes = f.read()
    response = requests.post(f"http://127.0.0.1:2998/log_string",
                         json={"ID": "check_json bytes",
                               "name": str(local_file_path.name),
                               "bytes": file_bytes.hex()})

5.) check file in REST API server folder

Expected Results

Downloading the file.

Actual Results

Getting a corrupted file.

Setup configuration

I install the app with an bash file I wrote.

#!/bin/bash
script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
cd "${script_dir}" || exit

APP_ID="basic_file_checker"
APP_SOURCE="127.0.0.1:5000"
NEXTCLOUD_DOCKER_CONTAINER_NAME="nextcloud-server"
DOCKER_CONFIG_NAME="docker_local"

APP_NAME="Basic_File_checker"
APP_VERSION="1.0.0"
APP_SECRET="12345"
APP_PORT="2981"

JSON_INFO="{\"id\":\"${APP_ID}\",\"name\":\"${APP_NAME}\",\"source\":\"${APP_SOURCE}\",\"daemon_config_name\":\"${DOCKER_CONFIG_NAME}\",\"version\":\"${APP_VERSION}\",\"secret\":\"${APP_SECRET}\",\"port\":${APP_PORT},\"routes\":[{\"url\":\".*\",\"verb\":\"GET,POST,PUT,DELETE\",\"access_level\":1,\"headers_to_exclude\":[]}]}"
docker build -t 127.0.0.1:5000/${APP_ID} .
docker push 127.0.0.1:5000/${APP_ID}

echo docker exec ${NEXTCLOUD_DOCKER_CONTAINER_NAME} php occ app_api:app:unregister ${APP_ID} --silent --force || true
docker exec ${NEXTCLOUD_DOCKER_CONTAINER_NAME} php occ app_api:app:unregister ${APP_ID} --silent --force || true
echo docker exec ${NEXTCLOUD_DOCKER_CONTAINER_NAME} php occ app_api:app:register ${APP_ID} --force-scopes --json-info ${JSON_INFO} # docker_scope_all
docker exec ${NEXTCLOUD_DOCKER_CONTAINER_NAME} php occ app_api:app:register ${APP_ID} --force-scopes --json-info ${JSON_INFO} # docker_scope_all

The nextcloud server is in an docker, too.

version: '3.2'

services:
  db:
    image: postgres
    restart: always
    network_mode: host
    volumes:
      - db:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB=nextcloud
      - POSTGRES_USER=nextcloud
      - POSTGRES_PASSWORD=nextcloud
    container_name: nextcloud-db

  app:
    image: nextcloud
    restart: always
    privileged: true
    network_mode: host
    #ports:
    #  - 2980:2980
    volumes:
      - nextcloud:/var/www/html
      - nextcloud_config:/etc/apache2/
    environment:
      - POSTGRES_HOST=127.0.0.1
      - POSTGRES_DB=nextcloud
      - POSTGRES_USER=nextcloud
      - POSTGRES_PASSWORD=nextcloud
    depends_on:
      - db
    container_name: nextcloud-server

  cron:
    image: nextcloud
    restart: always
    privileged: true
    network_mode: host
    volumes:
      - nextcloud:/var/www/html
    entrypoint: /cron.sh
    depends_on:
      - db
    container_name: nextcloud-cron
    extra_hosts:
    - "host.docker.internal:host-gateway"


  nextcloud_appapi_ds:
    image: ghcr.io/nextcloud/nextcloud-appapi-dsp:release
    restart: always
    privileged: true
    network_mode: host
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - NC_HAPROXY_PASSWORD=some_secure_password
    container_name: nextcloud-appapi-dsp
    extra_hosts:
      - "host.docker.internal:host-gateway"

volumes:
  db:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/patrickpromitzer/PycharmProjects/dm-nextcloud/data/db/
  nextcloud:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/patrickpromitzer/PycharmProjects/dm-nextcloud/data/nextcloud/
  nextcloud_config:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /home/patrickpromitzer/PycharmProjects/dm-nextcloud/data/nextcloud_config/

Version
Nextcloud Hub 10 (31.0.2)

[bigcat88]

Hello.

68B file with one " " character at the end:
68b_file_with_one_space_at_the_end.json

So if I will place this file inside Nextcloud and use nc.files.download2stream it will error and download invalid content?

[PatrickPromitzer]

Hello.

68B file with one " " character at the end:
68b_file_with_one_space_at_the_end.json

So if I will place this file inside Nextcloud and use nc.files.download2stream it will error and download invalid content?

the 68b_file_with_one_space_at_the_end.json file is the result after the nc.files.download2stream download.

upload 68b_file.json and add a character anywhere to make the file bigger. The download works before the edit, after adding one more character, it doesn't

[bigcat88]

Oh, it is a gzipped content..

[PatrickPromitzer]

I checked the file.
adding an ".gz" at the end makes it possible to open with the Ubuntu achieve manager.

Looks like you are right.

[bigcat88]

Should be fixed in 0.20.0 version. Thank you.