Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple errors during PartKeepr database import due to case sensitive database table names #286

Closed
mindsolve opened this issue May 1, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@mindsolve
Copy link

Describe the bug
When trying to import a PartKeepr database XML on a clean installation (after following the workaround from #285), the command to import the database fails with the error message Could not find schema version in XML dump!. This is (directly) caused by the mismatch between the referenced table name schemaversions and the actual table name SchemaVersions in PKImportHelper.php#L58-L62.

But even after changing that, another error occurs with basically the same problem, this time in PKDatastructureImporter.php#L67-L68, here caused by the mismatch between distributor in the code and Distributor in the export.

After that, I've given up manually patching stuff - as this probably points to a different problem/solution:

  • MySQL on Windows is not case sensitive, but is on Linux and Mac. This also affects the exports with mysqldump.
  • Maybe some kind of case conversation is missing in the XML importer/converter?

I've attached my XML export (step 1 below). It was generated by mysqldump Ver 10.16 Distrib 10.1.48-MariaDB running on Debian Stretch (9.13).

To Reproduce
Steps to reproduce the behavior:

  1. Export the database from PartKeepr MySQL - pk-censored.zip
  2. Install Part-DB according to install instructions
  3. Apply workaround from ClassNotFoundError for PurgerInterface when importing data from PartKeepr #285 (run sudo -u www-data composer install -o)
  4. Follow the "How to migrate" steps exactly on a fresh Part-DB installation.
  5. Observe the error message on step 3.

Expected behavior
The parkeepr database to be imported without errors, as described in the documentation.

Screenshots
No screenshots, but here is the full command output (before the first of the above mentioned patches):

root@ng-partdb:/var/www/partdb# sudo -u www-data php bin/console -vvv partdb:migrations:import-partkeepr /var/www/pk.xml

 ! [NOTE] This command is still in development. If you encounter any problems, please report them to the issue tracker
 !        on GitHub.


 [WARNING] This command will delete all existing data in the database (except users). Make sure that you have no
           important data in the database before you continue!


 Please type "DELETE ALL DATA" to continue. []:
 > DELETE ALL DATA

23:45:54 CRITICAL  [console] Error thrown while running command "-vvv partdb:migrations:import-partkeepr '/var/www/pk.xml'". Message: "Could not find schema version in XML dump!"
[
  "exception" => RuntimeException {
    #message: "Could not find schema version in XML dump!"
    #code: 0
    #file: "/var/www/partdb/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php"
    #line: 59
    trace: {
      /var/www/partdb/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:59 {
        App\Services\ImportExportSystem\PartKeeprImporter\PKImportHelper->getDatabaseSchemaVersion(array $data): string^
        › if (!isset($data['schemaversions'])) {
        ›     throw new \RuntimeException('Could not find schema version in XML dump!');
        › }
      }
      /var/www/partdb/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:72 { …}
      /var/www/partdb/src/Command/Migrations/ImportPartKeeprCommand.php:100 { …}
      /var/www/partdb/vendor/symfony/console/Command/Command.php:298 { …}
      /var/www/partdb/vendor/symfony/console/Application.php:1058 { …}
      /var/www/partdb/vendor/symfony/framework-bundle/Console/Application.php:96 { …}
      /var/www/partdb/vendor/symfony/console/Application.php:301 { …}
      /var/www/partdb/vendor/symfony/framework-bundle/Console/Application.php:82 { …}
      /var/www/partdb/vendor/symfony/console/Application.php:171 { …}
      /var/www/partdb/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:54 { …}
      /var/www/partdb/vendor/autoload_runtime.php:35 { …}
      /var/www/partdb/bin/console:11 { …}
    }
  },
  "command" => "-vvv partdb:migrations:import-partkeepr '/var/www/pk.xml'",
  "message" => "Could not find schema version in XML dump!"
]
[
  "token" => null,
  "command" => [
    "name" => "partdb:migrations:import-partkeepr",
    "arguments" => [
      "command" => "partdb:migrations:import-partkeepr",
      "file" => "/var/www/pk.xml"
    ]
  ]
]
23:45:54 DEBUG     [console] Command "-vvv partdb:migrations:import-partkeepr '/var/www/pk.xml'" exited with code "1"
[
  "command" => "-vvv partdb:migrations:import-partkeepr '/var/www/pk.xml'",
  "code" => 1
]
[
  "token" => null,
  "command" => [
    "name" => "partdb:migrations:import-partkeepr",
    "arguments" => [
      "command" => "partdb:migrations:import-partkeepr",
      "file" => "/var/www/pk.xml"
    ]
  ]
]

In PKImportHelper.php line 59:

  [RuntimeException]
  Could not find schema version in XML dump!


Exception trace:
  at /var/www/partdb/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:59
 App\Services\ImportExportSystem\PartKeeprImporter\PKImportHelper->getDatabaseSchemaVersion() at /var/www/partdb/src/Services/ImportExportSystem/PartKeeprImporter/PKImportHelper.php:72
 App\Services\ImportExportSystem\PartKeeprImporter\PKImportHelper->checkVersion() at /var/www/partdb/src/Command/Migrations/ImportPartKeeprCommand.php:100
 App\Command\Migrations\ImportPartKeeprCommand->execute() at /var/www/partdb/vendor/symfony/console/Command/Command.php:298
 Symfony\Component\Console\Command\Command->run() at /var/www/partdb/vendor/symfony/console/Application.php:1058
 Symfony\Component\Console\Application->doRunCommand() at /var/www/partdb/vendor/symfony/framework-bundle/Console/Application.php:96
 Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /var/www/partdb/vendor/symfony/console/Application.php:301
 Symfony\Component\Console\Application->doRun() at /var/www/partdb/vendor/symfony/framework-bundle/Console/Application.php:82
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /var/www/partdb/vendor/symfony/console/Application.php:171
 Symfony\Component\Console\Application->run() at /var/www/partdb/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php:54
 Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run() at /var/www/partdb/vendor/autoload_runtime.php:35
 require_once() at /var/www/partdb/bin/console:11

partdb:migrations:import-partkeepr [--no-projects] [--import-users] [--] <file>

Server Side

  • Part-DB Version: Part-DB master @ v1.3.2 (021e28aca877b27c4efc8f495f35575fe5df1176)
  • PHP Version: PHP 8.1.2-1ubuntu2.11
  • Database Server: none, sqlite
  • Server OS: Ubuntu 22.04.2 LTS

Desktop (please complete the following information):
none, via console on server

Smartphone (please complete the following information):
none, via console on server

Additional context
none

@mindsolve mindsolve added the bug Something isn't working label May 1, 2023
jbtronics added a commit that referenced this issue May 1, 2023
On Linux mysql table names can contain uppercase characters, and we expect always lowercase character, so we now normalize the tablenames to lowercase.

Also fixed some type errors on part parameters and improved performace for orderdetails import.

This fixes issue #286
@jbtronics
Copy link
Member

jbtronics commented May 1, 2023

The table names now get normalized (converted to lowercase) before import. I fixed some other stuff and with the latest master commit import should work fine. The import of the parts take a while, so you maybe have to wait a bit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants