Data seeding shell

[TamiasSibiricus]

Just another idea.
How about to implement data seeding shell for this plugin.
Data can be stored in /config/Data in app or plugin folder.
I think all data must be in actual state for last migration.
Data seeding process cna't start if migration is not complete.
What do you think?

[markstory]

Couldn't you make the data seeding a migration?

[TamiasSibiricus]

Yes.. I know about this feature. But i have found this process too repetitive.
Таке a look at this migration https://github.com/iteam-pro/passengers/blob/master/config/Migrations/20141015144419_initial_migration.php
Data seeding pieces of code for Roles table and Users table very similar. except password of course.
But i sure this logic

        $things = TableRegistry::get('PluginName.Things');
        foreach ($this->things as $thing){
            $thing = $things->newEntity($thing);
            $things->save($thing);
        }

acceptable for 99% data seeding processes.
Simple piece of code with some modifications can pickup data from file and insert it into database.
Developer only need to write data files or dump it by shell from own db(for example)
Also we can put callback function in data files to possibility transform it on the fly. Change password for example.

[lorenzo]

How could it be made less repetitive?

[TamiasSibiricus]

@lorenzo for example when my migration contain more than 5 tables.

[lorenzo]

I understand that it can be repetitive, but I'm just asking what would be a good way for you to lessen the repetitive stuff

[TamiasSibiricus]

I can write shell and sample plugin to demonstrate what i mean. And after this you with your teammates will take decision accept this idea as pull request or not. ok?

[rchavik]

Croogo already has something like this: https://github.com/croogo/croogo/blob/master/Extensions/Lib/Utility/DataMigration.php

Bulk load datafiles from a directory: https://github.com/croogo/croogo/blob/master/Install/Model/Install.php#L69-L82
Load selective records from a file: https://github.com/croogo/croogo/blob/master/Settings/Config/Migration/1392779470_setting_updates.php#L40-L50

[TamiasSibiricus]

@rchavik So why shouldn't to make this logic as default for migrtaions plugin?

[rchavik]

@TamiasSibiricus Sure, it's up to the migrations plugin maintainer. I'm just pointing those files out should you want to reuse the code as a base for the PR.

[lorenzo]

@TamiasSibiricus sure, I'd like to see your idea

[ravage84]

Why not? I'd like to see a PR, too.

[josegonzalez]

Closing as there is a PR open.

[TamiasSibiricus]

Ok. So my vision for seeding shell:

• data stores as [ yaml | json | toml | csv ] files. for first time we could realize csv or json. JSON can store data for multiple tables in one file. With csv this is little bit a problem.
• if possible data could be translatable. I know this is not first priority, but...
• make choice between seeding strategies: 1."data as part of migration process" vs 2."data as postmigration process"

[burzum]

Honestly I think this should be a plugin on it's own that could extend or make use of migrations. Either migrations gets events fired on before / after / up / down or extend the migrations plugin in another plugin to get this done.

[TamiasSibiricus]

@burzum I think about seeding as separate plugin. But here is ine problem. Callbacks runs only after whole migration process. So there is impossile relate seeding process to selected migration. Phing currently does not return any info about applied migrations. Also if we realize seeding process as callback it would run every migration and try overwrite existing data after each migration process(i think). Currently there is no mechanism to prevent run seeding twice or more.

[burzum]

@TamiasSibiricus Like I said before, this is easy to resolve with events:

new Event('Migration.beforeMigration', $this, ['direction' => 'up']);
new Event('Migration.beforeMigration', $this, ['direction' => 'down']);
new Event('Migration.afterMigration', $this, ['direction' => 'up']);
new Event('Migration.afterMigration', $this, ['direction' => 'down']);

Listen to the event(s) and put your logic in the event listener that deals with it. Identify the migration by class name and you get the direction from the event data and act accordingly. I think it should be possible to know what was done (or is going to be done) as well and to pass it to the event.

@lorenzo can I add them? 😄

[lorenzo]

@burzum someone did that already

[TamiasSibiricus]

@burzum Events for migration actions migrate and rollback already exists but without subject and options arguments. Just take a look at code for this actions. I think in current version options does not needed.
Also i was try to debug $input and $output vars. They does not contain any helpful info about applied migrations. So i take decision do not transfer them to event callback as options.

[sdustinh]

Ref: http://docs.phinx.org/en/latest/seeding.html

[beporter]

Everyone's situation is going to be different obviously, but my team and I ended up writing a Cake plugin to do data seeding. It still isn't tied to specific migrations, but there's no reason it couldn't be (just have your migration's after() (or equivalent event) invoke the appropriate Shell command and seed file name).

Anyway, here's another alternative in case it's useful to anyone that finds this thread:

https://github.com/loadsys/CakePHP-Basic-Seed

[kingsolmn]

Would love to see this implemented, would be a whole lot easier!

Ref: @sdustinh http://docs.phinx.org/en/latest/seeding.html

[josegonzalez]

@beporter I'd love to somehow have the following feature implemented for the CakePHP ORM: data-seeding from a production database. Maybe something you guys at Loadsys can investigate ;)

[justinatack]

You can still use the built in Phinx data seeding classes in your change() or up() methods. It would be nice though if we could bake separate Seeding files.

$data = [
    [
        'name' => 'Return of the Jedi',
        'series' => 'VI'
    ],
    [
        'name' => 'The Force Awakens',
        'series' => 'VII'
    ]
];

$this->table('movies')
       ->insert($data)
       ->save();

[josegonzalez]

What would be the initial seed data? Just empty?

[sdustinh]

I've been using the TableRegistry in the up() method with a migration file prefixed like 'SeedRoles'. Honestly the only complaint I have with this method is that it marks the migration as migrated. If I could toggle that behavior I'd be set.

[HavokInspiration]

With the support of seeders added to phinx and the migrations plugin enhancements following this, I think this can be closed.