Add more debug information to dbt seed

[pedromachados]

I am getting an error while and it would be nice to know which line/column is causing it. All I get now is this:

Creating table dbt_pedro.abc_revenue_data
Inserting 36783 records into table dbt_pedro.abc_revenue_data
Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/envs/abc35/bin/dbt", line 11, in <module>
    sys.exit(main())
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/main.py", line 29, in main
    handle(args)
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/main.py", line 47, in handle
    res = run_from_args(parsed)
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/main.py", line 101, in run_from_args
    result = task.run()
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/task/seed.py", line 12, in run
    seeder.seed(self.args.drop_existing)
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/seeder.py", line 134, in seed
    self.do_seed(schema, cursor, drop_existing)
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/seeder.py", line 104, in do_seed
    cursor, schema, table_name, virtual_table
  File "/home/ec2-user/miniconda3/envs/abc35/lib/python3.5/site-packages/dbt/seeder.py", line 68, in insert_into_table
    cursor.execute(insert_sql)
psycopg2.DataError: value too long for type character varying(65)

[drewbanin]

@pedromachados this is a little tough, as Redshift doesn't tell us which record in the CSV causes this error. I think that in this particular case, you can run dbt seed --drop-existing to destroy and recreate the table. That should help mitigate this particular problem.

Overall though, agreed: dbt's seed functionality needs some more polish

[pedromachados]

There was an issue with escaping single quotes or some sort of text encoding issue. I ended up removing the text columns so I could get the import done. Not sure if the csv import can be made more robust. Another thing that would be good is the ability to seed specific fIles and to define a different target schema if desired. This could be controlled via configuration in the dbt project file.

…

On May 27, 2017 5:29 PM, "Drew Banin" ***@***.***> wrote: @pedromachados <https://github.com/pedromachados> this is a little tough, as Redshift doesn't tell us which record in the CSV causes this error. I *think* that in this particular case, you can run dbt seed --drop-existing to destroy and recreate the table. That should help mitigate this particular problem. Overall though, agreed: dbt's seed functionality needs some more polish — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#448 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGjyPmW7re9nLxFJEPOdNBccwXhDyUqeks5r-KPhgaJpZM4NoHzp> .

[jthandy]

Hey @pedromachados -- this is a totally reasonable request, especially since with our most recent release users can now specify schemas for all models. I made an issue for this here:

#561

Closing this.