Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Should KiCost move back scrape module? #313

Closed
romain145 opened this issue Aug 17, 2018 · 15 comments
Closed

Should KiCost move back scrape module? #313

romain145 opened this issue Aug 17, 2018 · 15 comments
Labels
discussion Discution about implementation and new features.

Comments

@romain145
Copy link

KiCost ends the scraping before reaching the end without error.

romain@vps307675:$ kicost -i isolator.xml --debug > kicost.log
Progress: 67%|██████████████████████████▋ | 144/216 [13:01<24:31, 20.44s/part]romain@vps307675:$ tail kicost.log
Part ['U6'] not found at digikey.
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Part ['D1'] not found at digikey.
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Part ['Q1', 'Q2', 'Q4', 'Q5', 'Q6', 'Q7'] not found at digikey.
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Received 403, scraper possibly detected: Starting new session for https://www.digikey.com
Part ['J4'] not found at digikey.
romain@vps307675:$ kicost -v
KiCost 0.1.47
romain@vps307675:$ uname -a
Linux vpsxxx.ovh.net 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u2 (2017-06-26) x86_64 GNU/Linux

kicost.log
isolator.xml.txt
@hildogjr
Copy link
Owner

I could not reproduce, I have the 403 error, but KiCost always recover the connection.
@mmmaisel , do have idea?

@hildogjr
Copy link
Owner

@romain145 could you provide a complete log?

kicost -wi %YOUR_FILE% --include digikey -s --debug 8 > log_file.txt

@romain145
Copy link
Author 

romain@vps307675:~$ kicost -wi isolator.xml --include digikey -s --debug 8 > kicost.log
Started KiCost v.0.1.47 on Linux-3.16.0-4-amd64-x86_64-with-debian-9.5(64bit) Python 2.7.13
Exchange rate: 1 EUR = 1.16 USD
# Getting from XML 'isolator.xml' KiCad BoM...
Getting authorship data...
Getting parts library...
Getting components...
# Removing do not populate parts...
Splitting subparts in the manufacture / distributors codes...
Q1 >> ['manf#']
part/qty>> IPB044N15N5          part>>IPB044N15N5       qty>>
{u'datasheet': 'https://www.wolfspeed.com/media/downloads/145/C3M0065090J.pdf', u'libpart': 'mos_to267-7:IPB044N15N5', u'manf#': 'IPB044N15N5', u'footprint': 'isolator:TO-263-7_Tab_NoPin4', 'value': 'IPB044N15N5'}
LED1 >> ['manf#']
part/qty>> 150060RS55040                part>>150060RS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060RS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'RED'}
Q3 >> ['manf#']
part/qty>> 2N7002               part>>2N7002    qty>>
{u'datasheet': 'https://www.fairchildsemi.com/datasheets/2N/2N7002.pdf', u'libpart': 'Transistor_FET:2N7002', u'manf#': '2N7002', u'footprint': 'Package_TO_SOT_SMD:SOT-23', 'value': '2N7002'}
U7 >> ['manf#']
part/qty>> TMP36GRT             part>>TMP36GRT  qty>>
{u'datasheet': 'SOT-23-5 Analog Devices', u'libpart': 'TMP36:TMP36', u'manf#': 'TMP36GRT', u'footprint': 'Package_TO_SOT_SMD:SOT-23-5', 'value': 'TMP36'}
U6 >> ['manf#']
part/qty>> PC3H4J00001H         part>>PC3H4J00001H      qty>>
{u'footprint': 'isolator:PC3H4', u'libpart': 'pc3h4:PC3H4', 'value': 'PC3H4', u'manf#': 'PC3H4J00001H'}
U1 >> ['manf#']
part/qty>> 171021501            part>>171021501 qty>>
{u'footprint': 'isolator:VDRM_171021501', u'libpart': 'MagICModule:MagIC_VDRM', 'value': 'MagIC_171021501', u'manf#': '171021501'}
U3 >> ['manf#']
part/qty>> VOM1271T             part>>VOM1271T  qty>>
{u'libpart': 'VOM1271T:VOM1271T', u'field8': 'VOM1271T', u'field6': 'Unavailable', u'field7': 'None', u'field4': 'Vishay Semiconductor', u'field5': 'SOP-4 Vishay', u'datasheet': 'Relay SSR 50mA 1.6V DC-IN 4-Pin SOP T/R', 'value': 'VOM1271T', u'manf#': 'VOM1271T', u'footprint': 'Package_SO:SO-4_4.4x4.3mm_P2.54mm'}
U2 >> ['manf#']
part/qty>> CY8C4245AXI-483              part>>CY8C4245AXI-483   qty>>
{u'manf#': 'CY8C4245AXI-483', u'libpart': 'psoc:CY8C4245AXI-483', 'manf': 'Cypress', u'footprint': 'Package_QFP:TQFP-44_10x10mm_P0.8mm', 'value': 'CY8C4245AXI-483'}
C4 >> ['manf#']
part/qty>> CC0603KRX7R8BB105            part>>CC0603KRX7R8BB105 qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': 'CC0603KRX7R8BB105', u'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'value': '1u'}
C11 >> ['manf#']
part/qty>> CC0603KRX7R9BB104            part>>CC0603KRX7R9BB104 qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': 'CC0603KRX7R9BB104', u'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'value': '100n'}
C6 >> ['manf#']
part/qty>> 875105144010         part>>875105144010      qty>>
{u'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.4', u'libpart': 'Device:CP', 'value': '330u 6.3V', u'manf#': '875105144010'}
R10 >> ['manf#']
part/qty>> PCF0603R-24K3BT1             part>>PCF0603R-24K3BT1  qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'PCF0603R-24K3BT1', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '24.3k 0.1%'}
R11 >> ['manf#']
part/qty>> RC0603FR-0710KL              part>>RC0603FR-0710KL   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-0710KL', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '10k'}
D3 >> ['manf#']
part/qty>> BZA856A              part>>BZA856A   qty>>
{u'footprint': 'Package_TO_SOT_SMD:SOT-353_SC-70-5', u'libpart': 'bza856al:BZA856AL', 'value': 'BZA856A', u'manf#': 'BZA856A'}
R34 >> ['manf#']
part/qty>> RC0603FR-072K2L              part>>RC0603FR-072K2L   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-072K2L', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '2.2k'}
R4 >> ['manf#']
part/qty>> ERJ3GEYJ115V         part>>ERJ3GEYJ115V      qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'ERJ3GEYJ115V', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '1.1M 1%'}
R6 >> ['manf#']
part/qty>> CPF0603B174KE1               part>>CPF0603B174KE1    qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'CPF0603B174KE1', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '174k 0.1%'}
R7 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{u'footprint': 'Resistor_SMD:R_0805_2012Metric', u'libpart': 'Device:R', 'value': '390k', u'manf#': 'KTR10EZPF3903'}
R1 >> ['manf#']
part/qty>> RN73C1J52K3BTDF              part>>RN73C1J52K3BTDF   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RN73C1J52K3BTDF', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '52.3k 0.1%'}
R2 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{u'footprint': 'Resistor_SMD:R_0805_2012Metric', u'libpart': 'Device:R', 'value': '390k', u'manf#': 'KTR10EZPF3903'}
LED2 >> ['manf#']
part/qty>> 150060YS55040                part>>150060YS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060YS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'AMBER'}
POT1 >> ['manf#']
part/qty>> TS53YJ103MR10                part>>TS53YJ103MR10     qty>>
{u'datasheet': 'TS53YJ103MR10', u'libpart': 'Device:R_POT', u'manf#': 'TS53YJ103MR10', u'footprint': 'Potentiometer_SMD:Potentiometer_Vishay_TS53YJ_Vertical', 'value': '10k'}
C2 >> ['manf#']
part/qty>> 18121C225KAT2A               part>>18121C225KAT2A    qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': '18121C225KAT2A', u'footprint': 'Capacitor_SMD:C_1812_4532Metric', 'value': '2.2u 100V'}
R25 >> ['manf#']
part/qty>> ERJ-3GEYJ181V                part>>ERJ-3GEYJ181V     qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'ERJ-3GEYJ181V', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '180'}
F1 >> ['manf#']
part/qty>> 2920L050DR           part>>2920L050DR        qty>>
{u'datasheet': '~', u'libpart': 'Device:Fuse', u'manf#': '2920L050DR', u'footprint': 'Fuse:Fuse_2920_7451Metric', 'value': '2920L050'}
J4 >> ['manf#']
part/qty>> M20-9990346          part>>M20-9990346       qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': 'M20-9990346', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'value': 'PWM_OUT'}
J5 >> ['manf#']
part/qty>> 61300821121          part>>61300821121       qty>>
{u'datasheet': 'M20-9980446', u'libpart': 'Connector_Generic:Conn_02x04_Odd_Even', u'manf#': '61300821121', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_2x04_P2.54mm_Vertical', 'value': 'DEBUG'}
J6 >> ['manf#']
part/qty>> 61300411121          part>>61300411121       qty>>
{u'datasheet': 'M20-9990446', u'libpart': 'Connector_Generic:Conn_01x04', u'manf#': '61300411121', u'footprint': 'isolator:OLED_SSD1306_128x32', 'value': 'I2C'}
J1 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x02', u'manf#': 'B2B-XH-A', u'footprint': 'Connector_JST:JST_XH_B02B-XH-A_1x02_P2.50mm_Vertical', 'value': 'POWER_IN'}
J2 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': 'B2B-XH-A', u'footprint': 'Connector_JST:JST_XH_B03B-XH-A_1x03_P2.50mm_Vertical', 'value': 'TRIGGER_IN'}
J3 >> ['manf#']
part/qty>> 61300311121          part>>61300311121       qty>>
{u'datasheet': 'M20-9990346', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': '61300311121', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'value': 'PWM_IN'}
LED3 >> ['manf#']
part/qty>> 150060GS55040                part>>150060GS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060GS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'GREEN'}
SW1 >> ['manf#']
part/qty>> KMR211GLFS           part>>KMR211GLFS        qty>>
{u'footprint': 'Button_Switch_SMD:SW_SPST_KMR2', u'libpart': 'Switch:SW_SPST', 'value': 'SPST', u'manf#': 'KMR211GLFS'}
C1 >> ['manf#']
part/qty>> 875105844002         part>>875105844002      qty>>
{u'manf#': '875105844002', u'libpart': 'Device:CP', 'manf': 'Wurth Electronics', u'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.8', 'value': '8.2u 63V'}
R21 >> ['manf#']
part/qty>> RC0603FR-071KL               part>>RC0603FR-071KL    qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-071KL', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '1k'}
D1 >> ['manf#']
part/qty>> SM6T56A              part>>SM6T56A   qty>>
{u'datasheet': '~', u'libpart': 'Device:D_Schottky', u'manf#': 'SM6T56A', u'footprint': 'Diode_SMD:D_SMB', 'value': 'SM6T56A'}
# Grouping parts...
Getting groups of identical components...
Checking the seemingly identical parts group...
Merging field asked in the identical components groups...
Propagating field values to identical components...
Create HTML page for parts with custom pricing...
Initialising scraper with 1 threads
throttling_delay=5
Progress:   0%|                                                                                                                                                  | 0/36 [39:46<?, ?part/s]Creating the 'isolator.xlsx' spreadsheet...
Writing the global parts informations...
Qty>> ['C1', 'C5']       1*875105844002
Qty>> ['C3', 'C2']       1*18121C225KAT2A
Qty>> ['C4', 'C15', 'C14']       1*CC0603KRX7R8BB105
Qty>> ['C9', 'C19', 'C18', 'C11', 'C17', 'C16']  1*CC0603KRX7R9BB104
Qty>> ['C8', 'C6', 'C7']         1*875105144010
Qty>> ['D1']     1*SM6T56A
Qty>> ['D4', 'D3', 'D2']         1*BZA856A
Qty>> ['F1']     1*2920L050DR
Qty>> ['J1']     1*B2B-XH-A
Qty>> ['J2']     1*B2B-XH-A
Qty>> ['J3']     1*61300311121
Qty>> ['J4']     1*M20-9990346
Qty>> ['J5']     1*61300821121
Qty>> ['J6']     1*61300411121
Qty>> ['LED1']   1*150060RS55040
Qty>> ['LED2']   1*150060YS55040
Qty>> ['LED3']   1*150060GS55040
Qty>> ['MH6', 'MH4', 'MH5', 'MH2', 'MH3']        None*None
Qty>> ['POT2', 'POT1']   1*TS53YJ103MR10
Qty>> ['Q1', 'Q2', 'Q5', 'Q4', 'Q7', 'Q6']       1*IPB044N15N5
Qty>> ['Q3', 'Q9', 'Q8']         1*2N7002
Qty>> ['R1']     1*RN73C1J52K3BTDF
Qty>> ['R10']    1*PCF0603R-24K3BT1
Qty>> ['R7', 'R2']       1*KTR10EZPF3903
Qty>> ['R28', 'R25', 'R31']      1*ERJ-3GEYJ181V
Qty>> ['R36', 'R30', 'R3', 'R33', 'R11', 'R18', 'R19', 'R24', 'R27', 'R20']      1*RC0603FR-0710KL
Qty>> ['R35', 'R34']     1*RC0603FR-072K2L
Qty>> ['R4']     1*ERJ3GEYJ115V
Qty>> ['R6']     1*CPF0603B174KE1
Qty>> ['R32', 'R22', 'R23', 'R29', 'R26', 'R21']         1*RC0603FR-071KL
Qty>> ['SW1', 'SW2']     1*KMR211GLFS
Qty>> ['U1']     1*171021501
Qty>> ['U2']     1*CY8C4245AXI-483
Qty>> ['U6']     1*PC3H4J00001H
Qty>> ['U7']     1*TMP36GRT
Qty>> ['U5', 'U4', 'U3']         1*VOM1271T
Sorting the distributors...
Writing the distributors parts informations...
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/home/romain/.local/lib/python2.7/site-packages/tqdm/_monitor.py", line 72, in run
  File "/home/romain/.local/lib/python2.7/site-packages/tqdm/_tqdm.py", line 106, in __enter__
  File "/home/romain/.local/lib/python2.7/site-packages/tqdm/_tqdm.py", line 99, in acquire
  File "/usr/lib/python2.7/threading.py", line 168, in acquire
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

@romain145
Copy link
Author

Doing the same test on LSS under W10, the scraping finishes and the generated xlsx is valid (with only digi key results of course). Is it an issue with Python 2.7? How can I force KiCost to use Python 3.5 on the VPS?

@hildogjr hildogjr mentioned this issue Aug 23, 2018
Closed
@hildogjr
Copy link
Owner

My system is also Py3. Could you test on it, @romain145?

@romain145
Copy link
Author

Test ran on LSS under Win10 with the default Python (2.7.12):

$ uname -a
Linux VIGLEN 4.4.0-17134-Microsoft #137-Microsoft Thu Jun 14 18:46:00 PST 2018 x86_64 x86_64 x86_64 GNU/Linux
$ python -V
Python 2.7.12
$ kicost -v
KiCost 0.1.47
$ kicost -wi isolator.xml --include digikey -s --debug 8 > log_file.txt
Started KiCost v.0.1.47 on Linux-4.4.0-17134-Microsoft-x86_64-with-Ubuntu-16.04-xenial(64bit) Python 2.7.12
Exchange rate: 1 EUR = 1.16 USD
# Getting from XML 'isolator.xml' KiCad BoM...
Getting authorship data...
Getting parts library...
Getting components...
# Removing do not populate parts...
Splitting subparts in the manufacture / distributors codes...
Q1 >> ['manf#']
part/qty>> IPB044N15N5          part>>IPB044N15N5       qty>>
{u'datasheet': 'https://www.wolfspeed.com/media/downloads/145/C3M0065090J.pdf', u'libpart': 'mos_to267-7:IPB044N15N5', u'manf#': 'IPB044N15N5', u'footprint': 'isolator:TO-263-7_Tab_NoPin4', 'value': 'IPB044N15N5'}
LED1 >> ['manf#']
part/qty>> 150060RS55040                part>>150060RS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060RS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'RED'}
Q3 >> ['manf#']
part/qty>> 2N7002               part>>2N7002    qty>>
{u'datasheet': 'https://www.fairchildsemi.com/datasheets/2N/2N7002.pdf', u'libpart': 'Transistor_FET:2N7002', u'manf#': '2N7002', u'footprint': 'Package_TO_SOT_SMD:SOT-23', 'value': '2N7002'}
U7 >> ['manf#']
part/qty>> TMP36GRT             part>>TMP36GRT  qty>>
{u'datasheet': 'SOT-23-5 Analog Devices', u'libpart': 'TMP36:TMP36', u'manf#': 'TMP36GRT', u'footprint': 'Package_TO_SOT_SMD:SOT-23-5', 'value': 'TMP36'}
U6 >> ['manf#']
part/qty>> PC3H4J00001H         part>>PC3H4J00001H      qty>>
{u'footprint': 'isolator:PC3H4', u'libpart': 'pc3h4:PC3H4', 'value': 'PC3H4', u'manf#': 'PC3H4J00001H'}
U1 >> ['manf#']
part/qty>> 171021501            part>>171021501 qty>>
{u'footprint': 'isolator:VDRM_171021501', u'libpart': 'MagICModule:MagIC_VDRM', 'value': 'MagIC_171021501', u'manf#': '171021501'}
U3 >> ['manf#']
part/qty>> VOM1271T             part>>VOM1271T  qty>>
{u'libpart': 'VOM1271T:VOM1271T', u'field8': 'VOM1271T', u'field6': 'Unavailable', u'field7': 'None', u'field4': 'Vishay Semiconductor', u'field5': 'SOP-4 Vishay', u'datasheet': 'Relay SSR 50mA 1.6V DC-IN 4-Pin SOP T/R', 'value': 'VOM1271T', u'manf#': 'VOM1271T', u'footprint': 'Package_SO:SO-4_4.4x4.3mm_P2.54mm'}
U2 >> ['manf#']
part/qty>> CY8C4245AXI-483              part>>CY8C4245AXI-483   qty>>
{u'manf#': 'CY8C4245AXI-483', u'libpart': 'psoc:CY8C4245AXI-483', 'manf': 'Cypress', u'footprint': 'Package_QFP:TQFP-44_10x10mm_P0.8mm', 'value': 'CY8C4245AXI-483'}
C4 >> ['manf#']
part/qty>> CC0603KRX7R8BB105            part>>CC0603KRX7R8BB105 qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': 'CC0603KRX7R8BB105', u'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'value': '1u'}
C11 >> ['manf#']
part/qty>> CC0603KRX7R9BB104            part>>CC0603KRX7R9BB104 qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': 'CC0603KRX7R9BB104', u'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'value': '100n'}
C6 >> ['manf#']
part/qty>> 875105144010         part>>875105144010      qty>>
{u'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.4', u'libpart': 'Device:CP', 'value': '330u 6.3V', u'manf#': '875105144010'}R10 >> ['manf#']
part/qty>> PCF0603R-24K3BT1             part>>PCF0603R-24K3BT1  qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'PCF0603R-24K3BT1', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '24.3k 0.1%'}
R11 >> ['manf#']
part/qty>> RC0603FR-0710KL              part>>RC0603FR-0710KL   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-0710KL', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '10k'}
D3 >> ['manf#']
part/qty>> BZA856A              part>>BZA856A   qty>>
{u'footprint': 'Package_TO_SOT_SMD:SOT-353_SC-70-5', u'libpart': 'bza856al:BZA856AL', 'value': 'BZA856A', u'manf#': 'BZA856A'}
R34 >> ['manf#']
part/qty>> RC0603FR-072K2L              part>>RC0603FR-072K2L   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-072K2L', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '2.2k'}
R4 >> ['manf#']
part/qty>> ERJ3GEYJ115V         part>>ERJ3GEYJ115V      qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'ERJ3GEYJ115V', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '1.1M 1%'}
R6 >> ['manf#']
part/qty>> CPF0603B174KE1               part>>CPF0603B174KE1    qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'CPF0603B174KE1', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '174k 0.1%'}
R7 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{u'footprint': 'Resistor_SMD:R_0805_2012Metric', u'libpart': 'Device:R', 'value': '390k', u'manf#': 'KTR10EZPF3903'}
R1 >> ['manf#']
part/qty>> RN73C1J52K3BTDF              part>>RN73C1J52K3BTDF   qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RN73C1J52K3BTDF', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '52.3k 0.1%'}
R2 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{u'footprint': 'Resistor_SMD:R_0805_2012Metric', u'libpart': 'Device:R', 'value': '390k', u'manf#': 'KTR10EZPF3903'}
LED2 >> ['manf#']
part/qty>> 150060YS55040                part>>150060YS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060YS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'AMBER'}
POT1 >> ['manf#']
part/qty>> TS53YJ103MR10                part>>TS53YJ103MR10     qty>>
{u'datasheet': 'TS53YJ103MR10', u'libpart': 'Device:R_POT', u'manf#': 'TS53YJ103MR10', u'footprint': 'Potentiometer_SMD:Potentiometer_Vishay_TS53YJ_Vertical', 'value': '10k'}
C2 >> ['manf#']
part/qty>> 18121C225KAT2A               part>>18121C225KAT2A    qty>>
{u'datasheet': '~', u'libpart': 'Device:C', u'manf#': '18121C225KAT2A', u'footprint': 'Capacitor_SMD:C_1812_4532Metric', 'value': '2.2u 100V'}
R25 >> ['manf#']
part/qty>> ERJ-3GEYJ181V                part>>ERJ-3GEYJ181V     qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'ERJ-3GEYJ181V', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '180'}
F1 >> ['manf#']
part/qty>> 2920L050DR           part>>2920L050DR        qty>>
{u'datasheet': '~', u'libpart': 'Device:Fuse', u'manf#': '2920L050DR', u'footprint': 'Fuse:Fuse_2920_7451Metric', 'value': '2920L050'}
J4 >> ['manf#']
part/qty>> M20-9990346          part>>M20-9990346       qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': 'M20-9990346', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'value': 'PWM_OUT'}
J5 >> ['manf#']
part/qty>> 61300821121          part>>61300821121       qty>>
{u'datasheet': 'M20-9980446', u'libpart': 'Connector_Generic:Conn_02x04_Odd_Even', u'manf#': '61300821121', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_2x04_P2.54mm_Vertical', 'value': 'DEBUG'}
J6 >> ['manf#']
part/qty>> 61300411121          part>>61300411121       qty>>
{u'datasheet': 'M20-9990446', u'libpart': 'Connector_Generic:Conn_01x04', u'manf#': '61300411121', u'footprint': 'isolator:OLED_SSD1306_128x32', 'value': 'I2C'}
J1 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x02', u'manf#': 'B2B-XH-A', u'footprint': 'Connector_JST:JST_XH_B02B-XH-A_1x02_P2.50mm_Vertical', 'value': 'POWER_IN'}
J2 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{u'datasheet': '~', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': 'B2B-XH-A', u'footprint': 'Connector_JST:JST_XH_B03B-XH-A_1x03_P2.50mm_Vertical', 'value': 'TRIGGER_IN'}
J3 >> ['manf#']
part/qty>> 61300311121          part>>61300311121       qty>>
{u'datasheet': 'M20-9990346', u'libpart': 'Connector_Generic:Conn_01x03', u'manf#': '61300311121', u'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'value': 'PWM_IN'}
LED3 >> ['manf#']
part/qty>> 150060GS55040                part>>150060GS55040     qty>>
{u'datasheet': '~', u'libpart': 'Device:LED', u'manf#': '150060GS55040', u'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'value': 'GREEN'}
SW1 >> ['manf#']
part/qty>> KMR211GLFS           part>>KMR211GLFS        qty>>
{u'footprint': 'Button_Switch_SMD:SW_SPST_KMR2', u'libpart': 'Switch:SW_SPST', 'value': 'SPST', u'manf#': 'KMR211GLFS'}
C1 >> ['manf#']
part/qty>> 875105844002         part>>875105844002      qty>>
{u'manf#': '875105844002', u'libpart': 'Device:CP', 'manf': 'Wurth Electronics', u'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.8', 'value': '8.2u 63V'}
R21 >> ['manf#']
part/qty>> RC0603FR-071KL               part>>RC0603FR-071KL    qty>>
{u'datasheet': '~', u'libpart': 'Device:R', u'manf#': 'RC0603FR-071KL', u'footprint': 'Resistor_SMD:R_0603_1608Metric', 'value': '1k'}
D1 >> ['manf#']
part/qty>> SM6T56A              part>>SM6T56A   qty>>
{u'datasheet': '~', u'libpart': 'Device:D_Schottky', u'manf#': 'SM6T56A', u'footprint': 'Diode_SMD:D_SMB', 'value': 'SM6T56A'}
# Grouping parts...
Getting groups of identical components...
Checking the seemingly identical parts group...
Merging field asked in the identical components groups...
Propagating field values to identical components...
Create HTML page for parts with custom pricing...
Initialising scraper with 1 threads
throttling_delay=5
Progress: 100%|███████████████████████████████████████████████████████████████████████| 36/36 [11:16<00:00, 16.25s/part]Creating the 'isolator.xlsx' spreadsheet...
Writing the global parts informations...
Qty>> ['C1', 'C5']       1*875105844002
Qty>> ['C3', 'C2']       1*18121C225KAT2A
Qty>> ['C4', 'C15', 'C14']       1*CC0603KRX7R8BB105
Qty>> ['C9', 'C19', 'C18', 'C11', 'C17', 'C16']  1*CC0603KRX7R9BB104
Qty>> ['C8', 'C6', 'C7']         1*875105144010
Qty>> ['D1']     1*SM6T56A
Qty>> ['D4', 'D3', 'D2']         1*BZA856A
Qty>> ['F1']     1*2920L050DR
Qty>> ['J1']     1*B2B-XH-A
Qty>> ['J2']     1*B2B-XH-A
Qty>> ['J3']     1*61300311121
Qty>> ['J4']     1*M20-9990346
Qty>> ['J5']     1*61300821121
Qty>> ['J6']     1*61300411121
Qty>> ['LED1']   1*150060RS55040
Qty>> ['LED2']   1*150060YS55040
Qty>> ['LED3']   1*150060GS55040
Qty>> ['MH6', 'MH4', 'MH5', 'MH2', 'MH3']        None*None
Qty>> ['POT2', 'POT1']   1*TS53YJ103MR10
Qty>> ['Q1', 'Q2', 'Q5', 'Q4', 'Q7', 'Q6']       1*IPB044N15N5
Qty>> ['Q3', 'Q9', 'Q8']         1*2N7002
Qty>> ['R1']     1*RN73C1J52K3BTDF
Qty>> ['R10']    1*PCF0603R-24K3BT1
Qty>> ['R7', 'R2']       1*KTR10EZPF3903
Qty>> ['R28', 'R25', 'R31']      1*ERJ-3GEYJ181V
Qty>> ['R36', 'R30', 'R3', 'R33', 'R11', 'R18', 'R19', 'R24', 'R27', 'R20']      1*RC0603FR-0710KL
Qty>> ['R35', 'R34']     1*RC0603FR-072K2L
Qty>> ['R4']     1*ERJ3GEYJ115V
Qty>> ['R6']     1*CPF0603B174KE1
Qty>> ['R32', 'R22', 'R23', 'R29', 'R26', 'R21']         1*RC0603FR-071KL
Qty>> ['SW1', 'SW2']     1*KMR211GLFS
Qty>> ['U1']     1*171021501
Qty>> ['U2']     1*CY8C4245AXI-483
Qty>> ['U6']     1*PC3H4J00001H
Qty>> ['U7']     1*TMP36GRT
Qty>> ['U5', 'U4', 'U3']         1*VOM1271T
Sorting the distributors...
Writing the distributors parts informations...
# Writing Digi-Key
Exception KeyError: KeyError(<weakref at 0x7f31c2665d60; to 'tqdm' at 0x7f31db3dd950>,) in <bound method tqdm.__del__ of Progress: 100%|███████████████████████████████████████████████████████████████████████| 36/36 [11:17<00:00, 16.25s/part]> ignored

See attached xml BOM file, generated xlsx file and log file.
isolator.xml.txt
isolator.xlsx
log_file.txt

Will now change the python version and rerun the test.

@romain145
Copy link
Author

I followed this to change the python version used:
https://linuxconfig.org/how-to-change-from-default-to-alternative-python-version-on-debian-linux

$ python3 -V
Python 3.5.2
$ kicost -wi isolator.xml --include digikey -s --debug 8 > log_file.txt
Started KiCost v.0.1.47 on Linux-4.4.0-17134-Microsoft-x86_64-with-Ubuntu-16.04-xenial(64bit) Python 3.5.2
Exchange rate: 1 EUR = 1.16 USD
# Getting from XML 'isolator.xml' KiCad BoM...
Getting authorship data...
Getting parts library...
Getting components...
# Removing do not populate parts...
Splitting subparts in the manufacture / distributors codes...
LED1 >> ['manf#']
part/qty>> 150060RS55040                part>>150060RS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060RS55040', 'value': 'RED'}
D1 >> ['manf#']
part/qty>> SM6T56A              part>>SM6T56A   qty>>
{'datasheet': '~', 'footprint': 'Diode_SMD:D_SMB', 'libpart': 'Device:D_Schottky', 'manf#': 'SM6T56A', 'value': 'SM6T56A'}
SW1 >> ['manf#']
part/qty>> KMR211GLFS           part>>KMR211GLFS        qty>>
{'manf#': 'KMR211GLFS', 'footprint': 'Button_Switch_SMD:SW_SPST_KMR2', 'libpart': 'Switch:SW_SPST', 'value': 'SPST'}
J2 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{'datasheet': '~', 'footprint': 'Connector_JST:JST_XH_B03B-XH-A_1x03_P2.50mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': 'B2B-XH-A', 'value': 'TRIGGER_IN'}
J5 >> ['manf#']
part/qty>> 61300821121          part>>61300821121       qty>>
{'datasheet': 'M20-9980446', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_2x04_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_02x04_Odd_Even', 'manf#': '61300821121', 'value': 'DEBUG'}
U3 >> ['manf#']
part/qty>> VOM1271T             part>>VOM1271T  qty>>
{'libpart': 'VOM1271T:VOM1271T', 'field6': 'Unavailable', 'field5': 'SOP-4 Vishay', 'footprint': 'Package_SO:SO-4_4.4x4.3mm_P2.54mm', 'field4': 'Vishay Semiconductor', 'manf#': 'VOM1271T', 'field8': 'VOM1271T', 'datasheet': 'Relay SSR 50mA 1.6V DC-IN 4-Pin SOP T/R', 'field7': 'None', 'value': 'VOM1271T'}
C11 >> ['manf#']
part/qty>> CC0603KRX7R9BB104            part>>CC0603KRX7R9BB104 qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'libpart': 'Device:C', 'manf#': 'CC0603KRX7R9BB104', 'value': '100n'}
R21 >> ['manf#']
part/qty>> RC0603FR-071KL               part>>RC0603FR-071KL    qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-071KL', 'value': '1k'}
Q1 >> ['manf#']
part/qty>> IPB044N15N5          part>>IPB044N15N5       qty>>
{'footprint': 'isolator:TO-263-7_Tab_NoPin4', 'libpart': 'mos_to267-7:IPB044N15N5', 'manf#': 'IPB044N15N5', 'value': 'IPB044N15N5', 'datasheet': 'https://www.wolfspeed.com/media/downloads/145/C3M0065090J.pdf'}
C1 >> ['manf#']
part/qty>> 875105844002         part>>875105844002      qty>>
{'manf': 'Wurth Electronics', 'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.8', 'libpart': 'Device:CP', 'manf#': '875105844002', 'value': '8.2u 63V'}
C2 >> ['manf#']
part/qty>> 18121C225KAT2A               part>>18121C225KAT2A    qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_1812_4532Metric', 'libpart': 'Device:C', 'manf#': '18121C225KAT2A', 'value': '2.2u 100V'}
R34 >> ['manf#']
part/qty>> RC0603FR-072K2L              part>>RC0603FR-072K2L   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-072K2L', 'value': '2.2k'}
R6 >> ['manf#']
part/qty>> CPF0603B174KE1               part>>CPF0603B174KE1    qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'CPF0603B174KE1', 'value': '174k 0.1%'}
POT1 >> ['manf#']
part/qty>> TS53YJ103MR10                part>>TS53YJ103MR10     qty>>
{'datasheet': 'TS53YJ103MR10', 'footprint': 'Potentiometer_SMD:Potentiometer_Vishay_TS53YJ_Vertical', 'libpart': 'Device:R_POT', 'manf#': 'TS53YJ103MR10', 'value': '10k'}
J1 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{'datasheet': '~', 'footprint': 'Connector_JST:JST_XH_B02B-XH-A_1x02_P2.50mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x02', 'manf#': 'B2B-XH-A', 'value': 'POWER_IN'}
R25 >> ['manf#']
part/qty>> ERJ-3GEYJ181V                part>>ERJ-3GEYJ181V     qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'ERJ-3GEYJ181V', 'value': '180'}
R4 >> ['manf#']
part/qty>> ERJ3GEYJ115V         part>>ERJ3GEYJ115V      qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'ERJ3GEYJ115V', 'value': '1.1M 1%'}
D3 >> ['manf#']
part/qty>> BZA856A              part>>BZA856A   qty>>
{'manf#': 'BZA856A', 'footprint': 'Package_TO_SOT_SMD:SOT-353_SC-70-5', 'libpart': 'bza856al:BZA856AL', 'value': 'BZA856A'}
Q3 >> ['manf#']
part/qty>> 2N7002               part>>2N7002    qty>>
{'footprint': 'Package_TO_SOT_SMD:SOT-23', 'libpart': 'Transistor_FET:2N7002', 'manf#': '2N7002', 'value': '2N7002', 'datasheet': 'https://www.fairchildsemi.com/datasheets/2N/2N7002.pdf'}
U7 >> ['manf#']
part/qty>> TMP36GRT             part>>TMP36GRT  qty>>
{'datasheet': 'SOT-23-5 Analog Devices', 'footprint': 'Package_TO_SOT_SMD:SOT-23-5', 'libpart': 'TMP36:TMP36', 'manf#': 'TMP36GRT', 'value': 'TMP36'}
U1 >> ['manf#']
part/qty>> 171021501            part>>171021501 qty>>
{'manf#': '171021501', 'footprint': 'isolator:VDRM_171021501', 'libpart': 'MagICModule:MagIC_VDRM', 'value': 'MagIC_171021501'}
C6 >> ['manf#']
part/qty>> 875105144010         part>>875105144010      qty>>
{'manf#': '875105144010', 'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.4', 'libpart': 'Device:CP', 'value': '330u 6.3V'}
J6 >> ['manf#']
part/qty>> 61300411121          part>>61300411121       qty>>
{'datasheet': 'M20-9990446', 'footprint': 'isolator:OLED_SSD1306_128x32', 'libpart': 'Connector_Generic:Conn_01x04', 'manf#': '61300411121', 'value': 'I2C'}
U2 >> ['manf#']
part/qty>> CY8C4245AXI-483              part>>CY8C4245AXI-483   qty>>
{'manf': 'Cypress', 'libpart': 'psoc:CY8C4245AXI-483', 'manf#': 'CY8C4245AXI-483', 'footprint': 'Package_QFP:TQFP-44_10x10mm_P0.8mm', 'value': 'CY8C4245AXI-483'}
J4 >> ['manf#']
part/qty>> M20-9990346          part>>M20-9990346       qty>>
{'datasheet': '~', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': 'M20-9990346', 'value': 'PWM_OUT'}
LED2 >> ['manf#']
part/qty>> 150060YS55040                part>>150060YS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060YS55040', 'value': 'AMBER'}
R11 >> ['manf#']
part/qty>> RC0603FR-0710KL              part>>RC0603FR-0710KL   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-0710KL', 'value': '10k'}
U6 >> ['manf#']
part/qty>> PC3H4J00001H         part>>PC3H4J00001H      qty>>
{'manf#': 'PC3H4J00001H', 'libpart': 'pc3h4:PC3H4', 'value': 'PC3H4', 'footprint': 'isolator:PC3H4'}
R2 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{'manf#': 'KTR10EZPF3903', 'footprint': 'Resistor_SMD:R_0805_2012Metric', 'libpart': 'Device:R', 'value': '390k'}
F1 >> ['manf#']
part/qty>> 2920L050DR           part>>2920L050DR        qty>>
{'datasheet': '~', 'footprint': 'Fuse:Fuse_2920_7451Metric', 'libpart': 'Device:Fuse', 'manf#': '2920L050DR', 'value': '2920L050'}
C4 >> ['manf#']
part/qty>> CC0603KRX7R8BB105            part>>CC0603KRX7R8BB105 qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'libpart': 'Device:C', 'manf#': 'CC0603KRX7R8BB105', 'value': '1u'}
LED3 >> ['manf#']
part/qty>> 150060GS55040                part>>150060GS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060GS55040', 'value': 'GREEN'}
R1 >> ['manf#']
part/qty>> RN73C1J52K3BTDF              part>>RN73C1J52K3BTDF   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RN73C1J52K3BTDF', 'value': '52.3k 0.1%'}
R7 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{'manf#': 'KTR10EZPF3903', 'footprint': 'Resistor_SMD:R_0805_2012Metric', 'libpart': 'Device:R', 'value': '390k'}
J3 >> ['manf#']
part/qty>> 61300311121          part>>61300311121       qty>>
{'datasheet': 'M20-9990346', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': '61300311121', 'value': 'PWM_IN'}
R10 >> ['manf#']
part/qty>> PCF0603R-24K3BT1             part>>PCF0603R-24K3BT1  qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'PCF0603R-24K3BT1', 'value': '24.3k 0.1%'}
# Grouping parts...
Getting groups of identical components...
Checking the seemingly identical parts group...
Merging field asked in the identical components groups...
Propagating field values to identical components...
Create HTML page for parts with custom pricing...
Initialising scraper with 1 threads
throttling_delay=5
Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [11:20<00:00, 19.55s/part]Creating the 'isolator.xlsx' spreadsheet...
Writing the global parts informations...
Qty>> ['C1', 'C5']       1*875105844002
Qty>> ['C2', 'C3']       1*18121C225KAT2A
Qty>> ['C4', 'C15', 'C14']       1*CC0603KRX7R8BB105
Qty>> ['C18', 'C19', 'C9', 'C16', 'C11', 'C17']  1*CC0603KRX7R9BB104
Qty>> ['C8', 'C7', 'C6']         1*875105144010
Qty>> ['D1']     1*SM6T56A
Qty>> ['D3', 'D2', 'D4']         1*BZA856A
Qty>> ['F1']     1*2920L050DR
Qty>> ['J1']     1*B2B-XH-A
Qty>> ['J2']     1*B2B-XH-A
Qty>> ['J3']     1*61300311121
Qty>> ['J4']     1*M20-9990346
Qty>> ['J5']     1*61300821121
Qty>> ['J6']     1*61300411121
Qty>> ['LED1']   1*150060RS55040
Qty>> ['LED2']   1*150060YS55040
Qty>> ['LED3']   1*150060GS55040
Qty>> ['MH3', 'MH6', 'MH5', 'MH2', 'MH4']        None*None
Qty>> ['POT2', 'POT1']   1*TS53YJ103MR10
Qty>> ['Q6', 'Q5', 'Q2', 'Q1', 'Q4', 'Q7']       1*IPB044N15N5
Qty>> ['Q9', 'Q8', 'Q3']         1*2N7002
Qty>> ['R1']     1*RN73C1J52K3BTDF
Qty>> ['R10']    1*PCF0603R-24K3BT1
Qty>> ['R7', 'R2']       1*KTR10EZPF3903
Qty>> ['R25', 'R31', 'R28']      1*ERJ-3GEYJ181V
Qty>> ['R33', 'R18', 'R24', 'R36', 'R19', 'R30', 'R3', 'R20', 'R11', 'R27']      1*RC0603FR-0710KL
Qty>> ['R34', 'R35']     1*RC0603FR-072K2L
Qty>> ['R4']     1*ERJ3GEYJ115V
Qty>> ['R6']     1*CPF0603B174KE1
Qty>> ['R21', 'R23', 'R32', 'R22', 'R26', 'R29']         1*RC0603FR-071KL
Qty>> ['SW1', 'SW2']     1*KMR211GLFS
Qty>> ['U1']     1*171021501
Qty>> ['U2']     1*CY8C4245AXI-483
Qty>> ['U6']     1*PC3H4J00001H
Qty>> ['U7']     1*TMP36GRT
Qty>> ['U5', 'U4', 'U3']         1*VOM1271T
Sorting the distributors...
Writing the distributors parts informations...
# Writing Digi-Key
Exception ignored in: <bound method tqdm.__del__ of Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [11:20<00:00, 19.55s/part]>
Traceback (most recent call last):
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 885, in __del__
    self.close()
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 1090, in close
    self._decr_instances(self)
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/usr/lib/python3.5/threading.py", line 1051, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

See attached XML, XMSX and log files:
[isolator.xlsx](https://github.com/xesscorp/KiCost/files/2314847/isolator.xlsx)
[log_file.txt](https://github.com/xesscorp/KiCost/files/2314848/log_file.txt)
[isolator.xml.txt](https://github.com/xesscorp/KiCost/files/2314849/isolator.xml.txt)

@romain145
Copy link
Author

romain145 commented Aug 23, 2018 •
edited
Loading

I followed this to change the python version used:
https://linuxconfig.org/how-to-change-from-default-to-alternative-python-version-on-debian-linux

$ python3 -V
Python 3.5.2
$ kicost -wi isolator.xml --include digikey -s --debug 8 > log_file.txt
Started KiCost v.0.1.47 on Linux-4.4.0-17134-Microsoft-x86_64-with-Ubuntu-16.04-xenial(64bit) Python 3.5.2
Exchange rate: 1 EUR = 1.16 USD
# Getting from XML 'isolator.xml' KiCad BoM...
Getting authorship data...
Getting parts library...
Getting components...
# Removing do not populate parts...
Splitting subparts in the manufacture / distributors codes...
LED1 >> ['manf#']
part/qty>> 150060RS55040                part>>150060RS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060RS55040', 'value': 'RED'}
D1 >> ['manf#']
part/qty>> SM6T56A              part>>SM6T56A   qty>>
{'datasheet': '~', 'footprint': 'Diode_SMD:D_SMB', 'libpart': 'Device:D_Schottky', 'manf#': 'SM6T56A', 'value': 'SM6T56A'}
SW1 >> ['manf#']
part/qty>> KMR211GLFS           part>>KMR211GLFS        qty>>
{'manf#': 'KMR211GLFS', 'footprint': 'Button_Switch_SMD:SW_SPST_KMR2', 'libpart': 'Switch:SW_SPST', 'value': 'SPST'}
J2 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{'datasheet': '~', 'footprint': 'Connector_JST:JST_XH_B03B-XH-A_1x03_P2.50mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': 'B2B-XH-A', 'value': 'TRIGGER_IN'}
J5 >> ['manf#']
part/qty>> 61300821121          part>>61300821121       qty>>
{'datasheet': 'M20-9980446', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_2x04_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_02x04_Odd_Even', 'manf#': '61300821121', 'value': 'DEBUG'}
U3 >> ['manf#']
part/qty>> VOM1271T             part>>VOM1271T  qty>>
{'libpart': 'VOM1271T:VOM1271T', 'field6': 'Unavailable', 'field5': 'SOP-4 Vishay', 'footprint': 'Package_SO:SO-4_4.4x4.3mm_P2.54mm', 'field4': 'Vishay Semiconductor', 'manf#': 'VOM1271T', 'field8': 'VOM1271T', 'datasheet': 'Relay SSR 50mA 1.6V DC-IN 4-Pin SOP T/R', 'field7': 'None', 'value': 'VOM1271T'}
C11 >> ['manf#']
part/qty>> CC0603KRX7R9BB104            part>>CC0603KRX7R9BB104 qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'libpart': 'Device:C', 'manf#': 'CC0603KRX7R9BB104', 'value': '100n'}
R21 >> ['manf#']
part/qty>> RC0603FR-071KL               part>>RC0603FR-071KL    qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-071KL', 'value': '1k'}
Q1 >> ['manf#']
part/qty>> IPB044N15N5          part>>IPB044N15N5       qty>>
{'footprint': 'isolator:TO-263-7_Tab_NoPin4', 'libpart': 'mos_to267-7:IPB044N15N5', 'manf#': 'IPB044N15N5', 'value': 'IPB044N15N5', 'datasheet': 'https://www.wolfspeed.com/media/downloads/145/C3M0065090J.pdf'}
C1 >> ['manf#']
part/qty>> 875105844002         part>>875105844002      qty>>
{'manf': 'Wurth Electronics', 'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.8', 'libpart': 'Device:CP', 'manf#': '875105844002', 'value': '8.2u 63V'}
C2 >> ['manf#']
part/qty>> 18121C225KAT2A               part>>18121C225KAT2A    qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_1812_4532Metric', 'libpart': 'Device:C', 'manf#': '18121C225KAT2A', 'value': '2.2u 100V'}
R34 >> ['manf#']
part/qty>> RC0603FR-072K2L              part>>RC0603FR-072K2L   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-072K2L', 'value': '2.2k'}
R6 >> ['manf#']
part/qty>> CPF0603B174KE1               part>>CPF0603B174KE1    qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'CPF0603B174KE1', 'value': '174k 0.1%'}
POT1 >> ['manf#']
part/qty>> TS53YJ103MR10                part>>TS53YJ103MR10     qty>>
{'datasheet': 'TS53YJ103MR10', 'footprint': 'Potentiometer_SMD:Potentiometer_Vishay_TS53YJ_Vertical', 'libpart': 'Device:R_POT', 'manf#': 'TS53YJ103MR10', 'value': '10k'}
J1 >> ['manf#']
part/qty>> B2B-XH-A             part>>B2B-XH-A  qty>>
{'datasheet': '~', 'footprint': 'Connector_JST:JST_XH_B02B-XH-A_1x02_P2.50mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x02', 'manf#': 'B2B-XH-A', 'value': 'POWER_IN'}
R25 >> ['manf#']
part/qty>> ERJ-3GEYJ181V                part>>ERJ-3GEYJ181V     qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'ERJ-3GEYJ181V', 'value': '180'}
R4 >> ['manf#']
part/qty>> ERJ3GEYJ115V         part>>ERJ3GEYJ115V      qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'ERJ3GEYJ115V', 'value': '1.1M 1%'}
D3 >> ['manf#']
part/qty>> BZA856A              part>>BZA856A   qty>>
{'manf#': 'BZA856A', 'footprint': 'Package_TO_SOT_SMD:SOT-353_SC-70-5', 'libpart': 'bza856al:BZA856AL', 'value': 'BZA856A'}
Q3 >> ['manf#']
part/qty>> 2N7002               part>>2N7002    qty>>
{'footprint': 'Package_TO_SOT_SMD:SOT-23', 'libpart': 'Transistor_FET:2N7002', 'manf#': '2N7002', 'value': '2N7002', 'datasheet': 'https://www.fairchildsemi.com/datasheets/2N/2N7002.pdf'}
U7 >> ['manf#']
part/qty>> TMP36GRT             part>>TMP36GRT  qty>>
{'datasheet': 'SOT-23-5 Analog Devices', 'footprint': 'Package_TO_SOT_SMD:SOT-23-5', 'libpart': 'TMP36:TMP36', 'manf#': 'TMP36GRT', 'value': 'TMP36'}
U1 >> ['manf#']
part/qty>> 171021501            part>>171021501 qty>>
{'manf#': '171021501', 'footprint': 'isolator:VDRM_171021501', 'libpart': 'MagICModule:MagIC_VDRM', 'value': 'MagIC_171021501'}
C6 >> ['manf#']
part/qty>> 875105144010         part>>875105144010      qty>>
{'manf#': '875105144010', 'footprint': 'Capacitor_SMD:CP_Elec_6.3x5.4', 'libpart': 'Device:CP', 'value': '330u 6.3V'}
J6 >> ['manf#']
part/qty>> 61300411121          part>>61300411121       qty>>
{'datasheet': 'M20-9990446', 'footprint': 'isolator:OLED_SSD1306_128x32', 'libpart': 'Connector_Generic:Conn_01x04', 'manf#': '61300411121', 'value': 'I2C'}
U2 >> ['manf#']
part/qty>> CY8C4245AXI-483              part>>CY8C4245AXI-483   qty>>
{'manf': 'Cypress', 'libpart': 'psoc:CY8C4245AXI-483', 'manf#': 'CY8C4245AXI-483', 'footprint': 'Package_QFP:TQFP-44_10x10mm_P0.8mm', 'value': 'CY8C4245AXI-483'}
J4 >> ['manf#']
part/qty>> M20-9990346          part>>M20-9990346       qty>>
{'datasheet': '~', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': 'M20-9990346', 'value': 'PWM_OUT'}
LED2 >> ['manf#']
part/qty>> 150060YS55040                part>>150060YS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060YS55040', 'value': 'AMBER'}
R11 >> ['manf#']
part/qty>> RC0603FR-0710KL              part>>RC0603FR-0710KL   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RC0603FR-0710KL', 'value': '10k'}
U6 >> ['manf#']
part/qty>> PC3H4J00001H         part>>PC3H4J00001H      qty>>
{'manf#': 'PC3H4J00001H', 'libpart': 'pc3h4:PC3H4', 'value': 'PC3H4', 'footprint': 'isolator:PC3H4'}
R2 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{'manf#': 'KTR10EZPF3903', 'footprint': 'Resistor_SMD:R_0805_2012Metric', 'libpart': 'Device:R', 'value': '390k'}
F1 >> ['manf#']
part/qty>> 2920L050DR           part>>2920L050DR        qty>>
{'datasheet': '~', 'footprint': 'Fuse:Fuse_2920_7451Metric', 'libpart': 'Device:Fuse', 'manf#': '2920L050DR', 'value': '2920L050'}
C4 >> ['manf#']
part/qty>> CC0603KRX7R8BB105            part>>CC0603KRX7R8BB105 qty>>
{'datasheet': '~', 'footprint': 'Capacitor_SMD:C_0603_1608Metric', 'libpart': 'Device:C', 'manf#': 'CC0603KRX7R8BB105', 'value': '1u'}
LED3 >> ['manf#']
part/qty>> 150060GS55040                part>>150060GS55040     qty>>
{'datasheet': '~', 'footprint': 'LED_SMD:LED_0805_2012Metric_Pad1.15x1.40mm_HandSolder', 'libpart': 'Device:LED', 'manf#': '150060GS55040', 'value': 'GREEN'}
R1 >> ['manf#']
part/qty>> RN73C1J52K3BTDF              part>>RN73C1J52K3BTDF   qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'RN73C1J52K3BTDF', 'value': '52.3k 0.1%'}
R7 >> ['manf#']
part/qty>> KTR10EZPF3903                part>>KTR10EZPF3903     qty>>
{'manf#': 'KTR10EZPF3903', 'footprint': 'Resistor_SMD:R_0805_2012Metric', 'libpart': 'Device:R', 'value': '390k'}
J3 >> ['manf#']
part/qty>> 61300311121          part>>61300311121       qty>>
{'datasheet': 'M20-9990346', 'footprint': 'Connector_PinHeader_2.54mm:PinHeader_1x03_P2.54mm_Vertical', 'libpart': 'Connector_Generic:Conn_01x03', 'manf#': '61300311121', 'value': 'PWM_IN'}
R10 >> ['manf#']
part/qty>> PCF0603R-24K3BT1             part>>PCF0603R-24K3BT1  qty>>
{'datasheet': '~', 'footprint': 'Resistor_SMD:R_0603_1608Metric', 'libpart': 'Device:R', 'manf#': 'PCF0603R-24K3BT1', 'value': '24.3k 0.1%'}
# Grouping parts...
Getting groups of identical components...
Checking the seemingly identical parts group...
Merging field asked in the identical components groups...
Propagating field values to identical components...
Create HTML page for parts with custom pricing...
Initialising scraper with 1 threads
throttling_delay=5
Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [11:20<00:00, 19.55s/part]Creating the 'isolator.xlsx' spreadsheet...
Writing the global parts informations...
Qty>> ['C1', 'C5']       1*875105844002
Qty>> ['C2', 'C3']       1*18121C225KAT2A
Qty>> ['C4', 'C15', 'C14']       1*CC0603KRX7R8BB105
Qty>> ['C18', 'C19', 'C9', 'C16', 'C11', 'C17']  1*CC0603KRX7R9BB104
Qty>> ['C8', 'C7', 'C6']         1*875105144010
Qty>> ['D1']     1*SM6T56A
Qty>> ['D3', 'D2', 'D4']         1*BZA856A
Qty>> ['F1']     1*2920L050DR
Qty>> ['J1']     1*B2B-XH-A
Qty>> ['J2']     1*B2B-XH-A
Qty>> ['J3']     1*61300311121
Qty>> ['J4']     1*M20-9990346
Qty>> ['J5']     1*61300821121
Qty>> ['J6']     1*61300411121
Qty>> ['LED1']   1*150060RS55040
Qty>> ['LED2']   1*150060YS55040
Qty>> ['LED3']   1*150060GS55040
Qty>> ['MH3', 'MH6', 'MH5', 'MH2', 'MH4']        None*None
Qty>> ['POT2', 'POT1']   1*TS53YJ103MR10
Qty>> ['Q6', 'Q5', 'Q2', 'Q1', 'Q4', 'Q7']       1*IPB044N15N5
Qty>> ['Q9', 'Q8', 'Q3']         1*2N7002
Qty>> ['R1']     1*RN73C1J52K3BTDF
Qty>> ['R10']    1*PCF0603R-24K3BT1
Qty>> ['R7', 'R2']       1*KTR10EZPF3903
Qty>> ['R25', 'R31', 'R28']      1*ERJ-3GEYJ181V
Qty>> ['R33', 'R18', 'R24', 'R36', 'R19', 'R30', 'R3', 'R20', 'R11', 'R27']      1*RC0603FR-0710KL
Qty>> ['R34', 'R35']     1*RC0603FR-072K2L
Qty>> ['R4']     1*ERJ3GEYJ115V
Qty>> ['R6']     1*CPF0603B174KE1
Qty>> ['R21', 'R23', 'R32', 'R22', 'R26', 'R29']         1*RC0603FR-071KL
Qty>> ['SW1', 'SW2']     1*KMR211GLFS
Qty>> ['U1']     1*171021501
Qty>> ['U2']     1*CY8C4245AXI-483
Qty>> ['U6']     1*PC3H4J00001H
Qty>> ['U7']     1*TMP36GRT
Qty>> ['U5', 'U4', 'U3']         1*VOM1271T
Sorting the distributors...
Writing the distributors parts informations...
# Writing Digi-Key
Exception ignored in: <bound method tqdm.__del__ of Progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 36/36 [11:20<00:00, 19.55s/part]>
Traceback (most recent call last):
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 885, in __del__
    self.close()
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 1090, in close
    self._decr_instances(self)
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/home/romain/.local/lib/python3.5/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/usr/lib/python3.5/threading.py", line 1051, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

See attached XML, XMSX and log files:
isolator.xlsx
log_file.txt
isolator.xml.txt

@romain145
Copy link
Author

On a different project, the scraping ends with the same error at 83%:

Progress:  83%|█████████████████████████████████████████████████████████▎           | 249/300 [37:23<45:36, 53.66s/part]Exception ignored in: <bound method tqdm.__del__ of Progress:  83%|█████████████████████████████████████████████████████████▌           | 250/300 [37:23<44:42, 53.66s/part]>
Traceback (most recent call last):
  File "/home/romain/.local/lib/python3.6/site-packages/tqdm/_tqdm.py", line 885, in __del__
    self.close()
  File "/home/romain/.local/lib/python3.6/site-packages/tqdm/_tqdm.py", line 1090, in close
    self._decr_instances(self)
  File "/home/romain/.local/lib/python3.6/site-packages/tqdm/_tqdm.py", line 454, in _decr_instances
    cls.monitor.exit()
  File "/home/romain/.local/lib/python3.6/site-packages/tqdm/_monitor.py", line 52, in exit
    self.join()
  File "/usr/lib/python3.6/threading.py", line 1053, in join
    raise RuntimeError("cannot join current thread")
RuntimeError: cannot join current thread

I'm retrying with the -s flag.

@romain145 romain145 mentioned this issue Sep 4, 2018
Closed
@hildogjr hildogjr added the wont fix Features that will be not implemented or that diverges from KiCost context. label Sep 15, 2018
@hildogjr
Copy link
Owner

hildogjr commented Sep 15, 2018 •
edited
Loading

@romain145 the KiCost scrape module are now deprecated #314 . In few days we will release the API concept.
Give a try downloading the code on the branch "octopart"
https://github.com/xesscorp/KiCost/tree/octopart

Check the result by this new code:
isolator.xlsx

@hildogjr hildogjr added already-fixed Reported issues already fixed on last code version of Git, missing offical release. wont fix Features that will be not implemented or that diverges from KiCost context. and removed wont fix Features that will be not implemented or that diverges from KiCost context. labels Sep 15, 2018
@mobilinkd
Copy link

Please consider re-opening this issue as the octopart experiment appears to be a complete bust. I have downgraded to 0.1.47 to do anything useful with KiCost.

@hildogjr
Copy link
Owner

@mobilinkd, @easyw , to reopen this discussion we will need someone with Python + web experience + java script working on.
Since the constant changes in the web distributors and the java script blocking scraps (your algorithm shall be classified as robot), keep it working stated to be a cumcubersome to the only two active developers.

I stated a campaign https://forum.kicad.info/t/kicost-call-of-developers/13271 to call some users to help us. But was unsuccessful.
Please, let us now if someone can provide us some coding help.

@easyw
Copy link

easyw commented Feb 19, 2019 •
edited
Loading

@hildogjr
I think we would need:

  1. a GH repo in which KiCost is downgraded to version 0.1.47 to let users to contribute easily to scrapers
    or an option to use scrapers to the actual release

Then it would be much easier to find if some user may will in contributing to scrapers as they did in the past...
As at the actual KiCost repo state, it is really difficult to contribute to scrapers...

At the moment we could even ask to @volunteerlabs if he could have a look at the digikey scraper, as he already implemented a working version for his tool...

@hildogjr
Copy link
Owner

hildogjr commented Feb 19, 2019 •
edited
Loading

May not be working (due web site changes) but the scrape modules are available in https://github.com/xesscorp/KiCost/tree/scrape_modules
I am re-open this as discussion for implementation.

@hildogjr hildogjr added discussion Discution about implementation and new features. and removed already-fixed Reported issues already fixed on last code version of Git, missing offical release. wont fix Features that will be not implemented or that diverges from KiCost context. labels Feb 19, 2019
@hildogjr hildogjr changed the title Scraping fails without error Should KiCost move back scrape module? Feb 19, 2019
@hildogjr hildogjr reopened this Feb 19, 2019
@hildogjr
Copy link
Owner

Discussion continues at #346

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discussion Discution about implementation and new features.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hildogjr @easyw @romain145 @mobilinkd