avoid using pkg_resources for faster command execution #426

ishidawataru · 2019-01-05T02:32:05Z

- What I did

speed up command execution by avoiding 'pkg_resources' import.

related issue : #408

- How I did it

Used https://github.com/ninjaaron/fast-entry_points

- How to verify it

- Previous command output (if the output of a command-line utility has changed)

admin@sonic:~$ time show
Usage: show [OPTIONS] COMMAND [ARGS]...

  SONiC command line - 'show' command

Options:
  -?, -h, --help  Show this message and exit.

Commands:
  aaa                   Show AAA configuration
  acl                   Show ACL related information
  arp                   Show IP ARP table
  clock                 Show date and time
  ecn                   Show ECN configuration
  environment           Show environmentals (voltages, fans, temps)                                                                                                                                                                                                                                                                                                         interfaces            Show details of the network interfaces
  ip                    Show IP (IPv4) commands
  ipv6                  Show IPv6 commands
  line                  Show all /dev/ttyUSB lines and their info
  lldp                  LLDP (Link Layer Discovery Protocol)...
  logging               Show system log
  mac                   Show MAC (FDB) entries
  mirror_session        Show existing everflow sessions
  mmu                   Show mmu configuration
  ndp                   Show IPv6 Neighbour table
  ntp                   Show NTP information
  pfc                   Show details of the priority-flow-control...
  platform              Show platform-specific hardware info                                                                                                                                                                                                                                                                                                                priority-group        Show details of the PGs
  processes             Display process information
  queue                 Show details of the queues
  reboot-cause          Show cause of most recent reboot
  runningconfiguration  Show current running configuration...
  services              Show all daemon services
  startupconfiguration  Show startup configuration information
  system-memory         Show memory information
  tacacs                Show TACACS+ configuration
  techsupport           Gather information for troubleshooting
  uptime                Show system uptime
  users                 Show users
  version               Show version information
  vlan                  Show VLAN information
  warm_restart          Show warm restart configuration and state
  watermark             Show details of watermark

real    0m1.376s
user    0m1.188s
sys     0m0.164s

- New command output (if the output of a command-line utility has changed)

admin@sonic:~$ time show
Usage: show [OPTIONS] COMMAND [ARGS]...

  SONiC command line - 'show' command

Options:
  -?, -h, --help  Show this message and exit.

Commands:
  aaa                   Show AAA configuration
  acl                   Show ACL related information
  arp                   Show IP ARP table
  clock                 Show date and time
  ecn                   Show ECN configuration
  environment           Show environmentals (voltages, fans, temps)
  interfaces            Show details of the network interfaces
  ip                    Show IP (IPv4) commands
  ipv6                  Show IPv6 commands                                                                                                                                                                                                                                                                                                                                  line                  Show all /dev/ttyUSB lines and their info
  lldp                  LLDP (Link Layer Discovery Protocol)...
  logging               Show system log
  mac                   Show MAC (FDB) entries
  mirror_session        Show existing everflow sessions
  mmu                   Show mmu configuration
  ndp                   Show IPv6 Neighbour table
  ntp                   Show NTP information
  pfc                   Show details of the priority-flow-control...
  platform              Show platform-specific hardware info
  priority-group        Show details of the PGs
  processes             Display process information
  queue                 Show details of the queues                                                                                                                                                                                                                                                                                                                          reboot-cause          Show cause of most recent reboot
  runningconfiguration  Show current running configuration...
  services              Show all daemon services
  startupconfiguration  Show startup configuration information
  system-memory         Show memory information
  tacacs                Show TACACS+ configuration
  techsupport           Gather information for troubleshooting
  uptime                Show system uptime
  users                 Show users
  version               Show version information
  vlan                  Show VLAN information
  warm_restart          Show warm restart configuration and state
  watermark             Show details of watermark

real    0m0.760s
user    0m0.608s
sys     0m0.140s

Signed-off-by: Wataru Ishida <ishida@nel-america.com>

jleveque · 2019-01-07T19:11:39Z

@ishidawataru: Thanks for looking into this issue. Does this change also speed up tab-completion?

ishidawataru · 2019-01-07T19:33:48Z

@jleveque Yes it should. Every tab-completion executes the command to complete internally.

This reverts commit da3934e.

jipanyang · 2019-01-21T20:02:42Z

This commit is causing deb build failure:

sonic-buildimage$ make target/debs/python-sonic-utilities_1.2-1_all.deb
+++ --- Making target/debs/python-sonic-utilities_1.2-1_all.deb --- +++
BLDENV=stretch make -f Makefile.work stretch
make[1]: Entering directory '/home/jipan/warm_reboot/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "vs"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "8"
"SONIC_CONFIG_MAKE_JOBS"          : "8"
"USERNAME"                        : "admin"
"PASSWORD"                        : "admin"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : "y"
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190121.194906"

make: Nothing to be done for 'stretch'.
make[1]: Leaving directory '/home/jipan/warm_reboot/sonic-buildimage'
make -f Makefile.work target/debs/python-sonic-utilities_1.2-1_all.deb
make[1]: Entering directory '/home/jipan/warm_reboot/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "vs"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "8"
"SONIC_CONFIG_MAKE_JOBS"          : "8"
"USERNAME"                        : "admin"
"PASSWORD"                        : "admin"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : "y"
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190121.194906"

[ FAIL LOG START ] [ target/debs/python-sonic-utilities_1.2-1_all.deb ]
/sonic/src/sonic-utilities /sonic
error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-156.write-test'

The installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    /usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory?  If the
installation directory is a system-owned directory, you may need to sign in
as the administrator or "root" account.  If you do not have administrative
access to this machine, you may wish to choose a different installation
directory, preferably one that is listed in your PYTHONPATH environment
variable.

For information on other options, you may wish to consult the
documentation at:

  https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

[  FAIL LOG END  ] [ target/debs/python-sonic-utilities_1.2-1_all.deb ]
slave.mk:315: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 1
Makefile.work:127: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make[1]: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 2
make[1]: Leaving directory '/home/jipan/warm_reboot/sonic-buildimage'
Makefile:6: recipe for target 'target/debs/python-sonic-utilities_1.2-1_all.deb' failed
make: *** [target/debs/python-sonic-utilities_1.2-1_all.deb] Error 2

But if I go to previous commit, no problem seen. Does the commit require new build from scratch including slave build docker?

    
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ git checkout da3934ec0b06d5a7f49785a62f13a798695f6509
Previous HEAD position was f7d46ed Revert "[reboot] stop docker service before rebooting (#423)" (#434)
HEAD is now at da3934e avoid using pkg_resources for faster command execution (#426)
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ 
jipan@jipan_sonic_vm_150:~/warm_reboot/sonic-buildimage/src/sonic-utilities$ git checkout 220e269e8334a89db5a2f931dea90e238feae7cc
Previous HEAD position was da3934e avoid using pkg_resources for faster command execution (#426)
HEAD is now at 220e269 [reboot] stop docker service before rebooting (#423)

jleveque · 2019-01-31T01:04:19Z

@ishidawataru: This change doesn't appear to speed up bash completion at all. Typing show and pressing tab twice hangs for 2.651 seconds before displaying results. Have you noticed any improvement with bash completion, or with anything else, for that matter?

I am currently building from the head of the master branch of sonic-buildimage (currently commit sonic-net/sonic-buildimage@2b01beb), and I am still seeing from pkg_resources import load_entry_point being called.

avoid using pkg_resources for faster command execution

e6c2c7a

Signed-off-by: Wataru Ishida <ishida@nel-america.com>

jleveque approved these changes Jan 8, 2019

View reviewed changes

jleveque merged commit da3934e into sonic-net:master Jan 11, 2019

ishidawataru deleted the fastentry branch January 11, 2019 02:10

stcheng added the Enhancement label Jan 11, 2019

yxieca added a commit that referenced this pull request Jan 16, 2019

Revert "avoid using pkg_resources for faster command execution (#426)"

0b98b54

This reverts commit da3934e.

jipanyang mentioned this pull request Jan 21, 2019

Build failure with https://github.com/Azure/sonic-utilities/commit/da3934ec0b06d5a7f49785a62f13a798695f6509 #438

Closed

ishidawataru mentioned this pull request Jan 23, 2019

[build]: add fastentrypoints which is used for sonic-utilities build sonic-net/sonic-buildimage#2479

Merged

ishidawataru mentioned this pull request Feb 8, 2019

Bash completion for show/config commands has gotten slow #408

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

avoid using pkg_resources for faster command execution #426

avoid using pkg_resources for faster command execution #426

ishidawataru commented Jan 5, 2019

jleveque commented Jan 7, 2019

ishidawataru commented Jan 7, 2019

jipanyang commented Jan 21, 2019

jleveque commented Jan 31, 2019 •

edited

Loading

avoid using pkg_resources for faster command execution #426

avoid using pkg_resources for faster command execution #426

Conversation

ishidawataru commented Jan 5, 2019

jleveque commented Jan 7, 2019

ishidawataru commented Jan 7, 2019

jipanyang commented Jan 21, 2019

jleveque commented Jan 31, 2019 • edited Loading

jleveque commented Jan 31, 2019 •

edited

Loading