Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add crystal tool dependencies #13613

Conversation

straight-shoota
Copy link
Member

This tool prints all source files read by the compiler in their require order.

It's a basic implementation with two format options, tree and flat. They are identical except that flat has no indent.
Files are printed exactly in the order they are required. Duplicates (i.e. when a file had already been required before) are not shown.

Example output for an empty file
src/prelude.cr
  src/crystal/once.cr
  src/lib_c.cr
  src/macros.cr
  src/object.cr
  src/comparable.cr
  src/exception.cr
    src/exception/call_stack.cr
      src/exception/call_stack/libunwind.cr
        src/lib_c/x86_64-linux-gnu/c/dlfcn.cr
        src/lib_c/x86_64-linux-gnu/c/stdio.cr
          src/lib_c/x86_64-linux-gnu/c/sys/types.cr
            src/lib_c/x86_64-linux-gnu/c/stddef.cr
            src/lib_c/x86_64-linux-gnu/c/stdint.cr
        src/lib_c/x86_64-linux-gnu/c/string.cr
        src/exception/lib_unwind.cr
        src/exception/call_stack/dwarf.cr
          src/crystal/dwarf.cr
            src/crystal/dwarf/abbrev.cr
            src/crystal/dwarf/info.cr
            src/crystal/dwarf/line_numbers.cr
            src/crystal/dwarf/strings.cr
          src/exception/call_stack/elf.cr
            src/crystal/elf.cr
            src/lib_c/x86_64-linux-gnu/c/link.cr
              src/lib_c/x86_64-linux-gnu/c/elf.cr
    src/system_error.cr
  src/iterable.cr
  src/iterator.cr
    src/enumerable.cr
  src/steppable.cr
  src/indexable.cr
    src/indexable/mutable.cr
  src/string.cr
    src/lib_c/x86_64-linux-gnu/c/stdlib.cr
      src/lib_c/x86_64-linux-gnu/c/sys/wait.cr
        src/lib_c/x86_64-linux-gnu/c/signal.cr
          src/lib_c/x86_64-linux-gnu/c/time.cr
    src/crystal/small_deque.cr
    src/crystal/iconv.cr
      src/lib_c/x86_64-linux-gnu/c/iconv.cr
    src/string/builder.cr
      src/io.cr
        src/lib_c/x86_64-linux-gnu/c/errno.cr
        src/lib_c/x86_64-linux-gnu/c/unistd.cr
        src/io/argf.cr
        src/io/buffered.cr
        src/io/byte_format.cr
        src/io/console.cr
        src/io/delimited.cr
        src/io/encoding.cr
        src/io/encoding_stubs.cr
        src/io/error.cr
        src/io/evented.cr
          src/crystal/thread_local_value.cr
        src/io/file_descriptor.cr
          src/crystal/system/file_descriptor.cr
            src/crystal/system/unix/file_descriptor.cr
              src/lib_c/x86_64-linux-gnu/c/fcntl.cr
                src/lib_c/x86_64-linux-gnu/c/sys/stat.cr
              src/termios.cr
                src/lib_c/x86_64-linux-gnu/c/termios.cr
        src/io/hexdump.cr
        src/io/memory.cr
        src/io/multi_writer.cr
        src/io/overlapped.cr
        src/io/sized.cr
        src/io/stapled.cr
    src/string/formatter.cr
    src/string/grapheme.cr
      src/string/grapheme/grapheme.cr
        src/string/grapheme/properties.cr
    src/string/utf16.cr
  src/number.cr
  src/primitives.cr
  src/annotations.cr
  src/array.cr
  src/atomic.cr
    src/llvm/enums/atomic.cr
  src/base64.cr
  src/bool.cr
  src/box.cr
  src/char.cr
  src/char/reader.cr
  src/class.cr
  src/concurrent.cr
    src/fiber.cr
      src/crystal/system/thread_linked_list.cr
        src/crystal/system/thread_mutex.cr
          src/crystal/system/unix/pthread_mutex.cr
            src/lib_c/x86_64-linux-gnu/c/pthread.cr
      src/fiber/context.cr
        src/fiber/context/aarch64.cr
        src/fiber/context/arm.cr
        src/fiber/context/i386.cr
        src/fiber/context/interpreted.cr
        src/fiber/context/wasm32.cr
        src/fiber/context/x86_64-microsoft.cr
        src/fiber/context/x86_64-sysv.cr
      src/fiber/stack_pool.cr
        src/crystal/system/fiber.cr
          src/crystal/system/unix/fiber.cr
            src/lib_c/x86_64-linux-gnu/c/sys/mman.cr
    src/channel.cr
      src/crystal/spin_lock.cr
      src/crystal/pointer_linked_list.cr
    src/crystal/scheduler.cr
      src/crystal/system/event_loop.cr
        src/crystal/system/unix/event_loop_libevent.cr
          src/crystal/system/unix/event_libevent.cr
            src/crystal/system/unix/lib_event2.cr
              src/lib_c/x86_64-linux-gnu/c/netdb.cr
                src/lib_c/x86_64-linux-gnu/c/netinet/in.cr
                  src/lib_c/x86_64-linux-gnu/c/sys/socket.cr
      src/crystal/system/print_error.cr
      src/crystal/fiber_channel.cr
      src/crystal/system/thread.cr
        src/crystal/system/thread_condition_variable.cr
          src/crystal/system/unix/pthread_condition_variable.cr
        src/crystal/system/unix/pthread.cr
          src/lib_c/x86_64-linux-gnu/c/sched.cr
  src/crystal/compiler_rt.cr
    src/crystal/compiler_rt/fixint.cr
    src/crystal/compiler_rt/float.cr
    src/crystal/compiler_rt/mul.cr
    src/crystal/compiler_rt/divmod128.cr
  src/crystal/main.cr
    src/process/executable_path.cr
  src/deque.cr
  src/dir.cr
    src/crystal/system/dir.cr
      src/crystal/system/unix/dir.cr
        src/lib_c/x86_64-linux-gnu/c/dirent.cr
    src/dir/glob.cr
  src/enum.cr
  src/env.cr
    src/crystal/system/env.cr
      src/crystal/system/unix/env.cr
  src/errno.cr
  src/winerror.cr
  src/wasi_error.cr
  src/file.cr
    src/file/error.cr
    src/crystal/system/file.cr
      src/crystal/system/unix/file.cr
        src/lib_c/x86_64-linux-gnu/c/sys/file.cr
    src/file/info.cr
      src/crystal/system/file_info.cr
        src/crystal/system/unix/file_info.cr
    src/file/preader.cr
    src/file/tempfile.cr
  src/float.cr
    src/float/printer.cr
      src/float/printer/cached_powers.cr
      src/float/printer/diy_fp.cr
        src/float/printer/ieee.cr
      src/float/printer/dragonbox.cr
        src/float/printer/dragonbox_cache.cr
      src/float/printer/grisu3.cr
  src/gc.cr
    src/gc/boehm.cr
  src/hash.cr
    src/crystal/hasher.cr
  src/int.cr
  src/intrinsics.cr
  src/kernel.cr
    src/crystal/at_exit_handlers.cr
  src/math/math.cr
    src/math/libm.cr
  src/mutex.cr
  src/named_tuple.cr
  src/nil.cr
  src/humanize.cr
  src/path.cr
    src/crystal/system/path.cr
      src/crystal/system/unix/path.cr
        src/crystal/system/unix/user.cr
          src/lib_c/x86_64-linux-gnu/c/pwd.cr
          src/crystal/system/unix.cr
  src/pointer.cr
  src/pretty_print.cr
  src/proc.cr
  src/process.cr
    src/crystal/system/process.cr
      src/crystal/system/unix/process.cr
        src/lib_c/x86_64-linux-gnu/c/sys/resource.cr
    src/process/shell.cr
    src/process/status.cr
  src/raise.cr
  src/random.cr
    src/random/pcg32.cr
      src/random/secure.cr
        src/crystal/system/random.cr
          src/crystal/system/unix/getrandom.cr
            src/crystal/system/unix/syscall.cr
              src/syscall.cr
                src/syscall/aarch64-linux.cr
                src/syscall/arm-linux.cr
                src/syscall/i386-linux.cr
                src/syscall/x86_64-linux.cr
  src/range.cr
    src/range/bsearch.cr
  src/reference.cr
  src/regex.cr
    src/regex/engine.cr
      src/regex/pcre2.cr
        src/regex/lib_pcre2.cr
    src/regex/match_data.cr
  src/set.cr
  src/signal.cr
    src/crystal/system/signal.cr
      src/crystal/system/unix/signal.cr
  src/slice.cr
    src/slice/sort.cr
  src/static_array.cr
  src/struct.cr
  src/symbol.cr
  src/system.cr
    src/crystal/system.cr
      src/crystal/system/unix/hostname.cr
      src/crystal/system/unix/sysconf_cpucount.cr
  src/time.cr
    src/crystal/system/time.cr
      src/crystal/system/unix/time.cr
        src/lib_c/x86_64-linux-gnu/c/sys/time.cr
    src/time/format.cr
      src/time/format/formatter.cr
        src/time/format/pattern.cr
      src/time/format/parser.cr
      src/time/format/custom/http_date.cr
        src/time/format/custom/rfc_2822.cr
      src/time/format/custom/iso_8601.cr
      src/time/format/custom/rfc_3339.cr
      src/time/format/custom/yaml_date.cr
    src/time/location.cr
      src/time/location/loader.cr
    src/time/span.cr
  src/tuple.cr
  src/unicode/unicode.cr
    src/unicode/data.cr
  src/union.cr
  src/va_list.cr
    src/lib_c/x86_64-linux-gnu/c/stdarg.cr
  src/value.cr

Resolves #11481

src/compiler/crystal/tools/dependencies.cr Outdated Show resolved Hide resolved
Comment on lines 21 to 25
config = create_compiler "tool dependencies", no_codegen: true
config.compiler.no_codegen = true

config.compiler.dependency_printer = dependency_printer || DependencyPrinter.new(STDOUT)
config.compile
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be expressed as an extra parameter of Crystal::Command#compile_no_codegen? In that case the command-line argument handling would be done inside #create_compiler itself, similar to hierarchy

Copy link
Member Author

@straight-shoota straight-shoota Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the benefit? It seems to me this is a pretty simple setup. Yes, it's some duplicate code. But that's not bad. Combining everthing in one place adds quite a bit of complexity.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't run this branch yet, but I believe this means the tool doesn't have access to any of the common command-line options that other tools have (-D, --prelude etc.)

Copy link
Contributor

@HertzDevil HertzDevil Jun 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is indeed a problem with this approach: crystal tool dependencies still shows the full help rather than this help, but the option parser only ever accepts this -f, and not the actually shown options like -h. In particular this -f tree|flat conflicts with the other -f text|json.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If crystal tool hierarchy accepts -D and --prelude, so should this tool; this is not merely a matter of code duplication

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actual common piece that should be reused for a single option parser instance is create_compiler. Utilizing compile_no_codegen would add unnecessary complexity for no real gain.

src/compiler/crystal/tools/dependencies.cr Outdated Show resolved Hide resolved
src/compiler/crystal/tools/dependencies.cr Outdated Show resolved Hide resolved
Co-authored-by: Quinton Miller <nicetas.c@gmail.com>
@beta-ziliani
Copy link
Member

beta-ziliani commented Jun 30, 2023

how hard would it be to remove a branch? like --ignore=prelude.cr removing the prelude so you can focus on the files on your project. and/or additionally --exclusive=./src to just display what's in ./src

@straight-shoota
Copy link
Member Author

straight-shoota commented Jul 1, 2023

@beta-ziliani It won't be hard. But I would prefer to keep this PR focused on a minimal implementation. Let's talk about these enhancements in a separate issue.

@dammer
Copy link
Contributor

dammer commented Jul 3, 2023

@straight-shoota Just as a suggestion, if it's possible to add redundant dependency warnings in the Crystal file that would be great.

@beta-ziliani beta-ziliani added this to the 1.10.0 milestone Jul 3, 2023
@Sija
Copy link
Contributor

Sija commented Jul 3, 2023

PR title still refers to the singular name (dependency vs dependencies)

@beta-ziliani beta-ziliani changed the title Add crystal tool dependency Add crystal tool dependencies Jul 3, 2023
@straight-shoota straight-shoota changed the base branch from master to feature/tool-dependencies July 4, 2023 15:23
@straight-shoota straight-shoota removed this from the 1.10.0 milestone Jul 4, 2023
@straight-shoota
Copy link
Member Author

straight-shoota commented Jul 4, 2023

I've changed the target branch to a feature branch on this repo (#13631). I'll be making a couple of individual PRs to that branch and we can merge that all together into master after the freeze period for 1.9 is over.

@straight-shoota straight-shoota merged commit 8ddf285 into crystal-lang:feature/tool-dependencies Jul 4, 2023
@straight-shoota straight-shoota deleted the feat/tool-dependencies branch July 4, 2023 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

crystal tool dependencies: Displaying compile-time source dependencies
5 participants