Cross-package dependencies

[rjackson]

Hi folks,

I'm working on a project where we're swapping out a copy of Zend Framework 1 on the filesystem for your decomposed packages, piece-by-piece.

A potential issue I've spotted is a few of the packages have references to other packages, which aren't listed as dependencies or suggestions in their respective composer.json.

I've done a bit of rough analysis on this by searching each package's directory for references to Zend_(something) where (something) excludes the package's own name. This does catch comments and instanceof checks, so it's not necessary representative of true dependencies. Some references are also within adapter classes, where it might be more appropriate to list a package as a suggested dependency rather than a direct dependency.

There are a lot of packages, so before digging in and reviewing any further I thought I'd raise an issue and ask your thoughts on this. Do we think this an issue, or are we confident the existing composer.json is suitably configured?

Rough analysis

Summary of search

Package	Dependencies in composer	Dependencies missing from composer	Unknown packages
packages/zend-acl	1	0	0
packages/zend-amf	5	5	0
packages/zend-application	3	14	0
packages/zend-auth	5	2	0
packages/zend-barcode	4	1	2
packages/zend-cache	2	3	0
packages/zend-captcha	5	5	0
packages/zend-cloud	6	0	0
packages/zend-codegenerator	4	0	0
packages/zend-config	2	1	3
packages/zend-console-getopt	0	0	0
packages/zend-controller	10	4	0
packages/zend-crypt	1	0	0
packages/zend-currency	2	3	0
packages/zend-date	2	2	1
packages/zend-db	4	2	0
packages/zend-debug	0	0	0
packages/zend-dojo	6	2	0
packages/zend-dom	2	0	0
packages/zend-eventmanager	2	0	0
packages/zend-exception	0	0	0
packages/zend-feed	6	8	0
packages/zend-file	0	0	0
packages/zend-file-transfer	4	3	1
packages/zend-filter	5	4	1
packages/zend-form	9	5	1
packages/zend-gdata	5	2	3
packages/zend-http	3	3	1
packages/zend-json	3	0	0
packages/zend-layout	5	1	0
packages/zend-ldap	1	3	0
packages/zend-loader	1	1	2
packages/zend-locale	4	1	2
packages/zend-log	2	5	0
packages/zend-mail	4	2	1
packages/zend-markup	4	0	0
packages/zend-measure	3	0	0
packages/zend-memory	2	2	1
packages/zend-mime	1	0	0
packages/zend-mobile	4	0	0
packages/zend-navigation	5	1	0
packages/zend-oauth	5	0	0
packages/zend-openid	4	1	0
packages/zend-paginator	6	3	0
packages/zend-pdf	3	2	0
packages/zend-progressbar	4	1	0
packages/zend-queue	4	2	0
packages/zend-reflection	2	0	0
packages/zend-registry	2	0	0
packages/zend-rest	5	3	0
packages/zend-search	1	0	0
packages/zend-search-lucene	2	0	1
packages/zend-serializer	5	1	0
packages/zend-server	1	0	0
packages/zend-service	2	0	0
packages/zend-service-akismet	4	0	0
packages/zend-service-amazon	4	4	0
packages/zend-service-audioscrobbler	3	0	0
packages/zend-service-console	1	0	0
packages/zend-service-delicious	5	0	0
packages/zend-service-ebay	3	3	0
packages/zend-service-flickr	4	0	0
packages/zend-service-livedocx	4	0	0
packages/zend-service-rackspace	2	1	0
packages/zend-service-recaptcha	3	2	0
packages/zend-service-shorturl	3	1	0
packages/zend-service-slideshare	4	0	0
packages/zend-service-strikeiron	2	0	0
packages/zend-service-twitter	3	2	0
packages/zend-service-windowsazure	3	1	0
packages/zend-service-yahoo	4	1	0
packages/zend-session	4	0	0
packages/zend-soap	4	1	1
packages/zend-stdlib	0	0	0
packages/zend-tag	3	0	0
packages/zend-test	7	3	0
packages/zend-text	2	1	2
packages/zend-timesync	3	1	0
packages/zend-tool	1	2	0
packages/zend-translate	4	3	1
packages/zend-uri	4	1	0
packages/zend-validate	6	6	0
packages/zend-version	0	0	0
packages/zend-view	7	11	1
packages/zend-wildfire	4	0	0
packages/zend-xml	1	0	0
packages/zend-xmlrpc	4	3	1

Search script (bash)

Expand to show script

#!/usr/bin/env bash

# This script will check each Zend package for any references to packages other than itself, and make sure those
# packages are referenced via Composer.json

summary=()

for package in packages/*; do
    printf "| %-30s | %-30s |\n" "$package" "Referenced in composer.json"
    printf "| "
    printf -- "-%.0s" {0..29}
    printf " | "
    printf -- "-%.0s" {0..29}
    printf " |\n"

    unrecognised_packages=()
    listed_as_dependency=0
    not_listed_as_dependency=0
    unrecognised=0

    depends_on=($(ggrep -IroP 'Zend_(?!'"$package"')\w*' "$package" | cut -d':' -f 2 | awk -F'_' '{print tolower($1 FS $2)}' | sort -u | tr '_' '-'))

    for dependency in "${depends_on[@]}"; do
        # Make sure each resolved package actually exists
        if [ ! -d packages/"$dependency" ]; then

            # Manually correct zend-console references to zend-console-getopt
            if [ "$dependency" == "zend-console" ]; then
                dependency="zend-console-getopt"
                break
            fi

            unrecognised_packages+=("$dependency")
            continue
        fi

        # If we have found ourselves, skip it
        if [ "packages/$dependency" == "$package" ]; then
            continue
        fi

        # Make sure its included in composer.json
        if grep -q "$dependency" "$package/composer.json"; then
            printf "| %-30s | %-30s |\n" "$dependency" "Yes"
            listed_as_dependency=$((listed_as_dependency + 1))
        else
            printf "| %-30s | %-30s |\n" "$dependency" "**No**"
            not_listed_as_dependency=$((not_listed_as_dependency + 1))
        fi
    done

    for unknown_package in "${unrecognised_packages[@]}"; do
        printf "| %-30s | %-30s |\n" "$unknown_package" "(package does not exist)"
        unrecognised=$((unrecognised + 1))
    done

    summary+=("$(printf '| %-40s | %30d | %40d | %20d |' "$package" "$listed_as_dependency" "$not_listed_as_dependency" "$unrecognised")")

    printf "\n\n"
done

printf "| %-40s | %-30s | %-40s | %-20s |\n" "Package" "Dependencies in composer" "Dependencies missing from composer" "Unknown packages"
printf "| "
printf -- "-%.0s" {0..39}
printf " | "
printf -- "-%.0s" {0..29}
printf " | "
printf -- "-%.0s" {0..39}
printf " | "
printf -- "-%.0s" {0..19}
printf " |\n"

for item in "${summary[@]}"; do
    echo "$item"
done

Full output

Show full output

packages/zend-acl	Referenced in composer.json
zend-exception	Yes

packages/zend-amf	Referenced in composer.json
zend-acl	No
zend-auth	No
zend-date	Yes
zend-db	No
zend-exception	Yes
zend-loader	Yes
zend-reflection	No
zend-server	Yes
zend-session	No
zend-xml	Yes

packages/zend-application	Referenced in composer.json
zend-cache	No
zend-config	Yes
zend-controller	Yes
zend-db	No
zend-dojo	No
zend-exception	Yes
zend-http	No
zend-layout	No
zend-loader	No
zend-locale	No
zend-log	No
zend-mail	No
zend-navigation	No
zend-registry	No
zend-session	No
zend-translate	No
zend-view	No

packages/zend-auth	Referenced in composer.json
zend-controller	No
zend-db	Yes
zend-exception	Yes
zend-http	No
zend-ldap	Yes
zend-openid	Yes
zend-session	Yes

packages/zend-barcode	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-loader	No
zend-pdf	Yes
zend-validate	Yes
zend-image	(package does not exist)
zend-renderer	(package does not exist)

packages/zend-cache	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-file	No
zend-loader	No
zend-log	Yes

packages/zend-captcha	Referenced in composer.json
zend-config	No
zend-crypt	Yes
zend-exception	Yes
zend-form	No
zend-loader	No
zend-service	Yes
zend-session	No
zend-text	Yes
zend-validate	Yes
zend-view	No

packages/zend-cloud	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-http	Yes
zend-loader	Yes
zend-queue	Yes
zend-service	Yes

packages/zend-codegenerator	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-loader	Yes
zend-reflection	Yes

packages/zend-config	Referenced in composer.json
zend-exception	Yes
zend-json	No
zend-xml	Yes
zend-configs	(package does not exist)
zend-package	(package does not exist)
zend-yaml	(package does not exist)

packages/zend-console-getopt	Referenced in composer.json

packages/zend-controller	Referenced in composer.json
zend-cache	No
zend-config	Yes
zend-dojo	Yes
zend-exception	Yes
zend-filter	Yes
zend-json	Yes
zend-layout	Yes
zend-loader	Yes
zend-locale	No
zend-registry	Yes
zend-session	No
zend-translate	No
zend-uri	Yes
zend-view	Yes

packages/zend-crypt	Referenced in composer.json
zend-exception	Yes

packages/zend-currency	Referenced in composer.json
zend-cache	No
zend-exception	Yes
zend-loader	No
zend-locale	Yes
zend-service	No

packages/zend-date	Referenced in composer.json
zend-cache	No
zend-exception	Yes
zend-locale	Yes
zend-timesync	No
zend-dateobject	(package does not exist)

packages/zend-db	Referenced in composer.json
zend-cache	No
zend-config	No
zend-exception	Yes
zend-loader	Yes
zend-registry	Yes
zend-wildfire	Yes

packages/zend-debug	Referenced in composer.json

packages/zend-dojo	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-filter	Yes
zend-form	Yes
zend-json	Yes
zend-loader	No
zend-registry	Yes
zend-view	Yes

packages/zend-dom	Referenced in composer.json
zend-exception	Yes
zend-xml	Yes

packages/zend-eventmanager	Referenced in composer.json
zend-exception	Yes
zend-stdlib	Yes

packages/zend-exception	Referenced in composer.json

packages/zend-feed	Referenced in composer.json
zend-cache	No
zend-config	No
zend-controller	No
zend-date	No
zend-db	No
zend-exception	Yes
zend-http	Yes
zend-loader	Yes
zend-oauth	No
zend-registry	No
zend-uri	Yes
zend-validate	No
zend-version	Yes
zend-xml	Yes

packages/zend-file	Referenced in composer.json

packages/zend-file-transfer	Referenced in composer.json
zend-exception	Yes
zend-file	Yes
zend-filter	No
zend-loader	Yes
zend-progressbar	No
zend-translate	No
zend-validate	Yes
zend-	(package does not exist)

packages/zend-filter	Referenced in composer.json
zend-config	No
zend-crypt	Yes
zend-date	No
zend-exception	Yes
zend-loader	Yes
zend-locale	Yes
zend-registry	No
zend-translate	No
zend-validate	Yes
zend-	(package does not exist)

packages/zend-form	Referenced in composer.json
zend-captcha	Yes
zend-config	No
zend-controller	Yes
zend-crypt	No
zend-exception	Yes
zend-file	No
zend-filter	Yes
zend-json	Yes
zend-loader	Yes
zend-registry	Yes
zend-session	Yes
zend-translate	No
zend-validate	Yes
zend-view	No
zend-	(package does not exist)

packages/zend-gdata	Referenced in composer.json
zend-crypt	No
zend-exception	Yes
zend-http	Yes
zend-loader	Yes
zend-uri	No
zend-version	Yes
zend-xml	Yes
zend-calendar	(package does not exist)
zend-extension	(package does not exist)
zend-framework	(package does not exist)

packages/zend-http	Referenced in composer.json
zend-config	No
zend-date	No
zend-exception	Yes
zend-loader	Yes
zend-session	No
zend-uri	Yes
zend-browser	(package does not exist)

packages/zend-json	Referenced in composer.json
zend-exception	Yes
zend-server	Yes
zend-xml	Yes

packages/zend-layout	Referenced in composer.json
zend-config	No
zend-controller	Yes
zend-exception	Yes
zend-filter	Yes
zend-loader	Yes
zend-view	Yes

packages/zend-ldap	Referenced in composer.json
zend-config	No
zend-crypt	No
zend-date	No
zend-exception	Yes

packages/zend-loader	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-	(package does not exist)
zend-example	(package does not exist)

packages/zend-locale	Referenced in composer.json
zend-cache	Yes
zend-date	No
zend-exception	Yes
zend-registry	Yes
zend-xml	Yes
zend-localec	(package does not exist)
zend-localel	(package does not exist)

packages/zend-log	Referenced in composer.json
zend-config	No
zend-db	No
zend-exception	Yes
zend-layout	No
zend-loader	No
zend-mail	No
zend-wildfire	Yes

packages/zend-mail	Referenced in composer.json
zend-config	No
zend-date	No
zend-exception	Yes
zend-loader	Yes
zend-mime	Yes
zend-validate	Yes
zend-transport	(package does not exist)

packages/zend-markup	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-filter	Yes
zend-loader	Yes

packages/zend-measure	Referenced in composer.json
zend-exception	Yes
zend-locale	Yes
zend-registry	Yes

packages/zend-memory	Referenced in composer.json
zend-cache	Yes
zend-controller	No
zend-exception	Yes
zend-loader	No
zend-memoryexception	(package does not exist)

packages/zend-mime	Referenced in composer.json
zend-exception	Yes

packages/zend-mobile	Referenced in composer.json
zend-exception	Yes
zend-http	Yes
zend-uri	Yes
zend-xml	Yes

packages/zend-navigation	Referenced in composer.json
zend-acl	Yes
zend-config	Yes
zend-controller	Yes
zend-exception	Yes
zend-loader	No
zend-view	Yes

packages/zend-oauth	Referenced in composer.json
zend-config	Yes
zend-crypt	Yes
zend-exception	Yes
zend-http	Yes
zend-uri	Yes

packages/zend-openid	Referenced in composer.json
zend-controller	Yes
zend-crypt	No
zend-exception	Yes
zend-http	Yes
zend-session	Yes

packages/zend-paginator	Referenced in composer.json
zend-cache	No
zend-config	No
zend-controller	Yes
zend-db	Yes
zend-exception	Yes
zend-filter	No
zend-json	Yes
zend-loader	Yes
zend-view	Yes

packages/zend-pdf	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-log	Yes
zend-memory	Yes
zend-mime	No

packages/zend-progressbar	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-json	Yes
zend-session	Yes
zend-text	No

packages/zend-queue	Referenced in composer.json
zend-cache	No
zend-config	Yes
zend-db	Yes
zend-exception	Yes
zend-loader	Yes
zend-log	No

packages/zend-reflection	Referenced in composer.json
zend-exception	Yes
zend-loader	Yes

packages/zend-registry	Referenced in composer.json
zend-exception	Yes
zend-loader	Yes

packages/zend-rest	Referenced in composer.json
zend-config	No
zend-controller	No
zend-exception	Yes
zend-http	No
zend-server	Yes
zend-service	Yes
zend-uri	Yes
zend-xml	Yes

packages/zend-search	Referenced in composer.json
zend-exception	Yes

packages/zend-search-lucene	Referenced in composer.json
zend-search	Yes
zend-xml	Yes
zend-environment	(package does not exist)

packages/zend-serializer	Referenced in composer.json
zend-amf	Yes
zend-config	No
zend-exception	Yes
zend-json	Yes
zend-loader	Yes
zend-xml	Yes

packages/zend-server	Referenced in composer.json
zend-exception	Yes

packages/zend-service	Referenced in composer.json
zend-exception	Yes
zend-http	Yes

packages/zend-service-akismet	Referenced in composer.json
zend-http	Yes
zend-service	Yes
zend-uri	Yes
zend-version	Yes

packages/zend-service-amazon	Referenced in composer.json
zend-crypt	No
zend-http	Yes
zend-json	No
zend-loader	No
zend-rest	Yes
zend-service	Yes
zend-uri	No
zend-xml	Yes

packages/zend-service-audioscrobbler	Referenced in composer.json
zend-http	Yes
zend-service	Yes
zend-xml	Yes

packages/zend-service-console	Referenced in composer.json
zend-service	Yes

packages/zend-service-delicious	Referenced in composer.json
zend-date	Yes
zend-json	Yes
zend-rest	Yes
zend-service	Yes
zend-xml	Yes

packages/zend-service-ebay	Referenced in composer.json
zend-config	No
zend-date	No
zend-http	No
zend-rest	Yes
zend-service	Yes
zend-xml	Yes

packages/zend-service-flickr	Referenced in composer.json
zend-rest	Yes
zend-service	Yes
zend-validate	Yes
zend-xml	Yes

packages/zend-service-livedocx	Referenced in composer.json
zend-config	Yes
zend-date	Yes
zend-service	Yes
zend-soap	Yes

packages/zend-service-rackspace	Referenced in composer.json
zend-http	No
zend-service	Yes
zend-validate	Yes

packages/zend-service-recaptcha	Referenced in composer.json
zend-config	No
zend-http	Yes
zend-json	Yes
zend-service	Yes
zend-validate	No

packages/zend-service-shorturl	Referenced in composer.json
zend-dom	Yes
zend-http	No
zend-service	Yes
zend-uri	Yes

packages/zend-service-slideshare	Referenced in composer.json
zend-cache	Yes
zend-http	Yes
zend-service	Yes
zend-xml	Yes

packages/zend-service-strikeiron	Referenced in composer.json
zend-loader	Yes
zend-service	Yes

packages/zend-service-twitter	Referenced in composer.json
zend-config	No
zend-http	Yes
zend-json	Yes
zend-oauth	No
zend-service	Yes

packages/zend-service-windowsazure	Referenced in composer.json
zend-http	No
zend-log	Yes
zend-service	Yes
zend-xml	Yes

packages/zend-service-yahoo	Referenced in composer.json
zend-rest	Yes
zend-service	Yes
zend-uri	No
zend-validate	Yes
zend-xml	Yes

packages/zend-session	Referenced in composer.json
zend-config	Yes
zend-db	Yes
zend-exception	Yes
zend-loader	Yes

packages/zend-soap	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-server	Yes
zend-uri	Yes
zend-xml	Yes
zend-soapclient	(package does not exist)

packages/zend-stdlib	Referenced in composer.json

packages/zend-tag	Referenced in composer.json
zend-config	Yes
zend-exception	Yes
zend-loader	Yes

packages/zend-test	Referenced in composer.json
zend-application	No
zend-controller	Yes
zend-db	No
zend-dom	Yes
zend-exception	Yes
zend-filter	No
zend-layout	Yes
zend-loader	Yes
zend-registry	Yes
zend-session	Yes

packages/zend-text	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-loader	Yes
zend-figlet	(package does not exist)
zend-zext	(package does not exist)

packages/zend-timesync	Referenced in composer.json
zend-date	Yes
zend-exception	Yes
zend-loader	Yes
zend-locale	No

packages/zend-tool	Referenced in composer.json
zend-application	No
zend-codegenerator	No
zend-config	Yes

packages/zend-translate	Referenced in composer.json
zend-cache	No
zend-config	No
zend-exception	Yes
zend-loader	Yes
zend-locale	Yes
zend-log	No
zend-xml	Yes
zend-translation	(package does not exist)

packages/zend-uri	Referenced in composer.json
zend-config	No
zend-exception	Yes
zend-filter	Yes
zend-loader	Yes
zend-validate	Yes

packages/zend-validate	Referenced in composer.json
zend-config	No
zend-date	Yes
zend-db	No
zend-exception	Yes
zend-file	No
zend-filter	Yes
zend-ldap	No
zend-loader	Yes
zend-locale	Yes
zend-registry	Yes
zend-translate	No
zend-uri	No

packages/zend-version	Referenced in composer.json

packages/zend-view	Referenced in composer.json
zend-acl	No
zend-config	No
zend-controller	Yes
zend-currency	No
zend-exception	Yes
zend-filter	No
zend-form	No
zend-http	No
zend-json	Yes
zend-layout	Yes
zend-loader	Yes
zend-locale	Yes
zend-navigation	No
zend-paginator	No
zend-registry	Yes
zend-translate	No
zend-uri	No
zend-validate	No
zend-navigaiton	(package does not exist)

packages/zend-wildfire	Referenced in composer.json
zend-controller	Yes
zend-exception	Yes
zend-json	Yes
zend-loader	Yes

packages/zend-xml	Referenced in composer.json
zend-exception	Yes

packages/zend-xmlrpc	Referenced in composer.json
zend-controller	No
zend-crypt	No
zend-date	No
zend-exception	Yes
zend-http	Yes
zend-server	Yes
zend-xml	Yes
zend-xmlrpcserver	(package does not exist)

[falkenhawk]

Wow @rjackson that's tremendous work you did here, thanks!
The cross-package dependencies were originally done by @deadbeef84 when he split the packages for https://github.com/zf1?tab=repositories, which later I based this monorepo on.

I think back then he had run a custom script to convert https://framework.zend.com/manual/1.12/en/requirements.introduction.html.

Of course it would be great if you could continue here and update the cross-dependencies to reflect the current state of the packages. I suspect though It might be very tedious to categorize dependencies, to those which are really required (without which there would always be fatal errors), through optional (required only when a package is configured/used in a particular way), to not-needed (e.g. only mentioned in a comment or instanceof check)...

[glensc]

No. the cross-package dependencies were provided by me @glensc:

• composer: fill ext-* dependencies #6

which itself was based on my previous work (described in the PR)

[deadbeef84]

I'm pretty sure @falkenhawk is right, as I remember doing exactly that. The PR you're referencing only adds deps on extensions.

Nice work with the mono-repo 👍 This is the first time I'm seeing it, as I no longer use PHP.

[falkenhawk]

@smhg you might also want to look into this list - but perhaps you've already seen it.

[smhg]

Thanks! I'm sending PR's for the components I encounter in my project. I won't be able to go through all, but this will cover some that are more likely to be in use today.