chore: Add MIT and Apache licenses and update crate licenses

[austinabell]

I just updated the dependency crates used from the SDK

Open questions before opening PR:

• Does the copyright year and holder need to be filled out for these?
  • Note: it's not filled out in the gpl3 license
• Is it fine for the other license files to be in the root directory?
• Should the licenses be symlinked to each crate specifically that uses it?
  • If this is the case, should the licenses be moved out of root to not apply to whatever gpl3 needs to be applied to? I'm not sure what parts need to be gpl3

[austinabell]

cc @nearmax @chefsale I'm leaving this PR as a draft until the questions above are resolved and I don't have the context required to answer those questions about the licenses.

[bowenwang1996]

Are you sure that these crates do not have GPL dependencies? In general we cannot switch to apache or MIT license in nearcore due to GPL dependencies

[austinabell]

Are you sure that these crates do not have GPL dependencies? In general we cannot switch to apache or MIT license in nearcore due to GPL dependencies

vm-logic:

~/dev/git/nea/nea/run/near-vm-logic (austin/licenses|✔) $ cargo license                                                             [1/1]
0BSD OR Apache-2.0 OR MIT (1): adler
Apache-2.0 (7): borsh-derive, borsh-derive-internal, borsh-schema-derive-internal, near-account-id, openssl, parity-scale-codec, parity-s
cale-codec-derive
Apache-2.0 OR Apache-2.0 WITH LLVM-exception OR MIT (2): wasi, wasi
Apache-2.0 OR BSL-1.0 (1): ryu
Apache-2.0 OR MIT (193): actix-codec, actix-http, actix-macros, actix-router, actix-rt, actix-server, actix-service, actix-tls, actix-uti
ls, actix-web, actix-web-codegen, ahash, ahash, anyhow, arrayvec, arrayvec, autocfg, base64, base64, bencher, bitflags, blake2, block-buf
fer, block-padding, borsh, brotli-sys, brotli2, bs58, bumpalo, bytestring, c2-chacha, cc, cfg-if, cfg-if, chrono, cipher, const_fn, cooki
e, cpuid-bool, crc32fast, crypto-mac, digest, dtoa, easy-ext, ed25519, either, encoding_rs, fixed-hash, flate2, fnv, foreign-types, forei
gn-types-shared, form_urlencoded, futures, futures-channel, futures-core, futures-executor, futures-io, futures-macro, futures-sink, futu
res-task, futures-util, getrandom, getrandom, hashbrown, heck, hermit-abi, hex, hex-literal, hex-literal-impl, http, httparse, idna, impl
-codec, impl-trait-for-tuples, indexmap, itertools, itoa, jemalloc-sys, jemallocator, jobserver, lazy_static, libc, linked-hash-map, loca
l-channel, local-waker, lock_api, log, mime, miow, near-crypto, near-primitives, near-primitives-core, near-rpc-error-core, near-rpc-erro
r-macro, near-vm-errors, near-vm-logic, ntapi, num-bigint, num-integer, num-rational, num-traits, num_cpus, once_cell, opaque-debug, open
ssl-src, paperclip, paperclip-actix, paperclip-core, paperclip-macros, parking_lot, parking_lot_core, paste, percent-encoding, pin-projec
t, pin-project-internal, pin-project-lite, pin-utils, pkg-config, ppv-lite86, primitive-types, proc-macro-crate, proc-macro-crate, proc-m
acro-error, proc-macro-error-attr, proc-macro-hack, proc-macro-nested, proc-macro2, quote, rand, rand, rand_chacha, rand_chacha, rand_cor
e, rand_core, rand_hc, rand_hc, regex, regex-syntax, ripemd160, rustc-hex, rustc_version, scopeguard, semver, semver-parser, serde, serde
_derive, serde_json, serde_urlencoded, serde_yaml, sha-1, sha2, sha3, signal-hook-registry, signature, smallvec, socket2, standback, stat
ic_assertions, stdweb, stdweb-derive, stdweb-internal-macros, stdweb-internal-runtime, syn, thiserror, thiserror-impl, thread_local, time
, time, time-macros, time-macros-impl, tokio-openssl, toml, typenum, uint, unicode-bidi, unicode-normalization, unicode-segmentation, uni
code-xid, url, vcpkg, version_check, wasm-bindgen, wasm-bindgen-backend, wasm-bindgen-macro, wasm-bindgen-macro-support, wasm-bindgen-sha
red, winapi, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, yaml-rust, zeroize, zeroize_derive
Apache-2.0 OR MIT OR Zlib (3): miniz_oxide, tinyvec, tinyvec_macros
BSD-2-Clause (1): arrayref
BSD-3-Clause (5): curve25519-dalek, ed25519-dalek, instant, sha1, subtle
CC0-1.0 (2): keccak, parity-secp256k1
MIT (33): base-x, bitvec, byte-slice-cast, bytes, crunchy, derive_more, discard, fs_extra, funty, generic-array, h2, language-tags, match
es, mio, openssl-sys, radium, redox_syscall, reed-solomon-erasure, slab, smart-default, strum, strum_macros, synstructure, tap, tokio, to
kio-macros, tokio-util, tracing, tracing-attributes, tracing-core, validator, validator_types, wyz
MIT OR Unlicense (3): aho-corasick, byteorder, memchr

primitives:

~/dev/git/nea/nea/cor/primitives (austin/licenses|✔) $ cargo license
0BSD OR Apache-2.0 OR MIT (1): adler
Apache-2.0 (7): borsh-derive, borsh-derive-internal, borsh-schema-derive-internal, near-account-id, openssl, parity-scale-codec, parity-scale-codec-derive
Apache-2.0 OR Apache-2.0 WITH LLVM-exception OR MIT (2): wasi, wasi
Apache-2.0 OR BSL-1.0 (1): ryu
Apache-2.0 OR MIT (190): actix-codec, actix-http, actix-macros, actix-router, actix-rt, actix-server, actix-service, actix-tls, actix-utils, actix-web, actix-web-codegen, ahash, ahash, anyhow, arrayvec, arrayvec, autocfg, base64, base64, bencher, bitflags, blake2, block-buffer, block-padding, borsh, brotli-sys, brotli2, bs58, bumpalo, bytestring, c2-chacha, cc, cfg-if, cfg-if, chrono, cipher, const_fn, cookie, cpuid-bool, crc32fast, crypto-mac, digest, dtoa, easy-ext, ed25519, either, encoding_rs, fixed-hash, flate2, fnv, foreign-types, foreign-types-shared, form_urlencoded, futures, futures-channel, futures-core, futures-executor, futures-io, futures-macro, futures-sink, futures-task, futures-util, getrandom, getrandom, hashbrown, heck, hermit-abi, hex, hex-literal, hex-literal-impl, http, httparse, idna, impl-codec, impl-trait-for-tuples, indexmap, itertools, itoa, jemalloc-sys, jemallocator, jobserver, lazy_static, libc, linked-hash-map, local-channel, local-waker, lock_api, log, mime, miow, near-crypto, near-primitives, near-primitives-core, near-rpc-error-core, near-rpc-error-macro, near-vm-errors, ntapi, num-bigint, num-integer, num-rational, num-traits, num_cpus, once_cell, opaque-debug, openssl-src, paperclip, paperclip-actix, paperclip-core, paperclip-macros, parking_lot, parking_lot_core, paste, percent-encoding, pin-project, pin-project-internal, pin-project-lite, pin-utils, pkg-config, ppv-lite86, primitive-types, proc-macro-crate, proc-macro-crate, proc-macro-error, proc-macro-error-attr, proc-macro-hack, proc-macro-nested, proc-macro2, quote, rand, rand, rand_chacha, rand_chacha, rand_core, rand_core, rand_hc, rand_hc, regex, regex-syntax, rustc-hex, rustc_version, scopeguard, semver, semver-parser, serde, serde_derive, serde_json, serde_urlencoded, serde_yaml, sha-1, sha2, signal-hook-registry, signature, smallvec, socket2, standback, static_assertions, stdweb, stdweb-derive, stdweb-internal-macros, stdweb-internal-runtime, syn, thiserror, thiserror-impl, thread_local, time, time, time-macros, time-macros-impl, tokio-openssl, toml, typenum, uint, unicode-bidi, unicode-normalization, unicode-segmentation, unicode-xid, url, vcpkg, version_check, wasm-bindgen, wasm-bindgen-backend, wasm-bindgen-macro, wasm-bindgen-macro-support, wasm-bindgen-shared, winapi, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, yaml-rust, zeroize, zeroize_derive
Apache-2.0 OR MIT OR Zlib (3): miniz_oxide, tinyvec, tinyvec_macros
BSD-2-Clause (1): arrayref
BSD-3-Clause (5): curve25519-dalek, ed25519-dalek, instant, sha1, subtle
CC0-1.0 (1): parity-secp256k1
MIT (33): base-x, bitvec, byte-slice-cast, bytes, crunchy, derive_more, discard, fs_extra, funty, generic-array, h2, language-tags, matches, mio, openssl-sys, radium, redox_syscall, reed-solomon-erasure, slab, smart-default, strum, strum_macros, synstructure, tap, tokio, tokio-macros, tokio-util, tracing, tracing-attributes, tracing-core, validator, validator_types, wyz
MIT OR Unlicense (3): aho-corasick, byteorder, memchr

All others changes are dependencies of these. Is there a concern that one of these is licensed other than these and just not updated in cargo? What are the dependencies that are gpl3? I can't even find any on neard

[bowenwang1996]

What are the dependencies that are gpl3?

There are some parity dependencies (pwasm-utils) for example.

[bowenwang1996]

@chefsale could you review this change?

[chefsale]

This looks reasonable to me. Let's just use little or instead of OR in the string. Migrating from Apache to Apache or MIT will not create any new issue except the ones which already exists there will still be there, so this can only be an improvement. We just need to take a look as a follow up, is there and GPL based licence which we include in those crates if yes all the crates should be GPL licenced.

[austinabell]

This looks reasonable to me. Let's just use little or instead of OR in the string. Migrating from Apache to Apache or MIT will not create any new issue except the ones which already exists there will still be there, so this can only be an improvement. We just need to take a look as a follow up, is there and GPL based licence which we include in those crates if yes all the crates should be GPL licenced.

Every rust license declaration I've seen uses OR over or, for example Rust itself https://github.com/rust-lang/rust/blob/master/library/std/Cargo.toml#L4. As for checking if any GPL, why not just do this before this comes in, to avoid any versions being cut with an invalid license? Is there anything I can do to help with this? I'm a bit unfamiliar with this process

[chefsale]

The scan results can be found here for this PR once done: https://app.fossa.com/projects/custom+21311%2Fgit@github.com:near%2Fnearcore.git/refs/branch/austin%2Flicenses/dd11010a22c65447890a5673653b8811ffa7b2c7

This is for the whole nearcore repo, some of them are GPL/LGPL issues on master which aren't resolved. @bowenwang1996 and @matklad will probably be able to say more about specific packages and dependencies.

[chefsale]

Update on the previous comment once the scan has finished, the dependencies which are in a violation are:

• gnuplot
• librocksdb-sys
• jemalloc-sys
• openssl-sys

If none of these are used in these crates transitively, we are good to proceed.

[matklad]

jemalloc is mit/apache: https://github.com/gnzlbg/jemallocator/blob/master/jemalloc-sys/Cargo.toml#L10

[bowenwang1996]

If none of these are used in these crates transitively, we are good to proceed.

@austinabell please confirm this and move this PR forward

[austinabell]

Update on the previous comment once the scan has finished, the dependencies which are in a violation are:

• gnuplot
• librocksdb-sys
• jemalloc-sys
• openssl-sys

If none of these are used in these crates transitively, we are good to proceed.

As well as jemalloc-sys being MIT/apache2 as Aleksey points out, openssl-sys and librocksdb-sys (although includes bsd3, is this an issue?) are not gpl3, so fine to just ensure that gnuplot is not used with any of these?

Also checked, gnuplot is only used in runtime-params-estimator, which doesn't seem to be used at all transitively within nearcore and definitely not through the deps I'm making as the cargo licenses would have included it (and checked cargo tree). I'm going to open this PR given no reasons against given yet, and reviewers can try to find any flaws with this.

[matklad]

not a lawyer, but LGTM!

After this PR, it'd be cool to add a CI check that we don't introduce gpl dependencies inadvertently. Here's how this check looks in rust-analyzer:

https://github.com/rust-analyzer/rust-analyzer/blob/master/crates/rust-analyzer/tests/slow-tests/tidy.rs#L213-L267

[chefsale]

@matklad we already use FOSSA and there could be such a policy implemented there once we have resolved all the current compliance issues.

[frol]

We already use cargo-deny to check duplicate dependencies, and it also can check the licenses

nearcore/.buildkite/pipeline.yml

Lines 22 to 28 in e5315e5

	- label: "sanity checks"
	command: |
	source ~/.cargo/env && set -eux
	rustc --version && cargo --version
	if [ -e deny.toml ]; then
	cargo-deny --all-features check bans
	fi

[chefsale]

We already use cargo-deny to check duplicate dependencies, and it also can check the licenses

nearcore/.buildkite/pipeline.yml

Lines 22 to 28 in e5315e5

	- label: "sanity checks"
	command: |
	source ~/.cargo/env && set -eux
	rustc --version && cargo --version
	if [ -e deny.toml ]; then
	cargo-deny --all-features check bans
	fi

Even better, didn't know this is in the CI :P

[frol]

I am not a lawyer either, it looks good to me

[bowenwang1996]

I am not a lawyer

Is this the new pick up phrase these days :)