Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Status 💛 Yarn. Lock dependencies. #6928

Merged
merged 1 commit into from
Nov 29, 2018
Merged

Status 💛 Yarn. Lock dependencies. #6928

merged 1 commit into from
Nov 29, 2018

Conversation

mandrigin
Copy link
Contributor

@mandrigin mandrigin commented Nov 28, 2018

One more step towards reproducible builds.

yarn (as opposed of npm) promises integrity checks and using lock files to download dependencies.
That way, we can avoid security issues with dependencies, just like this one.

Notable change: I had to rename package-lock.json in desktop_files and mobile_files to package-lock.json.orig. That is done to avoid duplicate symbol error when bundling JS.

🚨 🚨 🚨 We use yarn install --frozen-lockfile command. It will fail if the lockfile will change. Use yarn upgrade to upgrade the lockfile

  • Android
  • iOS
  • Setup scripts
  • Docs →→→ https://github.com/status-im/status.im/pull/172
  • Desktop
  • force immutable lock file
  • Jenkins
    • add custom cache folders per runner (so 2 parallel jobs can't conflict) not needed as soon as the cache is downloaded
    • macos-1 doesn't have yarn installed
    • outdated yarn everywhere else
    • desktop fails with react-native-webview-bridge error
    • some fails with yarn cache on macos-2

--

fixes #6906
fixes #3838


status: ready

@ghost
Copy link

ghost commented Nov 28, 2018

Pull Request Checklist

  • Have you updated the documentation, if impacted (e.g. docs.status.im)?

@mandrigin mandrigin self-assigned this Nov 28, 2018
@corpetty
Copy link
Contributor

In order to keep determinism in line, you have to use yarn upgrade <package> when dealing with dependencies. We should also fail builds for dependency changes so they can be properly audited.

@corpetty
Copy link
Contributor

also, great work @mandrigin!!

@mandrigin
Copy link
Contributor Author

We should also fail builds for dependency changes so they can be properly audited.

Yeah, I was thinking about it. we need to block PRs when the yarn.lock is changed.

@mandrigin
Copy link
Contributor Author

Okay, it looks like 2 parallel yarn commands can conflict if running on the same machine. Custom cache path will help with that, even though it will 2x increase a disk space per machine... That is only required for Jenkins.

@jakubgs
Copy link
Member

jakubgs commented Nov 29, 2018

We have plenty of space on jenkins machines, no worries.

@jakubgs
Copy link
Member

jakubgs commented Nov 29, 2018

But this also means I will have to upgrade node on all linux and macos hosts for Jenkins to 9.3.0.

@mandrigin
Copy link
Contributor Author

@jakubgs

But this also means I will have to upgrade node on all linux and macos hosts for Jenkins to 9.3.0.

why is that? it looks like yarn works just fine with 8.x that is installed. I need to double-check that Desktop is buildable on 9.3.0.

@jakubgs
Copy link
Member

jakubgs commented Nov 29, 2018

I guess if it works we can leave it as it is.

Copy link
Member

@jakubgs jakubgs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! Great work.

@status-comment-bot
Copy link

#9 CI BUILD SUCCESSFUL in 35 min and counting (9ab51c0)

Android(e2e) iOS MacOS AppImage Windows

@status-comment-bot
Copy link

#10 CI BUILD SUCCESSFUL in 24 min and counting (9ab51c0)

Android(e2e) iOS MacOS AppImage Windows

@statustestbot
Copy link

98% of end-end tests have passed

Total executed tests: 59
Failed tests: 1
Passed tests: 58

Failed tests (1)

Click to expand
1. test_network_mismatch_for_send_request_commands

Device 1: Tap on GotItButton
Device 1: Looking for a message by text: '0.01129123746'

Device 1: 'ChatElementByText' is not found on the screen

Device sessions

Passed tests (58)

Click to expand
1. test_create_account
Device sessions

2. test_user_can_switch_network
Device sessions

3. test_filters_from_daap
Device sessions

4. test_copy_and_paste_messages
Device sessions

5. test_send_transaction_from_daap
Device sessions

6. test_request_and_receive_tokens_in_1_1_chat
Device sessions

7. test_delete_cut_and_paste_messages
Device sessions

8. test_deploy_contract_from_daap
Device sessions

9. test_offline_login
Device sessions

10. test_pass_phrase_validation
Device sessions

11. test_public_chat_messaging
Device sessions

12. test_password_in_logcat_sign_in
Device sessions

13. test_set_profile_picture
Device sessions

14. test_text_message_1_1_chat
Device sessions

15. test_add_to_contacts
Device sessions

16. test_unread_messages_counter_1_1_chat
Device sessions

17. test_user_can_interact_with_public_chat
Device sessions

18. test_logcat_send_transaction_from_daap
Device sessions

19. test_onboarding_screen_when_requesting_tokens_for_new_account
Device sessions

20. test_logcat_send_transaction_from_wallet
Device sessions

21. test_send_token_with_7_decimals
Device sessions

22. test_modify_transaction_fee_values
Device sessions

23. test_token_with_more_than_allowed_decimals
Device sessions

24. test_send_eth_from_wallet_to_address
Device sessions

25. test_send_transaction_details_in_1_1_chat
Device sessions

26. test_manage_assets
Device sessions

27. test_wallet_set_up
Device sessions

28. test_logcat_send_transaction_in_1_1_chat
Device sessions

29. test_request_and_receive_eth_in_1_1_chat
Device sessions

30. test_swipe_to_delete_public_chat
Device sessions

31. test_passphrase_whitespaces_ignored_while_recovering_access
Device sessions

32. test_send_emoji
Device sessions

33. test_add_contact_by_pasting_public_key
Device sessions

34. test_logcat_recovering_account
Device sessions

35. test_messaging_in_different_networks
Device sessions

36. test_send_tokens_in_1_1_chat
Device sessions

37. test_logcat_sign_message_from_daap
Device sessions

38. test_swipe_to_delete_1_1_chat
Device sessions

39. test_switch_users_and_add_new_account
Device sessions

40. test_send_stt_from_wallet
Device sessions

41. test_send_eth_in_1_1_chat
Device sessions

42. test_login_with_new_account
Device sessions

43. test_send_eth_from_wallet_to_contact
Device sessions

44. test_add_contact_from_public_chat
Device sessions

45. test_send_request_not_enabled_tokens
Device sessions

46. test_send_message_to_newly_added_contact
Device sessions

47. test_password_in_logcat_creating_account
Device sessions

48. test_backup_recovery_phrase
Device sessions

49. test_offline_status
Device sessions

50. test_open_google_com_via_open_dapp
Device sessions

51. test_unread_messages_counter_public_chat
Device sessions

52. test_sign_message_from_daap
Device sessions

53. test_user_can_remove_profile_picture
Device sessions

54. test_share_contact_code_and_wallet_address
Device sessions

55. test_request_eth_in_wallet
Device sessions

56. test_refresh_button_browsing_app_webview
Device sessions

57. test_backup_recovery_phrase_warning_from_wallet
Device sessions

58. test_recover_account
Device sessions

@Serhy
Copy link
Contributor

Serhy commented Nov 29, 2018

@mandrigin, each build #6928 (comment) tested.
I could install, create account and join public channel. Offline messages received.
I could upgrade from 0.9.32 Release build (iOS, Android and MacOS)
Release 0.9.32 can exchange messages with contact using above builds.
Single failed autotest is known GH issue.

@mandrigin mandrigin changed the title [WIP] Status 💛 Yarn Status 💛 Yarn. Lock dependencies. Nov 29, 2018
@andytudhope
Copy link
Contributor

Please be sure to update this docs issue once this is merged 🎉

https://github.com/status-im/status.im/issues/51

@mandrigin
Copy link
Contributor Author

@oskarth
Copy link
Contributor

oskarth commented Nov 29, 2018

Wow, awesome work, let's hope this one goes through! We've had a bunch of attempts at getting yarn into the code base, so really impressive to see this one progressing so quickly and smoothly (and judging by PR checklist, with all the required changed). Fingers crossed :)

@corpetty
Copy link
Contributor

I would like to check determinism of the resulting builds. I'm happy to see this done quickly and smoothly, but we need to see if we're getting the same resulting hash across multiple platform builds.

@mandrigin
Copy link
Contributor Author

@corpetty it doesn't guarantee determinism in the whole build, because we use other package managers in the build chain too (like CocoaPods), they can affect that.

@corpetty
Copy link
Contributor

@mandrigin we could at least hash the part we've changed to see if that is deterministic, then we know we can work forward from there.

Copy link
Contributor

@corpetty corpetty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can worry about pushing determinism forward from here. For now, these changes are great and help multiple things. Great job @mandrigin

@mandrigin
Copy link
Contributor Author

@corpetty you mean the dependencies? hashes of all of them are in the lock file, and --frozen-lockfile that I use everywhere ensures that everything in the node_modules is verified against that. Hence, we can ensure that the dependencies always have the same hash.

Regarding the other parts of the toolchain, they are out of what yarn is doing.

@vkjr
Copy link
Contributor

vkjr commented Nov 29, 2018

@mandrigin, what kind of issues you had with react-native-webview-bridge? In latest Jenkins build I see that it finished successfully and see no output about this module in console log.

@mandrigin
Copy link
Contributor Author

@vkjr it looks like they were cause by the old yarn version, it failed on machine that had yarn 1.8.x. Upgrading yarn to 1.12.3 on the Jenkins machines solved that issue.

@vkjr
Copy link
Contributor

vkjr commented Nov 29, 2018

Great!
Cool that you renamed package.json. I'm attempted to switch desktop to react-native 0.57 but 2 package.json files caused errors. So waiting for this PR to be commited :)

Copy link
Contributor

@pedropombeiro pedropombeiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a bunch for doing this! Great job!

scripts/run-pre-build-check.sh Outdated Show resolved Hide resolved
@status-comment-bot
Copy link

#13 CI BUILD SUCCESSFUL in 24 min and counting (955db2565fa7c57159a820140406f163a15a086e)

Android(e2e) iOS MacOS AppImage Windows

Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
@mandrigin mandrigin merged commit 37ef82b into develop Nov 29, 2018
@mandrigin mandrigin deleted the feature/yarrrrn branch November 29, 2018 15:30
@status-comment-bot
Copy link

#14 CI BUILD SUCCESSFUL in 23 min and counting (37ef82b)

Android(e2e) iOS MacOS AppImage Windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Fail build if package-lock.json changes Move to yarn instead of npm