Node startup and synchronization improvements #559

hiddentao · 2016-05-04T11:52:17Z

I'm heavily refactoring the code which starts up geth and syncs to it. This is intended to both make the code easier to understand and to reduce the frequency of sync errors we seem to be getting with it.

Decoupled the UI from the connection and sync logic - using event-driven architecture instead. This will make stuff more easy to test now and in future.
Create a dedicated Socket class for managing socket connections more intelligently
Refactor ethereumNodes to more intelligently start, shutdown and manage the main Geth node IPC connection
Rewrote synchronisation logic (see nodeSync.js) to be less error-prone, more intelligent.
Refactor how we create and manage windows in the Electron layer so that we avoid errors occurring due to a window already having been destroyed
Splash screen logic simplified. It's now much more in tune with what's going on under the hood. We also now show which network you're on in the top-right to reduce confusion. "MAIN-NET", "TEST-NET" and "DEV-NET".
We have a "Goto app" button that's available as soon as IPC is connected (this replaces the previous "Skip peer search" button).
Switching between test and live network whilst running the app should be less painful now. Note that there is still some IPC connection logic which needs to be refactored for a truly painless experience.
Much more logging throughout the code to help diagnose issues in future
Updated connection logic code to handle Geth 1.4
Improved the fix for Fix Mist startup crash due to about:blank page #548
Upgraded to work with Electron 1.0.1

- remove nodeConnector, shift its functionality into ethereumNode - redo checkNodeSync to be more robust

# Conflicts: # README.md # modules/menuItems.js

hiddentao · 2016-05-10T06:25:44Z

I think this is ready for review now. There's still a bit more work I can do from the UI side of things to improve sync. But I thought it might be a good idea to review what's been done so far and make sure it works. Also, once Geth 1.4 I want to make sure these changes work with it.

hiddentao · 2016-05-10T14:14:21Z

Now that Geth 1.4 is out I think I should test it with these changes to make sure everything works. It also seems that users see sync stopping once the main UI screen is loaded. I haven't yet gotten to that part of the "sync" code, perhaps I can take a look at that too.

…on). - We now show network info in top-right of both splash screen and onboarding window

…e responsive to underlying connection process.

gsilvestri · 2016-05-16T13:35:59Z

Wallet needs to be synched before ANYTHING can be done with the wallet itself.

How difficult would it be to have the wallet loading and synching IN BACKGROUND like Bitcoin Core works?

I'm using Ethereum as an hobbyist and synching the wallet once a week: it takes more than 90 minutes to sync every time on a Fibre connection of more than 30Mb/s speed.
This is very much not user-friendly.

Thank You.

hiddentao · 2016-05-16T13:46:37Z

@zackkoGH This PR will allow you to go straight to the app as soon as the sync process begins - you won't have to wait anymore :)

alexvandesande · 2016-05-16T13:56:24Z

I'm doing a PR on this PR to fix some issues on the template mechanism

gsilvestri · 2016-05-16T14:33:12Z

@hiddentao Thank You for the update.
Will look forward for when this feature will be release.

alexvandesande · 2016-05-17T14:15:42Z

@hiddentao please see hiddentao#1

hiddentao · 2016-05-20T00:13:45Z

Thanks.

--
Ramesh Nair

Director
HiddenTao Ltd (UK company no. 6807289)

http://hiddentao.com
https://github.com/hiddentao
https://www.linkedin.com/in/hiddentao

On 20 May 2016 at 07:30:31, Alex Van de Sande (notifications@github.com) wrote:

Merged #559.

—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub

* Remove old git diff tags * updated nodes (#655) * Upgrade code to work with Electron 1.0.1 (#651) * Remove old git diff tags * fix code to work with Electron 1.0.1 * Korean Translation (#645) * Create app.kr.i18n.json Korean Translation * Create mist.kr.i18n.json * Update app.kr.i18n.json * Added mist.kr.i18n.json Added Korean Translation * Added Korean Translation - app.kr.i18n.json Added Korean Translation * Added Korean Translation -app.kr.i18n.json Added Korean Translation * Added Korean Translation - mist.kr.i18n.json Added Korean Translation * Added Korean (kr) Added Korean (kr) * fixed few Korean translation fixed few Korean translation * Update app.kr.i18n.json * Update mist.kr.i18n.json fixed some Korean translation * fixed linux32 node path and binary (#665) * Node startup and synchronization improvements (#559) * remove commented-out code, cleanup * work towards new socket system and ethereum nodes manager * main: work towards finishing up node connection * refactored node start code * remove old ethereum nodes code * get basic IPC connection to geth working * rename socket to sockets * - shift all node startup code into ethereumNode - remove nodeConnector, shift its functionality into ethereumNode - redo checkNodeSync to be more robust * nodeSync needs to be able to emit events * fixed a number of bugs, and finished sync rewrite. now need to test * basic sync working * slow down the sync check * check that onboarding followed by sync works for testnet * better log msgs, reduce socket disconnection timeout, fix bug * sync is much more resilient now and auto-restarts if node changes * ensure that switching network works once app is loaded * add node 4 dep * Fix menu items code * improve tab resetting logic to happen before Mist crashes * fix how we detect whether eth has started up, remove nodeConnector * ensure can switch network type while syncing * - Make sure it works when using Geth 1.4 (tested the Darwin x64 version). - We now show network info in top-right of both splash screen and onboarding window * Improved how we determine splash screen status messages. Now much more responsive to underlying connection process. * Remove old git diff tags * build windows management architecture, to avoid 'window already destroyed' errors * replace popupWindow with Windows manager interface * remove peer search timeout functionality and simplify splash screen logic * remove now-redundant meteor package * fix bugs and issues for electron 1.0 support * got tx sending working again * rename IPC message types * added i18n support to networkIndicator element, added better highlighting * dont auto-exit if startup fails - show logging info instead * fix bug preventing distributable build * fixes for electron 1.0, get wallet distributable working * remove old redundant code * Check for node startup error (#673) * Remove old git diff tags * check for geth fatal startup error * more helpful error message if geth unable to bind to port * UI Changes to network indicator (#675) * untracked public/i18n (#692) * small typo: some_where (#693) * Remove old, redundant code (#674) * Remove old git diff tags * remove old redundant code * Integrate #601 along with some improvements to IPC Path * Remove old git diff tags * integrate #601 * check for geth fatal startup error * remove old redundant code * UI Changes to network indicator * merge * updates nodes to 1.4.5 * update the menu string * Pass custom CLI options to the Ethereum node (#689) * integrate #601 * better cli arg parsingj * better cli arg parsingj * Simplify cliArgs API. * update readme * fix bug * Nav bar glitch and new art for Mist (#696) * Solves navbar glitch and adds new icon and splash screen art * adds new splash screen art for Mist * added a yellow tint to the image * added node linux 32 (#711) * added unlock and send (#700) * Check for update at startup and inform the user if a new version is available (#707) * refactor globals setup by preloader * final version of update checker mechanism * if in mist mode search for a mist update * replace image ids with better system * Remove dependency on node log output for detecting startup success (#706) * remove dependency on node log output for detecting startup success * only start the startup timeout once some data has been received * Norwegian Bokmål translation (#705) * bump version to 0.7.5 * Upgrade to web3 version in order to fix broken Send tx funtionality * update web3 to npm version * Suggestions and misspelling (#753) A few suggestions for a slightly less informal translation. * fixed preloader scripts * changed file permissions back * changed file permissions back * Fix Mist/Wallet app bundling (#721) * fix Mist/Wallet app bundling * update electron-packager to 7.0.2 * Minor changes (#769) Formal used also in the error message. * Full translation in italian (#768) File has been translated in Italian. * start ci * trigger build * only build on node 4g * fix deps * test develop branch * added alex and fabian to notification list * corrected %appdata% path in readme (#776) * Refactoring Dapp IPC connection (#724) * work towards new ipc backend wrapper * work towards refactoring ipc provider backend * work towards refactor * work towards refactoring request processing * work towards new ipc backend wrapper * work towards refactoring ipc provider backend * work towards refactor * work towards refactoring request processing * improved web3ipcsocket logic, work towards request/response processing in provider backend * split per-method handlers into separate files, dynamically loaded. - fixed web3 ipc socket response handlng - finalized sendRequest() initial draft * rename generic to base - fix bugs in use of socket * enforce timeout of 0 for sockets * work towards handling batch requests * remove old provider backend * fix batch result parsing * fix numerous bugs, improve how window id gets set * remove redundant error * have a connection per webview rather than per-window * defensive coding * adds default wallet hosted by github (#767) * adds default wallet hosted by github * change to ethereum.org page, adds id to avoid inserting multiple times * new icon * Refactor splash screen (#772) * Refactoring of the splash screen * center the checking network message * Show Start App button sooner * update geth to 1.4.6 + win32 support (#784) * update travis notification email addr * Fix problem with geth node being detached (#789) * fix node detaching, improve socket ipc backend connector * launch app link should show straight away * fix bug * Changes to onboarding screen (#786) * Fix bug that was preventing onboarding to start * allows account creation to be skipped and checks passwords * skip account page * Added number animation to blocknumbers * speed down to prevent hanging * removes log and adds comments to code * only math.floor the display block not the actual number * adds shapeshift reference * fix handling of array results * fix handling of results - ensure json result is always set * readme: beautify privatenet ipcpath * fix remaining issues * ensure error reponse doesn't include method and params * upgrade to electron 1.2.2 * Fix the testnet label alignment (#792) * fix the testnet label alignment * on the onboarding, just show states as percentage * fix web3.isConnected() issue * fixes issue where onboarding screen would not appear * fixed dragging * adds a drag region to the mist sidebar (#820) * download nodes * added download-nodes * remove unecessary packages * remove unecessary drags * added gulp update nodes to travis * Update menu items label * update eth to 1.2.9 (#825)

lock · 2018-04-01T12:23:22Z

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.

hiddentao added 6 commits May 4, 2016 16:15

remove commented-out code, cleanup

1d22378

work towards new socket system and ethereum nodes manager

dcc0bab

main: work towards finishing up node connection

fac0d41

refactored node start code

b38b14d

remove old ethereum nodes code

f25c83f

get basic IPC connection to geth working

311f901

alexvandesande added the Status: In progress label May 5, 2016

hiddentao added 10 commits May 5, 2016 23:18

rename socket to sockets

500d31c

- shift all node startup code into ethereumNode

09ad77c

- remove nodeConnector, shift its functionality into ethereumNode - redo checkNodeSync to be more robust

nodeSync needs to be able to emit events

21a3b82

fixed a number of bugs, and finished sync rewrite. now need to test

b137017

basic sync working

263b92c

slow down the sync check

8bc3e2f

check that onboarding followed by sync works for testnet

ed1a9ea

better log msgs, reduce socket disconnection timeout, fix bug

33d96cf

sync is much more resilient now and auto-restarts if node changes

6b389ba

ensure that switching network works once app is loaded

4553a0c

hiddentao mentioned this pull request May 10, 2016

Fix Mist startup crash due to about:blank page #548

Merged

hiddentao added 6 commits May 10, 2016 09:34

add node 4 dep

563db43

Merge branch 'develop' into node_startup_sync

e4cc994

# Conflicts: # README.md # modules/menuItems.js

Fix menu items code

9844dd6

improve tab resetting logic to happen before Mist crashes

423241f

fix how we detect whether eth has started up, remove nodeConnector

a91f44d

ensure can switch network type while syncing

eaff2b2

hiddentao added 4 commits May 11, 2016 14:53

- Make sure it works when using Geth 1.4 (tested the Darwin x64 versi…

7e4097c

…on). - We now show network info in top-right of both splash screen and onboarding window

Improved how we determine splash screen status messages. Now much mor…

9ee0f57

…e responsive to underlying connection process.

Remove old git diff tags

c6b66c6

Merge remote-tracking branch 'mist/develop' into develop

302f705

dont auto-exit if startup fails - show logging info instead

146bcfa

hiddentao mentioned this pull request May 17, 2016

Possibility to specify IPC path & RPC over HTTP support #420

Closed

hiddentao added 4 commits May 17, 2016 17:11

Merge remote-tracking branch 'mist/develop' into develop

58862cd

Merge branch 'develop' into node_startup_sync

df9d7e9

fix bug preventing distributable build

32ff38f

fixes for electron 1.0, get wallet distributable working

0904193

hiddentao mentioned this pull request May 17, 2016

Add a command line flag --ipcpath (close #600) #601

Closed

Merge remote-tracking branch 'mist/develop' into develop

21d3baa

hiddentao self-assigned this May 18, 2016

hiddentao added Status: Please review and removed Status: In progress Status: Please review labels May 18, 2016

hiddentao removed their assignment May 18, 2016

hiddentao added 2 commits May 19, 2016 08:03

Merge remote-tracking branch 'mist/develop' into develop

63a7cc3

Merge branch 'develop' into node_startup_sync

50a807f

alexvandesande merged commit af5b583 into ethereum:develop May 19, 2016

This was referenced May 20, 2016

Changes to the way TemplateVar is handled hiddentao/mist#1

Closed

Hiddentao node startup sync #672

Closed

luclu mentioned this pull request Aug 3, 2016

Add a command line flag to change the path of IPC socket #600

Closed

evertonfraga removed the Status: Please review label Nov 4, 2016

lock bot locked and limited conversation to collaborators Apr 1, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Node startup and synchronization improvements #559

Node startup and synchronization improvements #559

hiddentao commented May 4, 2016 •

edited

Loading

hiddentao commented May 10, 2016

hiddentao commented May 10, 2016

gsilvestri commented May 16, 2016

hiddentao commented May 16, 2016 •

edited

Loading

alexvandesande commented May 16, 2016

gsilvestri commented May 16, 2016

alexvandesande commented May 17, 2016

hiddentao commented May 20, 2016

lock bot commented Apr 1, 2018

Node startup and synchronization improvements #559

Node startup and synchronization improvements #559

Conversation

hiddentao commented May 4, 2016 • edited Loading

hiddentao commented May 10, 2016

hiddentao commented May 10, 2016

gsilvestri commented May 16, 2016

hiddentao commented May 16, 2016 • edited Loading

alexvandesande commented May 16, 2016

gsilvestri commented May 16, 2016

alexvandesande commented May 17, 2016

hiddentao commented May 20, 2016

lock bot commented Apr 1, 2018

hiddentao commented May 4, 2016 •

edited

Loading

hiddentao commented May 16, 2016 •

edited

Loading