Windows Binaries #5808
Comments
|
If you believe this:
then I don't see how you can believe this
could solve the problem. Surely Inc/Foundation would be blamed even more. The point is, we don't know whether a binary built for windows operates properly and don't want to represent that we do. Your second option is the same sort of variant. Your third option (Foundation builds) is a thing you can do, but just building isn't really a great idea. There could be bugs, and it doesn't sound like you are proposing that the Foundation take on the significant work of validating Windows builds. Just building and YOLOing. That sounds bad. In all cases, claiming "not officially supported" is not going to avoid the blame game if people use them. |
I was mainly talking about malicious actors, like the supply chain attack that happened to MyAlgo. That is a much bigger problem than bugs in the software, epsecially since we are telling people to use these third party binaries.
I would say that people will blame the Inc for bugs just the same for unsupported third-party binaries as unsupported first-party binaries. In both cases, the Inc's response is "Windows is unsupported". I don't see how a clearly marked unsupported binary changes that. |
I found the problem!
Perhaps I am naive, but I don't expect to get blamed if people run the software on a Commodore 64. Windows is a different platform, and if the "bug" is platform specific code differences (say the threading/network/process handling behavior is different, which seem to be the most common Windows differences) I actually don't expect blame. In my view "UNSUPPORTED" is an insufficiently strong caveat. It's not that we don't support you if you have a problem, it's that we're not even claiming it's correct. |
Yeah that's fair. Probably better for us to just tell people to build from source to de-risk from malicious actors. For bugs though I don't think unsupported binary vs unsupported source code is all that different.
Considering people believed things like Tinyman or MyAlgo hack happened because of network vulnerability, I fully believe that a negative headline about Algorand software will reflect poorly on Algorand regardless of whether it was official or not.
I think the connatation of "unsupported" in software is well understood to generally mean "use at your own risk" but there might be better wording. |
|
Here's something else to consider. As of right now, there have been 406 keys generated using Aust's One-Click Node. The stats collection I setup for this is rudimentary and doesn't distinguish between Algorand/Voi nor does it distinguish between OS. Also a generated key doesn't necessarily equate to someone participating. However, I'd say a good guess for how many of those are Algo/Windows users would be ~100. If you don't care about which chain, I'd guess ~200 people are using windows binaries. With that preface, I think the Foundation needs to start providing windows binaries, and here is why. Imagine that the next version has a protocol-level change and requires all nodes to update but a part of the code doesn't work on windows. All 200 of those current, non-technical node runners will suddenly be unable to participate in consensus. And when they ask me why I didn't update A1CN so they could participate, I'm just gonna pass the blame right down the line (no offense). And sure, the Foundation can say that they've never supported windows binaries, but I have a feeling that those node runners won't take that response very well. Speaking of bad headlines: "Algorand update prevents Windows users from participating in consensus." I understand that it would be a significant undertaking for the Foundation to support Windows binaries. And obviously I'm not privy to the financial details of the Foundation, but it does seem odd that the Foundation for a coin with a ~$800M market cap can't find the resources to support Windows binaries. Anyways, there's my thoughts on the subject. |
|
@AustP How many of those users are running Windows Server compared to a regular Windows build like Windows 10/11? My main concern with pushing out Windows binaries or an easy way to install on Windows is people who don't really know what they're doing and then turning their PC off every night causing online stake to be offline. Even if they're not turning off their computers every night, a regular desktop build of Windows is gonna have updates that cause random restarts |
I think the choice here could be to install WSL2 and run it there while still on Windows so does it actually make them unable to participate? Back to the topic - the repository here even has scripts and instructions on how to build on Windows and for me this isn't 100% clear I should not be running algod on Windows natively so I would either like to see the official binaries or no instructions suggesting (kind of) I just have to build it on my own for Windows. |
@Curtis-D This concern would be addressed by the work in #5757 |
This is a good point. Perhaps if liability is really the concern here and Inc + Foundation are unable to provide the binaries, then the |
@joe-p I'm not sure it would. Especially if they're turning their PC back on every day and spinning up the node. It's essentially going to be sending out a heartbeat which would stop them from being suspended (even more so if their stake is lower) |
I don't want to get to deep into this here because I think this is not a problem unique to Windows, thus the need for the heartbeat in the first place. But I think some probability of downtime and VRF overlap is acceptable. There also could be a supplemental mechanism to the node that monitors uptime and the heartbeat only gets sent out if the node has been up throughout its unlucky period. |
It's not unique to Windows but is going to happen significantly more on Windows than on a Linux machine that is more often than not running server grade OS. Anyway, I agree, probably more important stuff to discuss here |
|
@AustP Thank you so much for your work on A1CN. I'm responsible for about a dozen of those keys, most of them are actually participating. :) And thanks for the latest update that lets us choose the data directory! @Curtis-D I'm running AustP's binaries on Win 11 Pro. I also run BTC, XCH, and XMR nodes in Windows. Yea, there's downtime with automatic updates enabled. A few minutes once in a while, like every few months maybe. Most updates don't require reboot. If it does, the system reboots & the node self-starts. While I think it's safe to assume the "non-technical" (read WINDOWS) user doesn't know how to debug and compile code or know Linux doesn't mean that we can assume that they don't have at least some level of tech prowess and would probably at least let it run 24/7. Automatic updates/reboot can be disabled too, it doesn't take a developer to figure out how to turn them off. Anyway, gatekeeping who can run a node because some people don't want to learn a new OS or debug and compile code isn't helpful. If we want adoption, an idiot proof installer is what the foundation should be shooting for. Crypto for the masses, not just the highly tech literate. |
|
I maintain the .net sdk and the C# compiler for teal. I am also in touch with AlgoBharat, who have been promoting Algorand in India, where many of those they promote to are Windows users. My own experience and theirs trying to promote Algorand and tooling is that the majority of software engineers develop using Windows and irrespective of seniority a large proportiion of those are unfamiliar with WSL and Docker, and some even still run machines without virtualisation support. When trying to introduce decision makers and senior developers to Algorand for development, all it takes is a 5 minute obstacle before the pressures of daily work cause them to turn their attention elsewhere. It seems to me Algorand not providing Windows installers, , at least for development and sandbox , is self sabotage on a grand scale. What we need to help drive adoption through experimentation with Algorand as an artifact in software designs is a quick, one click install user experience, with corresponding docs and guidance on the developer portal that we can refer to when developing our own educational materials. |
|
Something like Dappflow's one click node manager is a far better choice for non technical users than windows binaries. Instead of requiring non technical users to run CLI commands, everything is nicely displayed and participation can be done with a click of a button |
What binaries does this run from? (i.e. how do I get the node manager to make my node run on Windows?) |
This video shows how to generate a participation key, but not how to set up a node. When I go to Dappflow, the Node Manager is different- there's no setup or log-in option. It goes straight to the dashboard. Possibly because my wallet is already connected to use the App and Asset managers. A node is connected, but it isn't mine. I try to add my node but get an "invalid algod configuration" error. It's probably the wrong port, but I don't have the bandwidth to run it to ground. I just need this stuff to work without spending a weekend learning all the details of Linux to make it work. So, tell me, how do I get a node started in Windows with Dappflow? Are there some easy-to-follow instructions somewhere that can have me up in under an hour? Seems more technical than @AustP's 1CN, which doesn't require any CLI commands: Download file, install, run, and done. Lastly, let's stop calling Windows users "non-technical". Non-technical people tend to be Windows users, but not all Windows users are non-technical people. Linux illiterate =/= nontechnical. |
|
To add to the above. As far as I know, the majority of software developers use Windows in the world, and probably more so in commercial/government software development settings (as opposed to academia, hobby and speculative crypto development). I and others want to help pursue a strategy where we use friendly, accessible tooling to raise awareness of Algorand as a potential component in software design. Frameworks like mine (the C# stuff) , Beaker and so on, attempt to wrap Algorand into developer friendly concepts, such as client proxies to call smart contracts, compiled smart contracts, helper functions to deploy and fund contracts, etc. Software development often involves technologies being selected not based on the merits of the technology entirely, but on other factors: familiarity with the tech in the team, managers wanting to offer crypto/blockchain as a word to add to CVs, software architects being the focus of sales pitches, and other reasons. The Windows users I have spoken to are for example lead developers/software architects/CTOs in sectors like energy and healthcare. In these settings, is quite normal for software development to go on in a development environment, where the production deployment (whether things are hosted in docker , natively, in azure, on aws, IaaS, serverless etc) is managed by an entirely different team. It generally does not follow that because someone is unfamiliar with or does not have available things like Docker, Linux or WSL that they should be considered non-technical. Some people I speak to know everything there is to know about Web APIs and SQL server but never used Docker in their lives. The other aspect I would like to raise is that the "node" API has a dual purpose: 1) to act as a CRUD interface for Algorand entities (accounts, payments etc) and 2) as a developer interface (compile contracts, debug them). The node is the gateway to the Algorand world for whichever set of use cases you happen to be interested in. The message being presented to these sorts of people is that Algorand is a distributed computer (an "operating system") and that participation in that computer requires the use of a node. Beyond that, tools are given to them that are similar to those used developing web apis, except that the API is a set of smart contract methods, and the server is that distributed computer. What we need to do is make it super simple for the majority of the developers in the world to get up and running, whether they are using my tooling, the upcoming Python tooling in AlgoKit, Beaker, TealScript or any other framework, and step 1 in all of those processes should be "install [a developer] Algorand node". I am not entirely sure what the reasoning is for not supporting Windows builds....is that Algorand does not have the resources to run end to end tests on Windows binaries? Is it that a comprehensive test suite is missing? |
|
How about an official Windows build with participation turned off? Does this make things easier for Inc. while still satisfying the Windows developers? |
The protocol team (we are at what is called "the Inc" today) has a few challenges with supporting this near term:
I consider the concerns around people shutting off their computers at night entirely legitimate - the heartbeat mechanism mentioned earlier is meant to help minimize risk here. |
I find that not having people_ available to deliver Algorand to the most common developer platform in the world entirely preprosterous. As I mentioned above, self sabotage on a grand scale. However, my outrage doesn't really help. What can I or we, the community, do to help you shift the priorities to where they should be? (Or should AF do it?) |
|
We'll take a further look in terms of support needs, what I laid out above has been our rationale to date. In practice, how many flavors of Windows in your mind need to be supported/individually tested against? We have a Github runner building Windows now that we intentionally were not running tests on given we were not ready to support actively. I will say that expressing outrage is not helpful in these threads and frankly is not motivating for developers working to support the protocol (or many of the people like yourself building on top of it). |
|
For those who see this and want to contribute very near term; to get test builds properly working, we need both |
Pretty much everyone I have spoken to on the topic expresses at least incredulity, so perhaps it's best for all you get it served up straight. I do think it's helpful to be pretty honest with one's opinions. |
When was this and why were you unable to hire the right people to do it? |
|
Frank, this back and forth isn't helping at this point so I'm not going to engage further. We'll update the community as support levels evolve for Windows builds. |
I'm not seeing outrage here, just incredulity and disappointment. I'm really sorry that you're feeling attacked; that is not the purpose or intent here. Every single person commenting wants to see your work succeed. Food for thought.
It's your opinion that it isn't helping. It's helped me to understand the reasoning behind lack of Windows support. Disengaging from the conversation not only doesn't help, but it also actively damages community relations. Frankly this is not motivating for the uncompensated community supporting the protocol to continue doing so with their own time and money.
Right? Like if you hand me a stack of punch cards and ask for a program in Fortran, I could give it to you. Need hardware diagnostic/repair on something obscure? I'm your guy. Ask me about Docker or any modern programming language see my face glaze over. Outside the cryptosphere I have 0 use for Linux. I want to support this ecosystem beyond just HODLing ALGOs, but I'm not spending every weekend for a month learning the ins and outs of WSL and Docker to make it happen. |
It is helping. And you shouldnt be ducking out like this. I want to know who or what is at fault so it can be rectified. Do you need more budget? Is your internal budget manager incompetent? Persuaded by other goals? Where is the problem. As our colleague points out above, DON'T feel attacked. This is about getting at the root of the problem. |
I need you to speak for me in future lol. Couldnt have said it better. |
Lol. You can speak for me too, there have been some really valid points you've made. |
|
Added these patches in order to be able to compile the node on Windows: On RandLabs we created time ago a service and the installer here: https://github.com/randlabs/algorand-windows-node. |
|
Update: Managed to compile downloading Go 1.20.2 package for MSYS2. By default, MSYS2 does not allow to specify package versions and other versions of Go can generate issues. |
|
Any variations across platforms in the sortition library would present a fork risk - something we have to be very cautious of in any type of Window-support endeavor. |
|
I for one support a single client and platform for now. Too many risks involved ,see Ethereum and their multi client problems. We should optimize what we have now and any dev worth their salt can pick up docker quickly and shouldn't be a show stopper. If you are not adapting and growing what are you doing. I run my node on windows btw on wsl and it works just fine. A third party can go build the windows version |
|
Pao, if you have changes that are portable/can help get the Windows test
builds passing, we'd happily take a look at them. We do worry about subtle
differences in say the sortition package that could put the protocol at
risk of a fork.
…On Tue, Oct 31, 2023 at 9:32 AM pao ***@***.***> wrote:
I for one support a single client and platform for now. Too many risks
involved ,see Ethereum and their multi client problems. We should optimize
what we have now and any dev worth their salt can pick up docker quickly
and shouldn't be a show stopper. If you are not adapting and growing what
are you doing. I run my node on windows btw on wsl and it works just fine.
A third party can go build the windows version
—
Reply to this email directly, view it on GitHub
<#5808 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHP3U5ANJ3XKYPEGOT2KDLYCD4WHAVCNFSM6AAAAAA6RSD7ICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBXGIZDMOBZG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I don't think you are listening. It is and has been a showstopper, and at least a headache. "Any developer worth their salt" includes plenty of people who are worth so much salt they aren't interested in installing Docker/WSL/etc in order to test out a concept. Ethereum has an entirely different situation. It aims for "decentralisation" by ensuring the Yellow Paper and other protocol specs define the client. It has multiple implementations by multiple vendors, irrespective of platform. Algorand has one source base and one client and multiple builds. |
|
Having one client doesn't make it centralized if that's what you're implying. Anyone can create a pull request and contribute to the codebase |
I am not implying that at all. I am just saying that your suggestion that Ethereum has issues because of multiple clients is incomparable to Algorand having multiple builds. |
Again, agreed. We're not all software developers here. I'm an end-user, an individual with an interest in providing infrastructure for cryptocurrency networks I use. I have a vested interest in providing maximum uptime and participation that contributes to security and reliability. I run nodes for several other chains, all in Windows. I have good hardware that far exceeds the minimum requirements for each of them, and keep it well maintained. I have plenty of low latency bandwidth and practically unlimited data available. It's been zero issues for me as an end-user for all those other chains. Why does support for Windows have to be such a controversial topic for Algorand? I just don't get it. |
Status
There is no way to get Windows binaries from Algorand Inc or Foundation. If someone comes to use asking for Windows binaries, we point them to third parties or tell them to build from source.
Problems for Developers
Right now docker is required for building on Algorand because of the need to run a localnet. This adds a significant amount of friction to the Algorand developer experience because some people might not already have docker installed. This is problematic particularly on Windows because it requires WSL to be installed as well. This has resulted in many hours of devrel time debugging Microsoft's software that we have no control over. It's also a non-trivial download for those in areas with poor internet connectivity.
On MacOS, docker is also not a light process because it requires a VM to be running the background. I personally have had issues with Docker daemon on Mac and I'm sure I'm not the only one. Most of the time a simple force quite solves the problem, but still annoying.
Problems for Node Runners
Right now there are multiple third parties distributing binaries and node runners have to choose one to trust. Not only do they have to trust that there aren't any malicious code injections, but they also have to trust that the third party will continue to release binaries as there are new consensus updates. This is even more important once we get mining incentives.
Problems for Algorand
Because we are forcing people to use third party binaries, if problems were to ever arise with one of these binaries Algorand as a network and likely Inc/Foundation as entities would get blamed. "Algorand node software compromised with malware" is not a good headline for the perception of the network, regardless of who is actually responsible. While this situation may seem unlikely, take a look at MyAlgo and how many people still believe that this was a problem with Algorand and/or the Foundation.
Expected
There should be a way, even if not officially supported, for developers and node runners to acquire
go-algorandbinaries without having to go through a third party. This offers benefits for two separate groups of people.Solution
I see a couple of solutions as a reasonable path forward:
Include unsupported Windows binaries in
go-algorandreleasesFor every software release, a
UNSUPPORTED_node_stable_win-amd64_3.19.0.tar.gztarball could be releasedThis to me is the most preferable option.
Include unsupported Windows binaries in non-stable builds
In non-stable (or perhaps only nightly) builds include Windows binaries, but not in stable releases. This, however, increases the likelihood of someone trying to run future releases on mainnet.
Foundation builds Windows binaries
If the Inc does not want to release binaries, the Foundation could have a seperate pipeline specifically for Windows builds that are not officially supported.
Dependencies
Requires modification to the release workflows of
go-algorandor build server infra for the FoundationUrgency
Medium? See aforementioned problems.
The text was updated successfully, but these errors were encountered: