Adding powershell start scripts for windows, fix nodetool ping output #796

reganheath · 2020-05-15T01:03:18Z

I have created powershell versions of the bin_windows and extended_bin_windows start scripts called bin_windows_ps and extended_bin_windows_ps plus included a psutil utility script.

To enable use {start_script_type, powershell} in rebar.config relx section. Works with {extended_start_script, true} and {extended_start_script, false}

Resolves #657 by doing env replacement on .src and handling RELX_REPLACE_OS_VARS.

Resolves #470 by NOT writing erl.ini and instead setting the ERL_LIBS environment variable.

The extension support in extended_bin_windows was not ideal, these implement it fully (as in extended_bin).

I have also fixed "nodetool ping ..etc.." which was performing the ping correctly, but then outputting:

Other: ["ping"]
Usage: nodetool {ping|stop|restart|reboot|rpc|rpcterms|eval [Terms]} [RPC]

as if it didn't understand the ping command.

I have NOT added any common tests for this. 3 tests appear to be failing, but seem unrelated to my changes - rather I suspect they haven't been run on windows in a while?

I also have not built a release from this and worked out how to include it in rebar3 for a full test.

I have tested the powershell scripts by including them in a release, however I have no idea how to use upgrade/downgrade (relup) so I have not tested it but simply implemented what was already in the existing scripts.

Note, this was based on the 4.0.0 branch.

tsloughter

This is great, thank you! I had a couple questions and possible changes.

And preferably the github actions windows tests could be added on to actually build a release with the powershell script and start it, ping it and stop it. Similar to what we now do in shelltests/.

tsloughter · 2020-05-18T15:00:51Z

priv/templates/extended_bin_windows_ps

+        'Invoke' = 'Uninstall';
+        'PassArgs' = $true
+    };
+    'start' = @{


To match the changes to the other script I think this should now be daemon. Unless that makes less sense on Windows?

The equivalent of a "daemon" on windows is a "windows service":
https://en.wikipedia.org/wiki/Daemon_(computing)#Terminology

I have added 'daemon' as an alias for 'start'.

tsloughter · 2020-05-18T15:02:55Z

priv/templates/nodetool

@@ -86,6 +86,22 @@ main(Args) ->
                    io:format("RPC to ~p failed: ~p~n", [TargetNode, Reason]),
                    halt(1)
            end;
+        ["ping"] ->


Why did you add these to nodetool?

I have also fixed "nodetool ping ..etc.." which was performing the ping correctly, but then outputting:

Other: ["ping"] Usage: nodetool {ping|stop|restart|reboot|rpc|rpcterms|eval [Terms]} [RPC

That shouldn't be the case, something else is wrong if you are seeing the usage. The extended start script uses nodetool rpc to do all of the commands, so usage should not be printed because commands like nodetool ping should never be called.

The current extended_bin_windows is calling nodetool ping:
https://github.com/erlware/relx/blob/4.0.0/priv/templates/extended_bin_windows#L323

I can update these to use rpc instead if you would like?
But.. surely "nodetool ping" should be useable too?

Ooh, woops. I thought I did updates to the windows script but I must have missed some stuff. You can see in the extended_bin script that those commands have been replaced by just rpc because on OTP-23+ we'll be using erl_call instead of nodetool and eventually nodetool will be removed https://github.com/erlware/relx/blob/4.0.0/priv/templates/extended_bin#L240

Ok, no more nodetool ping, only rpc - added eval support too

src/rlx_state.erl

reganheath · 2020-05-18T15:20:35Z

I will have a look at shelltests/ and see what I can add :)

…mmand aliases

reganheath · 2020-05-19T01:24:54Z

I will have a look at shelltests/ and see what I can add :)

This looks complicated. I think it's going to require a new github workflow, wintests folder, powershell script and so on, right?

tsloughter · 2020-05-20T15:54:20Z

@reganheath I was thinking just a github action that ran under windows, built a powershell release (simply add an example release to the relx repo, could even be under shelltests/ with the other examples but just wouldn't have a .test file so shelltest would ignore it) and started it.

We can worry about that later if its too much, I wasn't sure how github action windows works anyway and if it gives you powershell by default and lets you just run commands like that or not.

reganheath · 2020-05-21T07:43:28Z

Well.. that was frustrating!

I have implemented a test. I had to implement my own erlang installation process - as the existing action only supports linux (I may contribute my implementation to that project later).

The test even found a few bugs, now fixed, and it even passes with OTP 20 and 22, however..

With OTP 19 - relx fails to build:
This looks like an actual bug and I am kinda surprised it's not happening in other versions?
Run: https://github.com/erlware/relx/runs/694952533?check_suite_focus=true
Error:

===> Compiling _checkouts/relx/src/rlx_relup.erl failed
_checkouts/relx/src/rlx_relup.erl:27: this clause cannot match because a previous clause at line 22 always matches

With OTP 21 - nodetool fails to find epmd.exe
os:find_executable simply fails (depite all my efforts to place it in the system PATH - not just local environment). This is especially frustrating as this is the version I have installed, and it has no issue without epmd.exe being in the PATH.
Run:
https://github.com/erlware/relx/runs/694977835?check_suite_focus=true
https://github.com/erlware/relx/runs/694991175?check_suite_focus=true

Debug Output / Error:

*** DEBUG PATH = D:\a\relx\relx\shelltests\erl\bin;D:\a\relx\relx\shelltests\erl\erts-10.0\bin;C:\Program Files\PowerShell\7; ..etc..

*** DEBUG nodetool d:\a\relx\relx\shelltests\erl\erts-10.0\bin\escript.exe D:\a\relx\relx\shelltests\powershell_release\_build\default\rel\powershell_release\bin\nodetool -sname powershell_release -setcookie powershell_release_cookie rpc erlang is_alive

Could not find epmd.

With OTP 23 - I have a bug running erlcall to continue debugging
Run: https://github.com/erlware/relx/runs/695034370?check_suite_focus=true
Output:

*** Ping service
Illegal argument 'erlang'.

ferd · 2020-05-21T11:51:25Z

Dang that's dedication, very appreciated. Nice find on the bug.
Re: the erlcall bug, be aware that @tsloughter found out issues with the OTP stuff in OTP 23 and has recently changed the 4.0.0 branch accordingly: #798 -- don't know if that could explain the erlcall issue.

tsloughter · 2020-05-21T14:06:39Z

Really great!

My first guess on the erl_call failure because it says erlang is illegal is bad quoting? For ping it'll run erl_call ... -a erlang is_alive.

tsloughter · 2020-05-21T14:12:13Z

In the rebar3 github actions I just use choco install erlang I see you are doing something more specific in your install script. Is that necessary or would choco be good enough?

tsloughter · 2020-05-21T14:14:46Z

Whoa, that build failure is definitely a real issue:

format_error({no_upfrom_release_found, Vsn})->
    io_lib:format("No previous version of release found for building relup to version ~s", [Vsn]);
format_error({no_upfrom_release_found, undefined}) ->
    io_lib:format("No earlier release for relup found", []);
format_error({no_upfrom_release_found, Vsn}) ->
    io_lib:format("Upfrom release version (~s) for relup not found", [Vsn]);

I have no idea how this was only caught by otp-19 on windows...

tsloughter · 2020-05-21T14:15:34Z

@reganheath do you mind just ripping out that first clause, as in remove these two lines in your PR:

format_error({no_upfrom_release_found, Vsn})->
    io_lib:format("No previous version of release found for building relup to version ~s", [Vsn]);

reganheath · 2020-05-21T22:01:06Z

In the rebar3 github actions I just use choco install erlang I see you are doing something more specific in your install script. Is that necessary or would choco be good enough?

I've never used it - I'll give it a go.

reganheath · 2020-05-21T22:01:18Z

@reganheath do you mind just ripping out that first clause, as in remove these two lines in your PR:
format_error({no_upfrom_release_found, Vsn})->
    io_lib:format("No previous version of release found for building relup to version ~s", [Vsn]);

Sure.

reganheath · 2020-05-21T22:01:50Z

I have no idea how this was only caught by otp-19 on windows...

OTP bug introduced in 20 perhaps?

reganheath · 2020-05-21T22:26:26Z

In the rebar3 github actions I just use choco install erlang I see you are doing something more specific in your install script. Is that necessary or would choco be good enough?

I think this will install Erlang 22.3 and not OTP 19, 20, 21, etc as required by the matrix.

list -> string Co-authored-by: Tristan Sloughter <[email protected]>

reganheath · 2020-05-22T00:52:29Z

Turns out the OTP 21 issue is a regression in that version, see:

So, we cannot test on this version

reganheath · 2020-05-22T00:59:51Z

Really great!

My first guess on the erl_call failure because it says erlang is illegal is bad quoting? For ping it'll run erl_call ... -a erlang is_alive.

You were correct :)

reganheath · 2020-05-22T01:02:36Z

Question see wintest.yml, what do we want to do in this MR for the install_erlang step. I have provided 3 options, 2 of which will work now and the 3rd being the correct version once my MR for gleam-lang/setup-erlang is accepted.

Update my pull request has been merged to gleam-lang/setup-erlang so our path is clear :)

Update to merged gleam-lang/[email protected]

No longer need local erlang install script

tsloughter · 2020-05-23T02:01:52Z

@reganheath awesome work! Is this ready to merge now?

ferd · 2020-05-23T02:15:45Z

Yeah that's really awesome work. It's good to see that part of the code get some real love instead of bare-minimum maintenance :)

reganheath · 2020-05-23T15:34:43Z

@reganheath awesome work! Is this ready to merge now?

Yeah that's really awesome work. It's good to see that part of the code get some real love instead of bare-minimum maintenance :)

Awesome, thanks guys!

Adding powershell start scripts for windows, fix nodetool ping output

042cd82

reganheath force-pushed the powershell_scripts branch from b60b95d to 042cd82 Compare May 15, 2020 01:09

Add missing config load

2eb1915

tsloughter requested changes May 18, 2020

View reviewed changes

Add erl_call support, use nodetool rpc not ping, improve help, add co…

1092e64

…mmand aliases

reganheath force-pushed the powershell_scripts branch from 95b7667 to 6738715 Compare May 21, 2020 00:36

reganheath and others added 8 commits May 22, 2020 01:11

Update src/rlx_state.erl

03adb17

list -> string Co-authored-by: Tristan Sloughter <[email protected]>

Fix start script extensions

9b66d72

Attempt to implement windows workflow

3e6af08

Manually install erlang

ed93ff9

Pass erlpath from install to test

d88534d

Fix OTP 19 compile error

c64253d

Try choco

87f5102

Fix quoting issues, install erl to default

b46d0ca

reganheath force-pushed the powershell_scripts branch 2 times, most recently from 46db569 to 769f427 Compare May 22, 2020 00:22

test setup-erlang windows support

86eaff0

reganheath force-pushed the powershell_scripts branch from 0723989 to 33d46f8 Compare May 22, 2020 00:50

OTP 21 does not work on windows, cannot test

a6803e1

reganheath force-pushed the powershell_scripts branch from 33d46f8 to a6803e1 Compare May 22, 2020 00:54

reganheath added 2 commits May 22, 2020 16:51

Update wintest.yml

3b8a759

Update to merged gleam-lang/[email protected]

Delete win-install-erl.ps1

819b9f9

No longer need local erlang install script

ferd approved these changes May 23, 2020

View reviewed changes

tsloughter approved these changes May 23, 2020

View reviewed changes

tsloughter merged commit 3ebc5ff into erlware:4.0.0 May 23, 2020

reganheath deleted the powershell_scripts branch May 23, 2020 15:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding powershell start scripts for windows, fix nodetool ping output #796

Adding powershell start scripts for windows, fix nodetool ping output #796

reganheath commented May 15, 2020

tsloughter left a comment

tsloughter May 18, 2020

reganheath May 18, 2020

reganheath May 19, 2020

tsloughter May 18, 2020

reganheath May 18, 2020

tsloughter May 18, 2020

reganheath May 18, 2020

tsloughter May 18, 2020

reganheath May 19, 2020

reganheath commented May 18, 2020

reganheath commented May 19, 2020

tsloughter commented May 20, 2020

reganheath commented May 21, 2020

ferd commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 22, 2020

reganheath commented May 22, 2020

reganheath commented May 22, 2020 •

edited

Loading

tsloughter commented May 23, 2020

ferd commented May 23, 2020

reganheath commented May 23, 2020

Adding powershell start scripts for windows, fix nodetool ping output #796

Adding powershell start scripts for windows, fix nodetool ping output #796

Conversation

reganheath commented May 15, 2020

tsloughter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reganheath commented May 18, 2020

reganheath commented May 19, 2020

tsloughter commented May 20, 2020

reganheath commented May 21, 2020

ferd commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

tsloughter commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 21, 2020

reganheath commented May 22, 2020

reganheath commented May 22, 2020

reganheath commented May 22, 2020 • edited Loading

tsloughter commented May 23, 2020

ferd commented May 23, 2020

reganheath commented May 23, 2020

reganheath commented May 22, 2020 •

edited

Loading