Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postinstall script hangs when installing with pnpm #28148

Open
1 of 4 tasks
keriati opened this issue Sep 26, 2024 · 24 comments
Open
1 of 4 tasks

Postinstall script hangs when installing with pnpm #28148

keriati opened this issue Sep 26, 2024 · 24 comments
Assignees
Labels
scope: core core nx functionality type: bug

Comments

@keriati
Copy link

keriati commented Sep 26, 2024

Current Behavior

Postinstall script hangs when installing with pnpm on MacOS.

Expected Behavior

Postinstall script finishes.

GitHub Repo

Every repository using nx, no specific repository is needed.

Steps to Reproduce

Update 2024-11-17 on how to reproduce:

  1. Add a global .gitignore file in the user's home directory
  2. run pnpm i with nx in package.json
  3. See installation hanging

Nx Report

nx report seems not to work

Failure Logs

❯ NX_VERBOSE_LOGGING=true NX_PERF_LOGGING=true NX_CACHE_PROJECT_GRAPH=false NX_NATIVE_LOGGING=nx node node_modules/nx/bin/post-install.js
TRACE nx::native::workspace::context: workspace_root="/Users/<username>/Projects/<project>"
TRACE nx::native::workspace::context: Initially locking files
TRACE nx::native::workspace::context: Gathering files in /Users/<username>/Projects/<project>
TRACE nx::native::glob: converted globs globs=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"] result=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"]
TRACE nx::native::walker: walking directory="/Users/<username>/Projects/<project>"
TRACE nx::native::walker: walked in 1.607167ms
TRACE nx::native::workspace::files_hashing: Found 0 files
TRACE nx::native::workspace::files_hashing: hashing workspace files in parallel
TRACE nx::native::workspace::files_hashing: hashed workspace files in 131.916µs
TRACE nx::native::workspace::context: hashed and sorted files in 1.923416ms
TRACE nx::native::workspace::files_archive: could not write files archive: No such file or directory (os error 2)
TRACE nx::native::workspace::context: files retrieved files_len=0

Package Manager Version

pnpm 9.3.0

Operating System

  • macOS
  • Linux
  • Windows
  • Other (Please specify)

Additional Information

os: MacOS 15.0 and MacOS 14
node: v20.17.0
pnpm 9.3.0
nx: 19.8.0

Did try node 22, few pnpm versions, always the same result.

Exact same issue is described in this closed ticket: #27056

@keriati keriati changed the title Hangs when pnpm install at postinstall on MacOS Postinstall script hangs when installing with pnpm on MacOS. Sep 26, 2024
@gitLinda
Copy link

I have the exact same problem on windows.
OS: Windows 11 Pro
node: 18.18.2
pnpm: 8.10.0
nx: 19.8.0

@keriati
Copy link
Author

keriati commented Sep 27, 2024

Based on comments from previous ticket, i can reproduce the error with this command:

NX_VERBOSE_LOGGING=true NX_PERF_LOGGING=true NX_CACHE_PROJECT_GRAPH=false NX_NATIVE_LOGGING=nx node node_modules/nx/bin/post-install.js

I also get this output:

TRACE nx::native::workspace::context: workspace_root="/Users/<user>/Projects/<project>"
TRACE nx::native::workspace::context: Initially locking files
TRACE nx::native::workspace::files_archive: read archive in 411.459µs
TRACE nx::native::workspace::context: Gathering files in /Users/<user>/Projects/<project>
TRACE nx::native::glob: converted globs globs=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"] result=["**/node_modules", "**/.git", "**/.nx/cache", "**/.nx/workspace-data", "**/.yarn/cache"]
TRACE nx::native::walker: walking directory="/Users/<user>/Projects/<project>"
TRACE nx::native::walker: walked in 2.835792ms
TRACE nx::native::workspace::files_hashing: filtered archive files in 667ns
TRACE nx::native::workspace::files_hashing: no additional files to hash
TRACE nx::native::workspace::context: hashed and sorted files in 2.993833ms
TRACE nx::native::workspace::files_archive: write archive in 263.834µs
TRACE nx::native::workspace::context: files retrieved files_len=0
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 0.29895799999999895
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/package-json' 0.3219169999999991
TRACE nx::native::workspace::context: waiting for files to be available
Time for 'Load Nx Plugin: /Users/<user>/Projects/<project>/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_smkezf5ssdaier6uufeirwp64i/node_modules/nx/src/plugins/js' 9.766417000000004

@keriati keriati changed the title Postinstall script hangs when installing with pnpm on MacOS. Postinstall script hangs when installing with pnpm Sep 27, 2024
@keriati
Copy link
Author

keriati commented Sep 27, 2024

Compared to machines where this works, my instance is getting stuck on this line:

TRACE nx::native::workspace::context: waiting for files to be available

while on other machines this is instantly followed by:

TRACE nx::native::workspace::context: files are available

This above line never shows up for me, just keeps waiting forever...

@lennybakkalian
Copy link

lennybakkalian commented Oct 3, 2024

I found a temporary workaround for me:

killall -9 node
rm -r node_modules
and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

@keriati
Copy link
Author

keriati commented Oct 3, 2024

I found a temporary workaround for me:

killall -9 node rm -r node_modules and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

Thanks, this worked for me too for the installation step!

Also running nx now works as long as the project stays in /tmp folder.

@jaysoo jaysoo added the scope: core core nx functionality label Oct 14, 2024
@keriati
Copy link
Author

keriati commented Oct 18, 2024

@Cammisuli If you need help to check and try to reproduce this issue, just contact me. (CET)

@alex-ruehe
Copy link

I just hit the same issue, but using yarn. Stuck at waiting for files to be available and but it works if I move my project to /tmp

  • OS: macOS 14.5
  • node 20.18
  • yarn 1.22.22
  • nx 20.0.10

ratson added a commit to admob-plus/admob-plus that referenced this issue Nov 14, 2024
@alex-ruehe
Copy link

@keriati any chance you have a global .gitignore? I just did hours of debugging and noticed some debug logs that mentioned my global .gitignore file. After temporarily removing it, everything worked fine.

Not sure what the reason is, since I have zero experience with node and was just trying to get an internal application up and running when I hit the issue.

@Cammisuli maybe this information is helpful?

@keriati
Copy link
Author

keriati commented Nov 16, 2024

@alex-ruehe it seems when I remove my .gitignore file from my home directory (that I use for versioning my dot files), it starts to work normally again.

Maybe this is also somehow connected to the fact that moving the project outside of the home directory makes it also work normally.

I would actually say that the .gitignore in the home directory seems to cause this problem.

@keriati
Copy link
Author

keriati commented Nov 18, 2024

We tested it also on another machine, and it really seems that the .gitignore file in the user's home directory is causing this issue. We managed to reproduce this issue by adding a .gitignore file in the home directory.

Steps to reproduce:

  • Add .gitignore file in home (~) with the content *. echo "*" > ~/.gitignore
  • Run your install script pnpm i
  • Observe that the process is stuck

Workaround:

  • Remove .gitignore from home directory rm ~/.gitignore
  • Run sudo killall -9 node to stop the stuck processes (or reboot)
  • Run your command again (pnpm i)

@awakia
Copy link
Contributor

awakia commented Nov 23, 2024

Thank you for the information. I also have this problem and works fine when I remove .gitignore in home directory

Other solution was

   "pnpm": {
     "overrides": {
       "nx": "19.4.4",

so, I think the cause is introduced right after 19.4.4.
Please fix if possible.

@yduman
Copy link

yduman commented Nov 26, 2024

This happens also, when installing in a Docker environment, where there is no .gitignore present. It's hanging for a couple of minutes until the error below is then displayed, then it continues.

OS: macOS 15.1.1
Node.js: 22.11.0
pnpm: 9.14.2
nx: 20.1.3

These are the postinstall logs from pnpm when executing a docker build
The error is Error: Attempted to open socket that exceeds the maximum socket length.

.../node_modules/nx postinstall$ node ./bin/post-install
.../node_modules/nx postinstall:  NX   Failed to start or connect to the Nx Daemon process.
.../node_modules/nx postinstall: Messages from the log:
.../node_modules/nx postinstall: /app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:38
.../node_modules/nx postinstall:         throw new Error([
.../node_modules/nx postinstall:         ^
.../node_modules/nx postinstall: Error: Attempted to open socket that exceeds the maximum socket length.
.../node_modules/nx postinstall: Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue.
.../node_modules/nx postinstall:     at assertValidSocketPath (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:38:15)
.../node_modules/nx postinstall:     at getFullOsSocketPath (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/socket-utils.js:20:5)
.../node_modules/nx postinstall:     at Object.<anonymous> (/app/node_modules/.pnpm/[email protected]_@[email protected]_@[email protected]_@[email protected]__@[email protected]_ty_rvghjyke2ityvwaziobvzpkeaq/node_modules/nx/src/daemon/server/watcher.js:16:44)
.../node_modules/nx postinstall:     at Module._compile (node:internal/modules/cjs/loader:1546:14)
.../node_modules/nx postinstall:     at Object..js (node:internal/modules/cjs/loader:1689:10)
.../node_modules/nx postinstall:     at Module.load (node:internal/modules/cjs/loader:1318:32)
.../node_modules/nx postinstall:     at Function._load (node:internal/modules/cjs/loader:1128:12)
.../node_modules/nx postinstall:     at TracingChannel.traceSync (node:diagnostics_channel:315:14)
.../node_modules/nx postinstall:     at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
.../node_modules/nx postinstall:     at Module.require (node:internal/modules/cjs/loader:1340:12)
.../node_modules/nx postinstall: Node.js v22.11.0
.../node_modules/nx postinstall: More information: /app/.nx/workspace-data/d/daemon.log
.../node_modules/nx postinstall: Done

This is the Dockerfile im am running

FROM node:22-alpine
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
ENV HUSKY=0
ENV COREPACK_ENABLE_AUTO_PIN=0
RUN corepack enable

WORKDIR /app
COPY . .
RUN pnpm install --filter=@scope/service... --frozen-lockfile
EXPOSE 3000
CMD ["pnpm", "nx", "run", "@scope/service:serve"]

Edit

Since the error message mentions to Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue. I logged first to what value it is set. If I didn't make any mistakes, this value is apparently empty

Step 10/12 : RUN echo $NX_SOCKET_DIR
 ---> Running in d8ef0f4cc359

 ---> Removed intermediate container d8ef0f4cc359

So I added the value that was recommended from the error message

ENV NX_SOCKET_DIR=/tmp/nx-tmp

The hang stopped and the build went smooth. Service ran as expected.

@pontus-aurora
Copy link

pontus-aurora commented Nov 29, 2024

I ran into this as well. I had a .gitignore in the directory just above my monorepo. Removing it solved the issue.

Edit: I run fedora 39, npm 10.8.2, node 20.18.0 and nx 20.1.2

@StanDarius24
Copy link

hello, unfortunately I have the same problem, tried anything in the thread, but for some reason my npm/pnpm is stuck at post-install
any ideas what can I try?

Thanks a lot!

@maxtschi
Copy link

maxtschi commented Dec 2, 2024

I found a temporary workaround for me:

killall -9 node

rm -r node_modules

and then to move the project to /tmp or /Users/shared or somewhere outside of /Users/<user> and do a fresh pnpm i.

This also worked for me. Having it outside of the /Users/<user> seems to do the trick, regardless of the .gitignore.
Still not sure what the original issue is here.

MacOs 15.1.1
NX 19.6.5
node 22.3.0
npm 10.8.0

@alecf
Copy link

alecf commented Dec 2, 2024

Another data point: I also had a ~/.gitignore in the directory above the one I'm working in - and everything I did with nx would hang - install, run commands, etc. When I removed this file, everything started working again.

Also note that my ~/.gitignore was just *, meaning to ignore everything.. so I wonder if any gitignore that has a generic wildcard causes all files to be ignored, so waiting for files to exist means none will ever exist?

@FrozenPandaz FrozenPandaz assigned AgentEnder and unassigned Cammisuli Dec 3, 2024
@mfisher87
Copy link

Reproducible with npm as well: https://github.com/mfisher87/sscce-nx-with-bare-repo-dotfiles-management-hangs-indefinitely

@mfisher87
Copy link

I did my best to debug the post-install script. I was able to narrow the hang in my reproduction example above to this line:

return workspaceContext.glob(globs, exclude);

I got a bit stuck here. I'm not sure what is being called here; there's some Rust code mixed in the "native" module this is imported from, and that's where I find myself a bit out of my depth.

@pontus-aurora
Copy link

An somewhat educated guess is that something in nx is "walking" the directory structure and gathering a list of file. Another thread / process is waiting for that list but it never comes... maybe a race condition or bug in the code walking the directory.

It's a bit weird and even "wrong" for nx to need to look outside the top level directory of the project.

@turnerguo
Copy link

turnerguo commented Dec 16, 2024

I have the same issue in Gitlab CI environment, I saw the doc says it will turned off in CI mode, guess here might be a bug since it still execute demon logic.
os: centos-linux-x64
node: 20.18
pnpm: 9.14.2
nx: 20.1.3

.../.pnpm/[email protected]/node_modules/nx postinstall:  NX   Failed to start or connect to the Nx Daemon process.
.../.pnpm/[email protected]/node_modules/nx postinstall: Messages from the log:
.../.pnpm/[email protected]/node_modules/nx postinstall: /builds/xxx/72781/xxx/node_modules/.pnpm/[email protected]/node_modules/nx/src/daemon/socket-utils.js:38
.../.pnpm/[email protected]/node_modules/nx postinstall:         throw new Error([
.../.pnpm/[email protected]/node_modules/nx postinstall:         ^
.../.pnpm/[email protected]/node_modules/nx postinstall: Error: Attempted to open socket that exceeds the maximum socket length.
.../.pnpm/[email protected]/node_modules/nx postinstall: Set NX_SOCKET_DIR to a shorter path (e.g. /tmp/nx-tmp) to avoid this issue.

@andreidiaconescu
Copy link

up

@andreidiaconescu
Copy link

andreidiaconescu commented Jan 15, 2025

Thank you for the information. I also have this problem and works fine when I remove .gitignore in home directory

Other solution was

   "pnpm": {
     "overrides": {
       "nx": "19.4.4",

so, I think the cause is introduced right after 19.4.4. Please fix if possible.

i tried with nx 19.4.4, i use npm, but issue still reproduces for me, practically i am not able to add nx to my project, which i was trying to start using nx on.

@albert118
Copy link

I can confirm another datapoint for the home .gitignore. Mine was generated by virtualenv and only includes a wildcard pattern. Removing this file (I just renamed it) solves the problem.

@mfisher87
Copy link

I work around this by cloning the project outside my home dir, then symlinking to that directory from my ~/Projects dir. This way I keep my dotfiles management strategy the same, and can still navigate to my project with cd ~/Projects/myproject instead of remembering the special location I stuck it at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope: core core nx functionality type: bug
Projects
None yet
Development

No branches or pull requests