Repository navigation
[rush] Delete stale build-cache-restore staging folders left by interrupted restores - #6213
Open
Heramb_Inamke (HerambInamke) wants to merge 1 commit into
Conversation
An interrupted build cache restore (Ctrl+C, SIGKILL) left its staging folder in the project's .rush/temp folder, because the folder was only deleted in a finally block. Name each staging folder after the process that made it, and before staging a restore, delete the folders of processes that are no longer running. Fixes microsoft#6145
Author
|
@microsoft-github-policy-service agree |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #6145.
When Rush restores an operation from a local build cache entry with tar, it extracts the entry into
<project>/.rush/temp/build-cache-restore-XXXXXX/and then moves the output folders into place. If the process was interrupted during the restore (Ctrl+C on a slowrush build/rush start, or SIGKILL), that staging folder stayed behind indefinitely with a copy of the outputs. Tools that scan the project folder (for example a lint glob) then picked up the stale files.With this change, the next restore of that project deletes the staging folders that interrupted restores left behind.
Details
Root cause: in
libraries/rush-lib/src/logic/buildCache/StagedRestore.tsthe staging folder was deleted only in afinallyblock. Afinallyblock does not run when the process is killed or exits on a signal, and nothing removed such folders later.Fix:
build-cache-restore-<process id>-XXXXXX.deleteStaleStagingFoldersAsyncdeletes thebuild-cache-restore-<pid>-*folders in the project's.rush/tempfolder whose process is no longer running. It checks withprocess.kill(pid, 0): onlyESRCHcounts as not running, andEPERMmeans the process exists.Not handled:
build-cache-restore-XXXXXX, with no process id) are kept, because there's no safe way to tell whether a running older Rush process is using them. They can be deleted by hand once, as before.Alternatives considered: staging under the common temp folder, as the issue suggests, would need a folder on the same volume as every project so that the renames still work. That is a larger change, and it would still leave stale folders, just in a different place.
No backwards-compatibility impact, and the cost is one
readdirof the project's.rush/tempfolder per staged restore.How it was tested
New test
U8insrc/logic/buildCache/test/StagedRestore.test.ts. It sets up three staging folders: one from a process that has exited, one from a running process, and one with no process id. After a restore, only the folder of the exited process is deleted, and the new staging folder's name has the current process id. The test fails without the fix and passes with it.Commands run:
Follow-up issues created: none.