| 9 Jun 2025 |
 Whovian9369 | Thanks for the thoughts, Corngood!
| 23:11:49 |
| 24 Jun 2025 |
|  Wulpey joined the room. | 17:51:36 |
| 26 Jun 2025 |
 Matt Sturgeon | Any ideas on https://github.com/NixOS/nixpkgs/issues/420121 ?
Getting reports of a build failure, which shows up on hydra too, but I can't reproduce the issue even on the same commit as the hydra failure...
Even stranger, I can't see any errors in the logs... | 21:08:17 |
 Corngood | I had this happen to me locally today. It must be intermittent | 22:03:23 |
| Corngood | Unfortunately I have no new info.
Is this normal?
> No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll`
| 22:06:41 |
| Corngood | I'll run some rebuilds to try to get more info and see how common it is. Mine actually failed on nexusmods-app, not -unfree. | 22:08:47 |
| Matt Sturgeon | In reply to @corngood:corngood.com
Unfortunately I have no new info.
Is this normal?
> No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll`
I think that's normal, I guess a specific test file just has no non-disabled tests?
It shows up several times in the logs. | 22:26:06 |
| Matt Sturgeon | In reply to @corngood:corngood.com
I had this happen to me locally today. It must be intermittent I think I saw it on my nix-config CI too once, with the unfree package. The hydra job linked in the issue was for the non-unfree package. The user report was for the unfree package. So we have a selection of failures on each variant 🫤 | 22:28:41 |
| 27 Jun 2025 |
| Corngood | I was 2/2 reproducing it and now I'm 0/~10 after adding set -x for tracing :|. When it did fail, the exit code reported by nix build was 1. | 13:34:23 |
| Matt Sturgeon | Thanks for looking into it. Nothing worse than an intermittent problem 😕 | 14:47:58 |
| Matt Sturgeon | As an aside, whenever I build with --check it always tells me the package isn't deterministic... Is that a fundamental issue with dotnet packages or something I could fix in my package with enough investigation? | 15:22:22 |
| Corngood | I suspect there are determinism issues in dotnet, so I wouldn't necessarily worry about it, but if you do get a chance to diff them it would be interesting to know what changes. | 15:24:42 |
| Matt Sturgeon | I reproduced the issue with set -x, but I don't see anything useful in the output:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.08] Discovering: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.11] Discovered: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.12] Starting: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.20] Finished: NexusMods.Networking.GitHub.Tests
No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll
+ exitHandler
+ exitCode=1
+ set +e
+ '[' -n '' ']'
+ (( 1 != 0 ))
+ runHook failureHook
+ local hookName=failureHook
+ shift
+ local 'hooksSlice=failureHooks[@]'
+ local hook
+ for hook in "_callImplicitHook 0 $hookName" ${!hooksSlice+"${!hooksSlice}"}
+ _logHook failureHook '_callImplicitHook 0 failureHook'
+ [[ -z 2 ]]
+ local hookKind=failureHook
+ local 'hookExpr=_callImplicitHook 0 failureHook'
+ shift 2
+ declare -F '_callImplicitHook 0 failureHook'
+ type -p '_callImplicitHook 0 failureHook'
+ [[ _callImplicitHook 0 failureHook != \_\c\a\l\l\I\m\p\l\i\c\i\t\H\o\o\k* ]]
+ _eval '_callImplicitHook 0 failureHook'
+ declare -F '_callImplicitHook 0 failureHook'
+ eval '_callImplicitHook 0 failureHook'
++ _callImplicitHook 0 failureHook
++ local def=0
++ local hookName=failureHook
++ declare -F failureHook
++ type -p failureHook
++ '[' -n '' ']'
++ return 0
+ return 0
+ '[' -n '' ']'
+ return 1
| 15:58:58 |
| Matt Sturgeon | Download set-x.log | 15:59:31 |
 K900 | In reply to @mattsturg:matrix.org
I reproduced the issue with set -x, but I don't see anything useful in the output:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.08] Discovering: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.11] Discovered: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.12] Starting: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.20] Finished: NexusMods.Networking.GitHub.Tests
No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll
+ exitHandler
+ exitCode=1
+ set +e
+ '[' -n '' ']'
+ (( 1 != 0 ))
+ runHook failureHook
+ local hookName=failureHook
+ shift
+ local 'hooksSlice=failureHooks[@]'
+ local hook
+ for hook in "_callImplicitHook 0 $hookName" ${!hooksSlice+"${!hooksSlice}"}
+ _logHook failureHook '_callImplicitHook 0 failureHook'
+ [[ -z 2 ]]
+ local hookKind=failureHook
+ local 'hookExpr=_callImplicitHook 0 failureHook'
+ shift 2
+ declare -F '_callImplicitHook 0 failureHook'
+ type -p '_callImplicitHook 0 failureHook'
+ [[ _callImplicitHook 0 failureHook != \_\c\a\l\l\I\m\p\l\i\c\i\t\H\o\o\k* ]]
+ _eval '_callImplicitHook 0 failureHook'
+ declare -F '_callImplicitHook 0 failureHook'
+ eval '_callImplicitHook 0 failureHook'
++ _callImplicitHook 0 failureHook
++ local def=0
++ local hookName=failureHook
++ declare -F failureHook
++ type -p failureHook
++ '[' -n '' ']'
++ return 0
+ return 0
+ '[' -n '' ']'
+ return 1
Well it exits with 1 | 16:00:04 |
| K900 | Presumably becuase it found no tests to run | 16:00:04 |
| Matt Sturgeon | Maybe... but we also see that output in successful builds, and 6x total in the this build 🤔
Unlike the exit 1, that error doesn't seem inconsistent.
| 16:02:20 |
| Corngood | So it looks like dotnet test is definitely returning 1. I guess my next idea would be to test -v:diag and see if there's any sort of difference there. If that doesn't show anything then maybe COREHOST_TRACE or strace? | 16:16:53 |
| Matt Sturgeon | Just for comparison, here's a successful build log (also with set -x). We see the same 6 matches for No test matches the given testcase filter. | 16:21:34 |
| Matt Sturgeon | Download set-x-success.log | 16:21:40 |
| Matt Sturgeon | Will try that out, just adding -v:diag to dotnetTestFlags? | 16:22:05 |
| Corngood | Yeah, that should work. It'll probably be a lot of output. | 16:23:12 |
| Matt Sturgeon | Ok took a couple tries, but got a build failure with set -x and -v:diag | 16:46:08 |
| Matt Sturgeon | Download v-diag-fail-1.log | 16:46:34 |
| Matt Sturgeon | For comparison, here's a successful build: | 16:49:11 |
| Matt Sturgeon | Download v-diag-success-1.log | 16:49:25 |
| Matt Sturgeon | This seems relevant:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.11] Discovering: NexusMods.Games.RedEngine.Tests
[xUnit.net 00:00:00.19] Discovered: NexusMods.Games.RedEngine.Tests
[xUnit.net 00:00:00.19] Starting: NexusMods.Games.RedEngine.Tests
Passed NexusMods.Games.RedEngine.Tests.Cyberpunk2077SynchronizerTests.ContentIsIgnoredWhenSettingIsSet [102 ms]
Passed NexusMods.Games.RedEngine.Tests.Cyberpunk2077DiagnosticTests.CyberpunkGameExposesPatternBasedDiagnostics [106 ms]
The active test run was aborted. Reason: Test host process crashed
Test Run Aborted.
Total tests: Unknown
Passed: 2
Total time: 2.3107 Seconds
The "VSTestTask" task returned false but did not log an error. (TaskId:381)
Done executing task "VSTestTask" -- FAILED. (TaskId:381)
Done building target "_VSTestConsole" in project "NexusMods.Games.RedEngine.Tests.csproj" -- FAILED.: (TargetId:99)
Done executing task "CallTarget" -- FAILED. (TaskId:361)
Done building target "VSTest" in project "NexusMods.Games.RedEngine.Tests.csproj" -- FAILED.: (TargetId:98)
| 16:51:35 |
| Corngood | I wonder if this can be reproduced in a dev shell by repeatedly running the test phase. That would make it quite a bit easier to investigate. | 16:52:46 |
| Matt Sturgeon | Probably... I just don't have that workflow committed to muscle memory yet, so I never bother trying to use devshells for packages... 🙈 | 16:53:55 |
| Corngood | It should be possible to run that 'test host process' directly, but I'm not exactly sure how to do it | 16:53:58 |
