| 26 Jun 2025 |
 Matt Sturgeon | In reply to @corngood:corngood.com
Unfortunately I have no new info.
Is this normal?
> No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll`
I think that's normal, I guess a specific test file just has no non-disabled tests?
It shows up several times in the logs. | 22:26:06 |
| Matt Sturgeon | In reply to @corngood:corngood.com
I had this happen to me locally today. It must be intermittent I think I saw it on my nix-config CI too once, with the unfree package. The hydra job linked in the issue was for the non-unfree package. The user report was for the unfree package. So we have a selection of failures on each variant 🫤 | 22:28:41 |
| 27 Jun 2025 |
 Corngood | I was 2/2 reproducing it and now I'm 0/~10 after adding set -x for tracing :|. When it did fail, the exit code reported by nix build was 1. | 13:34:23 |
| Matt Sturgeon | Thanks for looking into it. Nothing worse than an intermittent problem 😕 | 14:47:58 |
| Matt Sturgeon | As an aside, whenever I build with --check it always tells me the package isn't deterministic... Is that a fundamental issue with dotnet packages or something I could fix in my package with enough investigation? | 15:22:22 |
| Corngood | I suspect there are determinism issues in dotnet, so I wouldn't necessarily worry about it, but if you do get a chance to diff them it would be interesting to know what changes. | 15:24:42 |
| Matt Sturgeon | I reproduced the issue with set -x, but I don't see anything useful in the output:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.08] Discovering: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.11] Discovered: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.12] Starting: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.20] Finished: NexusMods.Networking.GitHub.Tests
No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll
+ exitHandler
+ exitCode=1
+ set +e
+ '[' -n '' ']'
+ (( 1 != 0 ))
+ runHook failureHook
+ local hookName=failureHook
+ shift
+ local 'hooksSlice=failureHooks[@]'
+ local hook
+ for hook in "_callImplicitHook 0 $hookName" ${!hooksSlice+"${!hooksSlice}"}
+ _logHook failureHook '_callImplicitHook 0 failureHook'
+ [[ -z 2 ]]
+ local hookKind=failureHook
+ local 'hookExpr=_callImplicitHook 0 failureHook'
+ shift 2
+ declare -F '_callImplicitHook 0 failureHook'
+ type -p '_callImplicitHook 0 failureHook'
+ [[ _callImplicitHook 0 failureHook != \_\c\a\l\l\I\m\p\l\i\c\i\t\H\o\o\k* ]]
+ _eval '_callImplicitHook 0 failureHook'
+ declare -F '_callImplicitHook 0 failureHook'
+ eval '_callImplicitHook 0 failureHook'
++ _callImplicitHook 0 failureHook
++ local def=0
++ local hookName=failureHook
++ declare -F failureHook
++ type -p failureHook
++ '[' -n '' ']'
++ return 0
+ return 0
+ '[' -n '' ']'
+ return 1
| 15:58:58 |
| Matt Sturgeon | Download set-x.log | 15:59:31 |
 K900 | In reply to @mattsturg:matrix.org
I reproduced the issue with set -x, but I don't see anything useful in the output:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.08] Discovering: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.11] Discovered: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.12] Starting: NexusMods.Networking.GitHub.Tests
[xUnit.net 00:00:00.20] Finished: NexusMods.Networking.GitHub.Tests
No test matches the given testcase filter `Category!=Disabled&FlakeyTest!=True&RequiresNetworking!=True&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.LegacyDatabaseSupportTests.TestDatabase&FullyQualifiedName!=NexusMods.DataModel.SchemaVersions.Tests.MigrationSpecificTests.TestsFor_0...` in /build/NexusMods.App/tests/NexusMods.Networking.GitHub.Tests/bin/Release/net9.0/NexusMods.Networking.GitHub.Tests.dll
+ exitHandler
+ exitCode=1
+ set +e
+ '[' -n '' ']'
+ (( 1 != 0 ))
+ runHook failureHook
+ local hookName=failureHook
+ shift
+ local 'hooksSlice=failureHooks[@]'
+ local hook
+ for hook in "_callImplicitHook 0 $hookName" ${!hooksSlice+"${!hooksSlice}"}
+ _logHook failureHook '_callImplicitHook 0 failureHook'
+ [[ -z 2 ]]
+ local hookKind=failureHook
+ local 'hookExpr=_callImplicitHook 0 failureHook'
+ shift 2
+ declare -F '_callImplicitHook 0 failureHook'
+ type -p '_callImplicitHook 0 failureHook'
+ [[ _callImplicitHook 0 failureHook != \_\c\a\l\l\I\m\p\l\i\c\i\t\H\o\o\k* ]]
+ _eval '_callImplicitHook 0 failureHook'
+ declare -F '_callImplicitHook 0 failureHook'
+ eval '_callImplicitHook 0 failureHook'
++ _callImplicitHook 0 failureHook
++ local def=0
++ local hookName=failureHook
++ declare -F failureHook
++ type -p failureHook
++ '[' -n '' ']'
++ return 0
+ return 0
+ '[' -n '' ']'
+ return 1
Well it exits with 1 | 16:00:04 |
| K900 | Presumably becuase it found no tests to run | 16:00:04 |
| Matt Sturgeon | Maybe... but we also see that output in successful builds, and 6x total in the this build 🤔
Unlike the exit 1, that error doesn't seem inconsistent.
| 16:02:20 |
| Corngood | So it looks like dotnet test is definitely returning 1. I guess my next idea would be to test -v:diag and see if there's any sort of difference there. If that doesn't show anything then maybe COREHOST_TRACE or strace? | 16:16:53 |
| Matt Sturgeon | Just for comparison, here's a successful build log (also with set -x). We see the same 6 matches for No test matches the given testcase filter. | 16:21:34 |
| Matt Sturgeon | Download set-x-success.log | 16:21:40 |
| Matt Sturgeon | Will try that out, just adding -v:diag to dotnetTestFlags? | 16:22:05 |
| Corngood | Yeah, that should work. It'll probably be a lot of output. | 16:23:12 |
| Matt Sturgeon | Ok took a couple tries, but got a build failure with set -x and -v:diag | 16:46:08 |
| Matt Sturgeon | Download v-diag-fail-1.log | 16:46:34 |
| Matt Sturgeon | For comparison, here's a successful build: | 16:49:11 |
| Matt Sturgeon | Download v-diag-success-1.log | 16:49:25 |
| Matt Sturgeon | This seems relevant:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.8.2+699d445a1a (64-bit .NET 9.0.6)
[xUnit.net 00:00:00.11] Discovering: NexusMods.Games.RedEngine.Tests
[xUnit.net 00:00:00.19] Discovered: NexusMods.Games.RedEngine.Tests
[xUnit.net 00:00:00.19] Starting: NexusMods.Games.RedEngine.Tests
Passed NexusMods.Games.RedEngine.Tests.Cyberpunk2077SynchronizerTests.ContentIsIgnoredWhenSettingIsSet [102 ms]
Passed NexusMods.Games.RedEngine.Tests.Cyberpunk2077DiagnosticTests.CyberpunkGameExposesPatternBasedDiagnostics [106 ms]
The active test run was aborted. Reason: Test host process crashed
Test Run Aborted.
Total tests: Unknown
Passed: 2
Total time: 2.3107 Seconds
The "VSTestTask" task returned false but did not log an error. (TaskId:381)
Done executing task "VSTestTask" -- FAILED. (TaskId:381)
Done building target "_VSTestConsole" in project "NexusMods.Games.RedEngine.Tests.csproj" -- FAILED.: (TargetId:99)
Done executing task "CallTarget" -- FAILED. (TaskId:361)
Done building target "VSTest" in project "NexusMods.Games.RedEngine.Tests.csproj" -- FAILED.: (TargetId:98)
| 16:51:35 |
| Corngood | I wonder if this can be reproduced in a dev shell by repeatedly running the test phase. That would make it quite a bit easier to investigate. | 16:52:46 |
| Matt Sturgeon | Probably... I just don't have that workflow committed to muscle memory yet, so I never bother trying to use devshells for packages... 🙈 | 16:53:55 |
| Corngood | It should be possible to run that 'test host process' directly, but I'm not exactly sure how to do it | 16:53:58 |
| Corngood | my usual thing is
nix develop -f. nexusmods-app
cd $NIX_BUILD_TOP
genericBuild
the problem with that is that it won't stop on errors, so it'll try to run all the phases, which may or may not be a problem. at the end you can do runPhase checkPhase
| 16:55:42 |
| Corngood | I also sometimes do (set -e; genericBuild) but the problem with that is that you lose the subshell environment, and some things might depend on shell vars etc | 16:56:42 |
| Matt Sturgeon | I've not managed to reproduce the issue with runPhase checkPhase, either I'm getting (un?)lucky, the underlying problem is outside the checkPhase, or something is cached. | 18:50:11 |
| 28 Jun 2025 |
| Matt Sturgeon | Looking around, it seems vtest has a thread discussing a similar Test host process crashed error. In that instance, it turned out to be an out-of-memory error.
Is there an easy way to tell the package to run time -v dotnet test instead of simply dotnet test?
If it turns out to be a memory issue, is there a way to increase dotnet test's available memory, or should we just disable RAM-intensive tests?
| 10:12:44 |
| Corngood | The `dotnet test` call is in `dotnet-check-hook.sh`. You could hack it in there.
If it was out of memory it would have to be using a lot of memory to be reproducible on my desktop machine. Enough that I would certainly consider it a bug in itself. | 12:54:38 |
| Corngood | Maybe it's hitting a file handle ulimit or something? It would be nice to know more about the child process invocation (args, output, exit code). | 13:00:19 |
