For what it's worth, even with the code below I'm not seeing anything being printed out, which I would expect due to the numerous printMsg calls.

void run(ref<Store> store, Installables && installables) override
{
    // TODO(@connorbaker): Unsure of how to catch eval errors so the other installables are still computed.
    const auto drvPaths = Installable::toDerivations(store, installables, true);
    auto jsonRoot = json::object();
    auto hasErrors = false;

    const auto tryAction = [&](const auto & action) {
        try {
            action();
        } catch (EvalError & e) {
            printMsg(lvlError, "settings.keepGoing: %s", settings.keepGoing);
            if (settings.keepGoing) {
                ignoreExceptionExceptInterrupt();
                hasErrors = true;
            } else {
                throw;
            }
        } catch (Error & e) {
            printMsg(lvlError, "settings.keepGoing: %s", settings.keepGoing);
            if (settings.keepGoing) {
                ignoreExceptionExceptInterrupt();
                hasErrors = true;
            } else {
                throw;
            }
        } catch (Interrupted & e) {
            printMsg(lvlError, "settings.keepGoing: %s", settings.keepGoing);
            if (settings.keepGoing) {
                ignoreExceptionExceptInterrupt();
                hasErrors = true;
            } else {
                throw;
            }
        } catch (const std::exception &e) {
            printMsg(lvlError, "settings.keepGoing: %s", settings.keepGoing);
            if (settings.keepGoing) {
                ignoreExceptionExceptInterrupt();
                hasErrors = true;
            } else {
                throw;
            }
        // Please, PLEASE JUST CATCH IT.
        } catch (...) {
            printMsg(lvlError, "settings.keepGoing: %s", settings.keepGoing);
            if (settings.keepGoing) {
                ignoreExceptionExceptInterrupt();
                hasErrors = true;
            } else {
                throw;
            }
        }
    };

    const auto addDrvPathsToJSONRoot = [&](const auto & drvPaths) {
        for (const auto & drvPath : drvPaths) {
            if (!drvPath.isDerivation()) {
                continue;
            }

            jsonRoot[store->printStorePath(drvPath)] =
                store->readDerivation(drvPath).toJSON(*store);
        }
    };

    if (recursive) {
        // In the recursive case, don't add the closure of a drvPath if it fails to evaluate.
        for (const auto & drvPath : drvPaths) {
            tryAction([&] {
                StorePathSet closure;
                store->computeFSClosure(drvPath, closure);
                addDrvPathsToJSONRoot(closure);
            });
        }
    } else {
        tryAction([&] { addDrvPathsToJSONRoot(drvPaths); });
    }

    logger->cout(jsonRoot.dump());

    if (hasErrors) {
        throw Error("some errors were encountered during the evaluation");
    }
}

Yesterday I got an error in a Nix build which was using Ninja (nothing new there, bad CMake config). However, I noticed I was able to reliably reproduce a Nix error: https://gist.github.com/ConnorBaker/5cebac5224ab430e67ee25d7a5bd0224

bad JSON log message from builder: [json.exception.parse_error.101] parse error at line 1, column 65539: syntax error while parsing array - invalid literal; last read: <snipped>

Any idea if there's a limit on the length of the output from a build? I believe this was using a remote builder if that's any help.

In reply to @connorbaker:matrix.org

Yesterday I got an error in a Nix build which was using Ninja (nothing new there, bad CMake config). However, I noticed I was able to reliably reproduce a Nix error: https://gist.github.com/ConnorBaker/5cebac5224ab430e67ee25d7a5bd0224

bad JSON log message from builder: [json.exception.parse_error.101] parse error at line 1, column 65539: syntax error while parsing array - invalid literal; last read: <snipped>

Any idea if there's a limit on the length of the output from a build? I believe this was using a remote builder if that's any help.

Running it again with that flag. For what it's worth, when I built locally and used sudo (so no daemon) I didn't get the error -- which is to say I saw

cuda12.6-onnxruntime> /nix/store/va7kw1i822h95im4jacci19v0cqajfyf-binutils-2.43.1/bin/ld: cannot find -lcudnn_frontend: No such file or directory
cuda12.6-onnxruntime> collect2: error: ld returned 1 exit status
cuda12.6-onnxruntime> ninja: build stopped: subcommand failed.

Running it again with that flag. For what it's worth, when I built locally and used sudo (so no daemon) I didn't get the error -- which is to say I saw

cuda12.6-onnxruntime> /nix/store/va7kw1i822h95im4jacci19v0cqajfyf-binutils-2.43.1/bin/ld: cannot find -lcudnn_frontend: No such file or directory
cuda12.6-onnxruntime> collect2: error: ld returned 1 exit status
cuda12.6-onnxruntime> ninja: build stopped: subcommand failed.
error: builder for '/nix/store/w9nxvfann68v2am1kym3hkyxmc4a8p1f-cuda12.6-onnxruntime-1.20.0-unstable-2024-11-14.drv' failed with exit code 1

In reply to @mightyiam:matrix.org

Is anyone interested in the core dump for this?

nix-repl> :env
Env level 0
static: step swayMsgPath pactl incVol toggleMuteSources

Env level 1


terminating due to unexpected unrecoverable internal error: Unexpected condition in operator[] at src/libexpr/symbol-table.hh:119
zsh: abort (core dumped)  nix build --debugger

In reply to @connorbaker:matrix.org

Yesterday I got an error in a Nix build which was using Ninja (nothing new there, bad CMake config). However, I noticed I was able to reliably reproduce a Nix error: https://gist.github.com/ConnorBaker/5cebac5224ab430e67ee25d7a5bd0224

bad JSON log message from builder: [json.exception.parse_error.101] parse error at line 1, column 65539: syntax error while parsing array - invalid literal; last read: <snipped>

Any idea if there's a limit on the length of the output from a build? I believe this was using a remote builder if that's any help.

got this for the first time today

nixinate-phone> building '/nix/store/z03lxdp085hqpaxglqm7iphvdsvcdgvs-cardinal-24.09.drv'...
nixinate-phone> bad JSON log message from builder: [json.exception.parse_error.101] parse error at line 1, column 4099: syntax error while parsing array - invalid literal; last read: '"\u001b[01m\u001b[K/nix/store/pxb3zpbg0qdccadh884fag33va0xb4ds-gcc-13.3.0/include/c++/13.3.0/variant:1522:33:\u001b[m\u001b[K   required from '\u001b[01m\u001b[Kstd::enable_if_t<(is_constructible_v<_Tp, _Args ...> && __exactly_once<_Tp>), _Tp&> std::variant<_Types>::\u001b[01;32m\u001b[Kemplace\u001b[m\u001b[K(_Args&& ...) \u001b[35m\u001b[K[with _Tp = RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 32, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 32, 1> >; _Args = {}; _Types = {NullModel, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 12, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 12, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 12, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 12, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 12, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 12, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 16, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 16, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 16, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 16, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 16, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 16, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 20, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 20, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 20, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 20, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 20, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 20, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 32, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 32, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 32, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 32, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 32, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 32, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 40, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 40, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 40, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 40, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 40, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 40, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::GRULayerT<float, 1, 64, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 64, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::GRULayerT<float, 2, 64, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 64, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::GRULayerT<float, 3, 64, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 64, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::LSTMLayerT<float, 1, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 2, 1, RTNeural::LSTMLayerT<float, 2, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 3, 1, RTNeural::LSTMLayerT<float, 3, 8, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 8, 1> >, RTNeural::ModelT<float, 1, 1, RTNeural::LSTMLayerT<float, 1, 12, RTNeural::SampleRateCorrectionMode::None>, RTNeural::DenseT<float, 12, 1> >, RTNeural::ModelT@nix {"a'; expected ']'

Hi. Wanted to share a pretty curious finding. I've been looking into speeding up the flex generated lexer by using the full scanner tables --full (a.k.a. -Cf) and --fast (-CF) https://westes.github.io/flex/manual/Options-for-Scanner-Speed-and-Size.html#Options-for-Scanner-Speed-and-Size. They can give pretty nice uplifts in performance while compiling into a bit larger binaries.

I've run into a funny issue, which really looks like either a flex bug or strange lexer rules on the cppnix side. When built with -Cf the scanner fails to match comments in the derivation primop:

nix-instantiate --parse ../flake.nix
--accepting rule at line 314 ("
")
--accepting rule at line 160 ("# This is the implementation of the ")
error: syntax error, unexpected DOLLAR_CURLY
       at <nix/derivation-internal.nix>:2:1:
            1|
            2| # This is the implementation of the ‘derivation’ builtin function.
             | ^
            3| # It's actually a wrapper around the ‘derivationStrict’ primop.

Looks like the generated scanner trips up lots of other cases with comments as well. Has anyone else previously looked into using full scanners previously?

Hi. Wanted to share a pretty curious finding. I've been looking into speeding up the flex generated lexer by using the full scanner tables --full (a.k.a. -Cf) and --fast (-CF) https://westes.github.io/flex/manual/Options-for-Scanner-Speed-and-Size.html#Options-for-Scanner-Speed-and-Size. They can give pretty nice uplifts in performance while compiling into a bit larger binaries.

I've run into a funny issue, which really looks like either a flex bug or strange lexer rules on the cppnix side. When built with -Cf the scanner fails to match comments in the derivation primop:

nix-instantiate --parse ../flake.nix
--accepting rule at line 314 ("
")
--accepting rule at line 160 ("# This is the implementation of the ")
error: syntax error, unexpected DOLLAR_CURLY
       at <nix/derivation-internal.nix>:2:1:
            1|
            2| # This is the implementation of the ‘derivation’ builtin function.
             | ^
            3| # It's actually a wrapper around the ‘derivationStrict’ primop.

Looks like the generated scanner trips up lots of other cases with comments as well. Has anyone else previously looked into using full scanners?