!9IQChSjwSHXPPWTa:lix.systems

Lix

1101 Members
Lix user channel. Feel free to discuss on-topic issues here and give each other help. For matrix.to links to the rest of the Lix channels, see: https://wiki.lix.systems/books/lix-organisation/page/matrix-rooms293 Servers

Load older messages


SenderMessageTime
6 Dec 2025
@aloisw:julia0815.dealoisw Yes, that's exactly why it is so slow and we're having this conversation. 14:57:32
@hexa:lossy.networkhexathanks for looking int o that14:57:36
@hexa:lossy.networkhexa* thanks for looking into that14:57:39
@aloisw:julia0815.dealoisw You eval on tmpfs, right? 14:58:07
@hexa:lossy.networkhexanot sure what "eval on tmpfs" means14:59:04
@aloisw:julia0815.dealoisw That the store that the machine performing the evaluation writes to is located on tmpfs. 14:59:39
@hexa:lossy.networkhexaoh, you mean a non-default store then15:00:05
@hexa:lossy.networkhexano15:00:19
@hexa:lossy.networkhexa* that'd be a no15:00:24
@kfears:matrix.orgKFears (burnt out)
In reply to @aloisw:julia0815.de
Yes, that's exactly why it is so slow and we're having this conversation.
I must say, I'm very surprised SQLite suffocates under that load
15:09:32
@aloisw:julia0815.dealoisw Ah lovely, when you try to do PRAGMA journal_mode = TRUNCATE on a WAL database concurrently it may just deadlock. 15:23:16
@aloisw:julia0815.dealoisw Of course, as they open a connection with WAL first, but you obviously can't have a WAL connection while trying to convert the database to truncate mode. In particular once you're in the deadlock state, no amount of retry within the connection (as Lix does) will try to help you. 15:27:51
@raitobezarius:matrix.orgraitobezarius
In reply to @hexa:lossy.network
oh, you mean a non-default store then
Technically /Nix can be tmpfs
15:56:42
@shalokshalom:kde.orgShalokShalom
In reply to @aloisw:julia0815.de
Yes, that's exactly why it is so slow and we're having this conversation.
Isn't there somebody rewriting SQLite in Rust?
16:30:32
@shalokshalom:kde.orgShalokShalomCould that make a difference 16:30:42
@k900:0upti.meK900Rust is not magic16:31:40
@k900:0upti.meK900And I'm not aware of a Rust API compatible SQLite replacement16:31:56
@aloisw:julia0815.dealoisw Yes, a VC-backed company plastering "AI" all over their website called Turso. It was already suggested above. 16:37:23
@aloisw:julia0815.dealoisw In any case Rust is not going to solve write amplification. 16:37:36
@aloisw:julia0815.dealoisw
diff --git a/overlay.nix b/overlay.nix
--- a/overlay.nix
+++ b/overlay.nix
@@ -26,7 +26,7 @@ meowLib.runOnDirectory (path: final.call
     _finalLixes: prevLixes: {
       git = prevLixes.git.overrideScope (
         _finalLix: prevLix: {
-          lix = prevLix.lix.overrideAttrs (old: {
+          lix = (prevLix.lix.override { sqlite = final.sqlite-wal2; }).overrideAttrs (old: {
             version = "2.95.0-pre-${builtins.substring 0 12 source.lix.rev}";
             src = source.lix;
             patches = old.patches or [ ] ++ meowLib.collectPatches ./patches/lix;
diff --git a/patches/lix/wal2.diff b/patches/lix/wal2.diff
new file mode 100644
--- /dev/null
+++ b/patches/lix/wal2.diff
@@ -0,0 +1,13 @@
+diff --git a/lix/libstore/sqlite.cc b/lix/libstore/sqlite.cc
+index fe606bce20..81f71a8b9f 100644
+--- a/lix/libstore/sqlite.cc
++++ b/lix/libstore/sqlite.cc
+@@ -134,7 +134,7 @@
+         // Set the SQLite journal mode.
+         // WAL mode is fastest, so it's the default.
+         if (settings.useSQLiteWAL) {
+-            exec("pragma main.journal_mode = wal", always_progresses);
++            exec("pragma main.journal_mode = wal2", always_progresses);
+              /* persist the WAL files when the DB connection is closed.
+              * This allows for read-only connections without any write permissions
+              * on the state directory to succeed on a closed database. Setting the
diff --git a/pkgs/sqlite-wal2/package.nix b/pkgs/sqlite-wal2/package.nix
new file mode 100644
--- /dev/null
+++ b/pkgs/sqlite-wal2/package.nix
@@ -0,0 +1,22 @@
+# SPDX-FileCopyrightText: Alois Wohlschlager <alois1@gmx-topmail.de>
+# SPDX-License-Identifier: MIT
+{
+  stdenv,
+  fetchurl,
+  readline,
+}:
+stdenv.mkDerivation {
+  pname = "sqlite-wal2";
+  version = "3.52.0-unstable-2025-11-28";
+
+  src = fetchurl {
+    url = "https://sqlite.org/src/tarball/sqlite-20251128204824-18d8e307df.tar.gz";
+    hash = "sha256-ELgz1zlIljJWFhunjixDjS2yeYGHjVl3m12v4XFKruU=";
+  };
+
+  strictDeps = true;
+
+  buildInputs = [ readline ];
+
+  configureFlags = [ "--with-readline-header=${readline.dev}/include/readline/readline.h" ];
+}

I wonder if I will regret this…

19:43:02
@raitobezarius:matrix.orgraitobezarius:D19:43:36
@aloisw:julia0815.dealoisw Great, the WAL hook is not called at all… 20:05:11
7 Dec 2025
@aloisw:julia0815.dealoisw @raitobezarius:matrix.org if you're feeling really adventurous, you can also give https://gerrit.lix.systems/c/lix/+/4725 a try on a fresh store. 07:15:37
@522_:catgirl.cloud522 it/its ⛯ΘΔoh what sqlite has a wal2?12:35:38
@522_:catgirl.cloud522 it/its ⛯ΘΔoh, not even in mainline12:35:59
@522_:catgirl.cloud522 it/its ⛯ΘΔah, https://sqlite.org/hctree/doc/wal2/doc/wal2.md12:37:23
@aloisw:julia0815.dealoisw https://sqlite.org/src/file?ci=wal2&name=doc%2Fwal2.md is the one for the wal2-only branch. 13:16:24
@aloisw:julia0815.dealoisw I do not mean to ship this, it's more a quick experiment to see whether or in what ways a better database may help. 13:17:31
@aloisw:julia0815.dealoisw Not that I expect wal2 to be perfect either, a client-server database might offer some advantages in terms of write amplification, but of course it's a pain to manage. 13:18:43
8 Dec 2025
@zitrone:utwente.iozitrone Anyone got an idea what error: cannot change ownership of '/nix/var/nix/b/5i50qd60sib1gcpvdmbrr667pd': Invalid argument means, and what i could do about it?
It happens on one of my machines any time when i try to build a package that actually needs building (not cached).
/nix/var/nix/b is drwxr-xr-x root root like on my laptop, where the error is not happening.
01:18:21

There are no newer messages yet.


Back to Room ListRoom Version: 10