| 19 Jun 2023 |
 infinisil | Also, each number should be associated with documentation that specifies how to do the migration | 12:22:42 |
| infinisil | And integrated so that it gets printed automatically when needed | 12:23:00 |
 adisbladis | In reply to @ronnypfannschmidt:matrix.org
is anyone aware of work on managing state versions and migration for components (like database server versions, database migration, data store migration)
i have had a number of times now where a upgrade that would ultimately fail at the end broke system state by runnign database migrations that the old versions couldnt use anymore (in particular nextcloud) This sounds like a module bug that should be fixed | 12:23:24 |
 @ronnypfannschmidt:matrix.org | In reply to @adis:blad.is
This sounds like a module bug that should be fixed right now there is no safe tooling for it, so its easy to mess up | 12:23:45 |
| infinisil | I think it should be possible to create a stateVersions.<name> option that then each service sets | 12:23:54 |
| infinisil | E.g. nginx would set e.g. stateVersions.nginx = { version = 1; migrations = [ "Run this command: ..." ]; } | 12:25:09 |
| infinisil | * E.g. nginx would set stateVersions.nginx = { version = 1; migrations = [ "Run this command: ..." ]; } | 12:25:22 |
| adisbladis | In reply to @infinisil:matrix.org
E.g. nginx would set e.g. stateVersions.nginx = { version = 1; migrations = [ "Run this command: ..." ]; } I'm iffy on doing automated data migrations | 12:25:47 |
| infinisil | adisbladis: Not automated, you print the message to the user | 12:26:03 |
| @ronnypfannschmidt:matrix.org | and state versions are sometimes a problem in and of themselfes (like when you need/want to di a migration between major versions, or need a backup/restore step)
also sometimes migrations may or may not cooperate (for example well managed projects may have migrations that evetually enable a new feature, but also still enable older versions to run | 12:26:10 |
| infinisil | Yeah such integration should definitely give the user a choice and present all the interactions (like, backup first if you want to roll back afterwards, but you will lose data by doing so, etc.) | 12:27:05 |
| @ronnypfannschmidt:matrix.org | with the final key questions for a administrator/operator to deal with being things like - are migrations avaliable, what subset can i apply safely - how to i ensure backups/maintenance modes in case of breakage | 12:27:37 |
| adisbladis | In reply to @infinisil:matrix.org
adisbladis: Not automated, you print the message to the user Ahh, that's totally fine | 12:37:07 |
| adisbladis | It just struck me that impure derivations might be interesting for this use case | 12:37:51 |
| @ronnypfannschmidt:matrix.org | 2 important details wold be a) rejecting activation if the current state version isnt supported and 2. listing either avaliable state upgrades and/or hinting at a command to show them | 12:38:28 |
| @ronnypfannschmidt:matrix.org | i wouldnt consider it as job for a derivation per se - more like a "effect" that ensures activation is safe and/or enables upgrades of mutable state in a safe manner | 12:39:22 |
| adisbladis | I was thinking of it as a way to get localized tooling & ensure that you don't have impurities | 12:42:03 |
| adisbladis | I think you'd want similar sandboxing guarantees in your migrations as you would at build time? | 12:42:46 |
| @ronnypfannschmidt:matrix.org | In reply to @adis:blad.is
I think you'd want similar sandboxing guarantees in your migrations as you would at build time? cant exactly run a remote database migration without network | 12:44:13 |
| adisbladis | In reply to @ronnypfannschmidt:matrix.org
cant exactly run a remote database migration without network No, but tooling like pg_upgrade doesn't work on network connections | 12:44:46 |
| @ronnypfannschmidt:matrix.org | but application level schema migrations do for example | 12:45:07 |
| @ronnypfannschmidt:matrix.org | same goes for online migration using replication to avoid downtime | 12:45:38 |
| adisbladis | ^ This is also why dealing with migrations in a generalised manner isn't really that feasible. Everyone has very different operational constraints. | 12:47:50 |
| adisbladis | If we took the example from infinisil before where we'd print out migration steps.. I don't think that would realistically contain the steps for how to do zero downtime deploys. | 12:48:45 |
 growpotkin1 | I'm not seeing how to review this. I think I already did | 20:24:43 |
 @piegames:matrix.org | Hm, then you can also give your approval in here and I'll start FCP | 20:51:55 |
| infinisil | growpotkin ( Alex Ameen ): Just review the RFC on GitHub again with an approve | 20:57:15 |
| infinisil | Like the standard way of reviewing PR's on GitHub | 20:57:42 |
 raitobezarius | In reply to @ronnypfannschmidt:matrix.org
is anyone aware of work on managing state versions and migration for components (like database server versions, database migration, data store migration)
i have had a number of times now where a upgrade that would ultimately fail at the end broke system state by runnign database migrations that the old versions couldnt use anymore (in particular nextcloud) I wrote an issue on this | 21:18:52 |
| raitobezarius | And I am planning to do research on that in the next year | 21:19:11 |
