| 7 Oct 2025 |
|  Primrose (== Léana) changed their profile picture. | 05:24:07 |
|  Robin Heggelund Hansen joined the room. | 08:14:41 |
 delroth | we will start the maintenance in 34 minutes at 18:00 CEST | 15:26:50 |
| delroth | starting now | 16:00:07 |
 raitobezarius | We are running into extra issues due to the H2 database corruption we reported the other day, we are investigating them. | 17:10:06 |
| raitobezarius | gerrit.lix.systems is back up again | 17:56:24 |
| raitobezarius | If you are curious as what happened (my soul is dried for now), you can enjoy this story: https://zulip.afnix.fr/#narrow/channel/4-infra/topic/gerrit.2Elix.2Esystems.20migration/near/7026 | 17:57:16 |
| raitobezarius | TL;DR: H2 database which contains "mark files as reviewed" self corrupted on each restarts, we observed CL which were merged to be "unmerged", leading us to worry about this data corruption. We realized later on that rsync didn't synchronize the actual state files and the NoteDB which contained the source of truth on reviews. We used another approach and nuked the H2 database for now. | 17:58:07 |
| raitobezarius | flokli: You will certainly be affected by this as you are on Gerrit 3.12 as well, H2 self corrupts after a certain scale: https://github.com/h2database/h2database/issues/4123 FYI | 17:59:41 |
| raitobezarius | TL;DR for Lix developers: all "mark files as reviewed" information is gone for now. I deem this information to be minor, if you disagree, please let me know and I will forensics the old copy we have to restore the database. | 18:00:30 |
| raitobezarius | All the rest of data is safe. | 18:00:36 |
 just1602 | I read the Zulip thread and things seems fixed, but on my side I get PR_CONNECT_RESET_ERROR when I try to access gerrit.lix.systems | 19:50:12 |
| just1602 | Oh it works on mobile, but not at work. It might be on my side | 19:52:03 |
| raitobezarius | A redeployment happened at that moment | 19:52:55 |
| raitobezarius | Do you still have the error at work? | 19:53:03 |
 Rutile (Commentator2.0) feel free to ping | i get an ominous "Server error" when trying to log in at gerrit | 19:53:17 |
| raitobezarius | I know what's going on, give me a sec | 19:53:27 |
| raitobezarius | Rutile (Commentator2.0) feel free to ping: can you give it a try again? | 19:55:30 |
| raitobezarius | aaaaaaaa | 19:55:46 |
| Rutile (Commentator2.0) feel free to ping | same error | 19:55:48 |
| raitobezarius | wait | 19:55:49 |
| raitobezarius | it won't work | 19:55:50 |
| raitobezarius | Rutile (Commentator2.0) feel free to ping: give it a try now | 19:57:13 |
| raitobezarius | Please give it a try as well | 19:57:18 |
 Katalin 🔪 | still happens for me | 19:57:50 |
| Rutile (Commentator2.0) feel free to ping | negative | 19:57:51 |
| raitobezarius | clearing some cache here and there | 19:58:07 |
| just1602 | Doesn't work neither | 19:58:16 |
| raitobezarius | Give me 2 minutes or so | 19:59:26 |
| raitobezarius | Alright, just1602 Rutile (Commentator2.0) feel free to ping Katalin 🔪 | 20:03:08 |
