| 9 Aug 2021 |
 Mic92 | manveru: what do you save this way? | 12:13:14 |
| Mic92 | * manveru: what do you safe this way? | 12:13:22 |
| Mic92 | Ah constructing a string. | 12:13:37 |
 manveru | yeah :) | 12:13:43 |
| manveru | would need to bench to see if it's worth it... just stumbled over it when reading the code | 12:14:08 |
| Mic92 | I was also trying to optimize the very same code yesterday | 12:14:25 |
| Mic92 | There are different hash table implementations one can use. | 12:14:41 |
| Mic92 | llvm also has a hash set specific to symbols that allocates strings within the hash table to save memory. | 12:15:08 |
| Mic92 | I guess a count should be cheaper than a malloc but maybe you can do the benchmark. | 12:16:08 |
| manveru | well, i'd imagine a count won't allocate anything in the GC, so it should help for stuff like haskell.nix which has a ton of symbols afaik... | 12:20:46 |
 andi- | Why count over find? Do you fear the allocation an allocator might produce? count could probably be more costly than find that (should?) return on the first positive match. | 12:40:15 |
| Mic92 | There are also other interesting hash tables. For example for pointer to pointer maps, a dense hash map is better. | 14:29:57 |
| andi- | We probably want to store only the ptr but sort/bucket them by a hash for the string value to make lookups fast(er)? | 15:54:12 |
|  dadada (they/them) changed their display name from dadada to dadada (they/them). | 15:59:41 |
| andi- | I did a very unscientific benchmark of one version that uses unique_ptr and allocates only for new elements (AFAICT) and one that is basically what we have right now: https://quick-bench.com/q/BI8_-Nfg9dYrGBEwKR-0xm1QKw4 | 17:19:38 |
| andi- | 
Download BI8_-Nfg9dYrGBEwKR-0xm1QKw4.png | 17:19:54 |
| andi- | If you move the constructor / destructor of the table out of the loop it is 200 / 1100 in favor of the ptr approach. | 17:23:23 |
| andi- | The sad story is that this doesn't really seem to have an effect in practice :( It is +- 0.02 sec difference when evaling the Firefox test. | 17:47:32 |
| manveru | andi-: i'd love to try that with something a bit more GC heavy to see memory impact... i didn't expect much CPU difference | 18:54:27 |
| andi- | But those Symbols aren't tracked by the GC? Otherwise we should set an allocator for them. | 18:55:10 |
| manveru | i don't know how C++ works unfortunately :( | 18:55:47 |
| andi- | They are allocated once and stay there until the process terminates | 18:56:16 |
| manveru | my assumption was that this std::string thing does an allocation that needs to be cleaned up again afterwards | 18:56:26 |
| andi- | I has to be cleaned up but that happens then the entire symboltable goes out of scope | 18:57:01 |
| andi- | With C++20 (and GCC 11) we could make use of Heterogeneous lookup for unordered containers (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0919r3.html) for the lookup using the std::string_view... so far that isn't possible without huge effort AFAICT. | 19:53:17 |
 nixinator | In reply to @manveru:matrix.org
i don't know how C++ works unfortunately :( even c++ programmers don't know how it works ;-) | 22:46:36 |
 pamplemousse | I am trying to compile nix a non nixos (ubuntu) using clang.
I am using nix develop .#nix-clang11Stdenv (thanks to https://github.com/NixOS/nix/pull/4996) , but I get plenty of errors when linking src/libutil/libnixutil.so
/nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/ld: src/libutil/libnixutil.so.p/archive.cc.o: in function `nix::dump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Sink&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)>&)':
/src/nix/build/../src/libutil/archive.cc:73: undefined reference to `std::allocator<char>::allocator()'
[... and many more…]
Feels like the flake.nix might be missing stdlib, but I tried adding libcxx without success, and libstdcxx5 fails to build.
| 23:52:00 |
| 10 Aug 2021 |
|  anubhavkini left the room. | 05:51:17 |
 Las | What is the point of the GitHub fetcher? I tested the speed difference compared to the Git fetcher and it wasn't so different. | 12:34:16 |
 sterni | I think it performs much worse for big repos | 12:35:20 |
