-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Faster implementation of maps #160
Comments
Yeah, it's a well known fact that the pmap is a lot slower than the corresponding built in map. I would not use it for any type of performance critical part of a system. I'd love to see a C-implementation or an implementation with a very thin wrapper on top of an existing C/C++ implementation like one of the linked ones included in pyrsistent. Writing an implementation from scratch in C requires a fair effort though. Adopting one of the existing implementations may be an option, but making it compatible with the current implementation and the different inherited types would require some thinking. Happy to take PRs on this! It's unlikely that I will engage in such an effort at this time since I don't have a personal need for it and have very limited time for hobby coding (which this would be) at the moment. |
Definitely understand you don't have time for this, just thought it'd be good to have somewhere to track this. FWIW, Immer seems to support the fancier stuff Pyrsistent does (temporarily-mutable objects), so I suspect it could be made to work with less trouble. |
Erlang added some new implementations of some of their data structures a couple of years ago, the same is used for dictionary and sets. http://www1.erlang.org/doc/man/gb_trees.html Prof. Arne Andersson's General Balanced Trees http://user.it.uu.se/~arnea/abs/gbimpl.html You find more on http://user.it.uu.se/~arnea/publications.html |
Thanks to both of you for the pointers above! |
Hi! Immer author here :) Transients for maps are being implemented right now. I do believe that Immer is the fastest implementation of HAMTs and RRBTS out there (I tested the later claim at a ICFP paper, where I also benchmarked it against the C-backed Pyrsistent vectors). I would be very happy to have Immer become the backend for Pyrsistent, and would love to help with reviewing the code! |
Also keen on this. Is https://github.com/MagicStack/immutables an option for this as a backend @tobgu? It's already a Python package so would be easier to manage. |
@samuelsinayoko Immer does include in it source tree some experiments binding For Maps there may be some other subtleties, but I think that could serve as good inspiration. Immer maps now support transients (very efficiently!) and also support structural-sharing-aware diffing, which has lots of super cool applications. Also Immer vectors support logarithmic concatenatio and slicing, features that Pyrsistent currently does not support. So I think using Immer as a backend for Pyrsistent is actually a fantastic idea! |
I recently had to rip out some pyrsistent code because using it was 10× slower than regular Python data structures. It's probably possible to do better than current implementation.
Some possibilities:
The text was updated successfully, but these errors were encountered: