-
-
Notifications
You must be signed in to change notification settings - Fork 10.9k
Unnecessary dependence in fromfile on the ability to seek into a file #28840
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
That would require memory resizing, etc. which |
Well, it seems - naively - that reading raw, then calling frombuffer, would mean requiring twice the memory, at least temoprarily... but regardless - the point is that its not justified for NumPy to fail. That is, the fact that the user of numpy could have gone to the effort of checking the seekability of the file and writing something differently is not reason enough for refusing to perform the read within the library itself. |
Just for the record (not 100% sure you are aware): There is no additional copy involed. |
Ok, that's a nice optimization, kudos... I'm not a Pythonista, so I wouldn't know. However, in that case - why not just write the few lines of raw reading and a frombuffer invocation, in case you've discovered that the file is unseekable? It seems like little effort on your part, based on what you've said; it rounds out the API; it doesn't fill your codebase with only-used-once functionality... seems like a win. |
Why not just improve the documentation by clearly stating the file is a raw binary file and not a buffered file? I think the same issue occurs with Lines 837 to 848 in d32cf93
This would avoid confusion and close the issue. If a maintainer agree with that I can do the PR. Otherwise, it would be better to tag this issue as an enhancement. |
I think that sounds great, it doesn't mean we can't ever extend it (although if we point to alternatives, I am not sure I think that it is necessary). |
If, in my script, my input file is a PIPE, e.g. I call my script like so:
and my script contains the line:
then - NumPy fails,
when I run the script, I get:
why is NumPy not "falling back" on reading individual values, without knowing the overall number of them?
The text was updated successfully, but these errors were encountered: