You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have been trying to write a multiprocessing function that works with headers and WCS, however I am running into an issue. I think the issue is with the method wcs.array_index_to_world, which returns something different to what I expect.
Expected behavior
No response
How to Reproduce
Here is a reduced version of the code I am running:
To use concurrent.ProcessPoolExecutor I am having to convert headers to strings (so they can be pickled) and then try rebuildling the header in the functions. The header is passed to WCS as I want to get spatial coordinates of the data.
The wcs object appears to be as it should (at least from a naiive print(wcs) inside the example function). However, after calling the method array_index_to_world I get a different result than when running the code without multiprocessing.
In single processing, xy has length 2 and xy[1] is a SkyCoord. The xy[0] is a SpectralCoord.
During multiprocessing, the variable xy is of length 3 and and the elements of the list are now Quantity.
The issue goes away if I swap concurrent.ProcessPoolExecutor for concurrent.ThreadPoolExecutor.
I don't really have a minimum working example of the issue, but happy to provide a fits file that this occurs on.
Versions
The version of astropy I am using is 6.17
The text was updated successfully, but these errors were encountered:
Richardjmorton
changed the title
Problem with creating fits headers during mutliprocessing
Potential problem with a wsc method during mutliprocessing
May 14, 2025
If you cannot provide minimally reproducible example, would still be nice to give us the input/output values for one problematic case you are seeing. Since the pseudocode provided is not really the whole picture, maybe the actual problem is not even presented here. My suspicion is that something you thought was pickled is not behaving as you thought it is, since it gives problem in process pool but not thread pool.
But before that, I recommend you first upgrade to use astropy v7.1rc1 (pip install astropy --pre -U) as we no longer support the v6.x series.
Thank you.
pllim
changed the title
Potential problem with a wsc method during mutliprocessing
Potential problem with a wsc method during multiprocessing
May 14, 2025
Uh oh!
There was an error while loading. Please reload this page.
Description
I have been trying to write a multiprocessing function that works with headers and
WCS
, however I am running into an issue. I think the issue is with the methodwcs.array_index_to_world
, which returns something different to what I expect.Expected behavior
No response
How to Reproduce
Here is a reduced version of the code I am running:
To use
concurrent.ProcessPoolExecutor
I am having to convert headers to strings (so they can be pickled) and then try rebuildling the header in the functions. The header is passed toWCS
as I want to get spatial coordinates of the data.The wcs object appears to be as it should (at least from a naiive
print(wcs)
inside the example function). However, after calling the methodarray_index_to_world
I get a different result than when running the code without multiprocessing.In single processing,
xy
has length 2 andxy[1]
is aSkyCoord
. Thexy[0]
is aSpectralCoord
.During multiprocessing, the variable
xy
is of length 3 and and the elements of the list are nowQuantity
.The issue goes away if I swap
concurrent.ProcessPoolExecutor
forconcurrent.ThreadPoolExecutor
.I don't really have a minimum working example of the issue, but happy to provide a fits file that this occurs on.
Versions
The version of astropy I am using is 6.17
The text was updated successfully, but these errors were encountered: