Content-Length: 284804 | pFad | http://github.com/nextstrain/forecasts-ncov/issues/38

CB TypeError: Object of type ArrayImpl is not JSON serializable · Issue #38 · nextstrain/forecasts-ncov · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: Object of type ArrayImpl is not JSON serializable #38

Open
corneliusroemer opened this issue May 3, 2023 · 5 comments
Open

TypeError: Object of type ArrayImpl is not JSON serializable #38

corneliusroemer opened this issue May 3, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@corneliusroemer
Copy link
Member

corneliusroemer commented May 3, 2023 •
edited
Loading

Current Behavior

When executing the workflow using my ambient nextstrain conda environment, I get the following error:

$ nextstrain build --ambient . --configfile config/config.yaml --config data_provenances=gisaid variant_classification=nextstrain_clades geo_resolutions=global
...
# Relevant snakemake rule:
python -u ./scripts/run-mlr-model.py             --config config/mlr-config.yaml             --seq-path data/gisaid/nextstrain_clades/global/prepared_seq_counts.tsv             --export-path results/gisaid/nextstrain_clades/global/mlr             --pivot '22B (Omicron)'             --data-name 2023-05-03 2>&1 | tee logs/gisaid/nextstrain_clades/global/mlr/2023-05-03.txt

# Fails with:
sample: 100%|██████████| 400/400 [00:31<00:00, 12.75it/s, 511 steps of size 7.34e-03. acc. prob=0.94] 
Traceback (most recent call last):
  File "/Users/corneliusromer/code/forecasts-ncov/./scripts/run-mlr-model.py", line 272, in <module>
    multi_posterior = fit_models(
  File "/Users/corneliusromer/code/forecasts-ncov/./scripts/run-mlr-model.py", line 151, in fit_models
    posterior.save_posterior(f"{path}/models/{location}.json")
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/site-packages/evofr/posterior/posterior_handler.py", line 37, in save_posterior
    json.dump(self.samples, file, cls=EvofrEncoder)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/site-packages/evofr/posterior/posterior_helpers.py", line 163, in default
    return json.JSONEncoder.default(self, obj)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ArrayImpl is not JSON serializable

Expected behavior

No error

How to reproduce

Steps to reproduce the current behavior:

  1. Execute nextstrain build . --configfile config/config.yaml --config data_provenances=gisaid variant_classification=nextstrain_clades geo_resolutions=global
  2. See error

Versions

macOS 13.3.1 (ARM)

Full env details 
$ nextstrain version --verbose
nextstrain.cli 6.2.1

Python
  /opt/homebrew/Caskroom/miniforge/base/envs/nextstrain/bin/python3.10
  3.10.8 | packaged by conda-forge | (main, Nov 22 2022, 08:31:57) [Clang 14.0.6 ]

Runners
  ambient (default)
    augur 21.1.0
    auspice 2.45.1

$ pip list
Package                       Version
----------------------------- ---------
absl-py                       1.4.0
aiobotocore                   2.4.2
aioeasywebdav                 2.4.0
aiohttp                       3.8.4
aioitertools                  0.11.0
aiosignal                     1.3.1
amply                         0.1.5
appdirs                       1.4.4
appnope                       0.1.3
asttokens                     2.2.1
async-timeout                 4.0.2
attmap                        0.13.2
attrs                         22.2.0
autopep8                      2.0.2
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
backrefs                      5.2
bcbio-gff                     0.6.9
bcrypt                        3.2.2
biopython                     1.80
black                         23.3.0
blackjax                      0.9.6
boltons                       23.0.0
boto3                         1.24.59
botocore                      1.27.59
bracex                        2.2.1
brotlipy                      0.7.0
bx-python                     0.9.0
cachetools                    5.3.0
certifi                       2022.12.7
cffi                          1.15.1
charset-normalizer            2.1.1
click                         8.1.3
colorama                      0.4.6
comm                          0.1.3
commonmark                    0.9.1
conda                         23.3.1
conda-package-handling        2.0.2
conda_package_streaming       0.7.0
ConfigArgParse                1.5.3
connection-pool               0.0.3
constellations                0.1.10
contourpy                     1.0.7
crc32c                        2.3.post0
cryptography                  39.0.0
cvxopt                        1.3.0
cycler                        0.11.0
dataclasses                   0.8
datrie                        0.8.2
debugpy                       1.6.7
decorator                     5.1.1
deepdiff                      6.3.0
defusedxml                    0.7.1
distlib                       0.3.6
docutils                      0.19
dpath                         2.1.5
dropbox                       11.36.0
entrypoints                   0.4
epiweeks                      2.2.0
evofr                         0.1.18
exceptiongroup                1.1.1
executing                     1.2.0
fasteners                     0.17.3
fastjsonschema                2.16.3
fastprogress                  1.0.3
filechunkio                   1.8
filelock                      3.12.0
fonttools                     4.39.3
frozenlist                    1.3.3
fsspec                        2023.4.0
ftputil                       5.0.4
future                        0.18.3
gitdb                         4.0.10
GitPython                     3.1.31
google-api-core               2.10.0
google-api-python-client      2.86.0
google-auth                   2.17.3
google-auth-httplib2          0.1.0
google-cloud-core             2.3.2
google-cloud-storage          2.8.0
google-crc32c                 1.1.2
google-resumable-media        2.5.0
googleapis-common-protos      1.57.0
grpcio                        1.46.3
httplib2                      0.22.0
humanfriendly                 10.0
idna                          3.4
importlib-metadata            6.6.0
importlib-resources           5.12.0
iniconfig                     2.0.0
ipdb                          0.13.13
ipykernel                     6.22.0
ipython                       8.7.0
isal                          1.1.0
isodate                       0.6.1
isort                         5.12.0
jax                           0.4.1
jaxlib                        0.3.22
jaxopt                        0.5.5
jedi                          0.18.2
Jinja2                        3.1.2
jmespath                      1.0.1
joblib                        1.2.0
jsonpatch                     1.32
jsonpointer                   2.0
jsonschema                    3.2.0
jupyter_client                8.2.0
jupyter_core                  5.3.0
kiwisolver                    1.4.4
libcst                        0.4.9
libmambapy                    1.2.0
llist                         0.7.1
logmuse                       0.2.6
mamba                         1.2.0
markdown-it-py                2.2.0
MarkupSafe                    2.1.2
matplotlib                    3.7.1
matplotlib-inline             0.1.6
mdurl                         0.1.0
memory-profiler               0.61.0
MonkeyType                    23.3.0
multidict                     6.0.4
multipledispatch              0.6.0
munkres                       1.1.4
mypy                          1.2.0
mypy-extensions               1.0.0
natsort                       8.3.1
nbformat                      5.8.0
nest-asyncio                  1.5.6
networkx                      2.8.8
nextstrain-augur              21.1.0
nextstrain-cli                6.2.1
nodeenv                       1.7.0
numpy                         1.24.3
numpyro                       0.11.0
nwkfmt                        0.1.1
oauth2client                  4.1.3
opt-einsum                    3.3.0
ordered-set                   4.1.0
orjson                        3.8.11
packaging                     23.1
pandas                        1.5.3
pango-aliasor                 0.3.0
pango-designation             1.19
paramiko                      3.1.0
parso                         0.8.3
pathspec                      0.11.1
peppy                         0.35.5
pexpect                       4.8.0
phylo-treetime                0.9.6
pickleshare                   0.7.5
Pillow                        9.2.0
pip                           23.1.2
pipenv                        2023.4.29
plac                          1.3.5
platformdirs                  3.5.0
pluggy                        1.0.0
ply                           3.11
polars                        0.17.11
pooch                         1.7.0
prettytable                   3.7.0
prompt-toolkit                3.0.38
protobuf                      3.20.3
psutil                        5.9.5
ptpython                      3.0.20
ptyprocess                    0.7.0
PuLP                          2.7.0
pure-eval                     0.2.2
py                            1.11.0
pyasn1                        0.4.8
pyasn1-modules                0.2.7
pycodestyle                   2.10.0
pycosat                       0.6.4
pycparser                     2.21
pyfastx                       0.8.4
Pygments                      2.15.1
pygraphviz                    1.10
PyJWT                         2.6.0
pyllist                       0.3
PyNaCl                        1.5.0
pyOpenSSL                     23.1.1
pyparsing                     3.0.9
pyright                       1.1.306
pyrsistent                    0.19.3
pysam                         0.20.0
pysftp                        0.2.9
PySocks                       1.7.1
pytest                        7.3.1
python-dateutil               2.8.2
python-irodsclient            1.1.6
python-lzo                    1.14
pytz                          2023.3
pyu2f                         0.1.5
PyYAML                        6.0
pyzmq                         25.0.2
ratelimiter                   1.2.0
regex                         2023.5.4
requests                      2.29.0
reretry                       0.11.8
retry                         0.9.2
rich                          13.3.5
rsa                           4.9
ruamel.yaml                   0.17.22
ruamel.yaml.clib              0.2.7
s3fs                          2023.3.0
s3transfer                    0.6.0
scikit-learn                  1.2.2
scipy                         1.10.1
setuptools                    67.7.2
setuptools-scm                7.1.0
shellingham                   1.5.1
six                           1.16.0
slacker                       0.14.0
smart-open                    6.3.0
smmap                         3.0.5
snakefmt                      0.8.4
snakemake                     7.25.2
stack-data                    0.6.2
stone                         3.3.1
stopit                        1.1.2
tabulate                      0.9.0
threadpoolctl                 3.1.0
throttler                     1.2.1
toml                          0.10.2
tomli                         2.0.1
toolz                         0.12.0
toposort                      1.10
tornado                       6.3
tqdm                          4.65.0
traitlets                     5.9.0
typer                         0.9.0
typing_extensions             4.5.0
typing-inspect                0.8.0
ubiquerg                      0.6.2
unicodedata2                  15.0.0
uritemplate                   4.1.1
urllib3                       1.26.15
veracitools                   0.1.3
virtualenv                    20.23.0
virtualenv-clone              0.5.4
wcmatch                       8.3
wcwidth                       0.2.6
wheel                         0.40.0
wrapt                         1.15.0
xopen                         1.7.0
xxhash                        0.0.0
yarl                          1.9.1
yte                           1.5.1
zipp                          3.15.0
zstandard                     0.19.0
@corneliusroemer corneliusroemer added the bug Something isn't working label May 3, 2023
@corneliusroemer
Copy link
Member Author

Interestingly, when running with nextstrain managed conda environment it doesn't crash:

nextstrain build --conda .  --configfile config/config.yaml --config data_provenances=gisaid variant_classification=nextstrain_clades geo_resolutions=global

These are the env details for the managed conda:

Managed conda env pip list 
Package                       Version
----------------------------- ----------
absl-py                       1.4.0
aiobotocore                   2.5.0
aioeasywebdav                 2.4.0
aiohttp                       3.8.4
aioitertools                  0.11.0
aiosignal                     1.3.1
amply                         0.1.5
appdirs                       1.4.4
async-timeout                 4.0.2
attmap                        0.13.2
attrs                         22.2.0
awscli                        1.27.76
backports.functools-lru-cache 1.6.4
backrefs                      5.2
bcbio-gff                     0.6.9
bcrypt                        3.2.2
biopython                     1.80
blackjax                      0.9.6
boto3                         1.26.76
botocore                      1.29.76
bracex                        2.2.1
brotlipy                      0.7.0
bx-python                     0.9.0
cachetools                    5.3.0
certifi                       2022.12.7
cffi                          1.15.1
chardet                       4.0.0
charset-normalizer            2.1.1
colorama                      0.4.4
ConfigArgParse                1.5.3
connection-pool               0.0.3
contourpy                     1.0.7
crc32c                        2.3.post0
cryptography                  40.0.2
cvxopt                        1.3.0
cycler                        0.11.0
datrie                        0.8.2
decorator                     5.1.1
defusedxml                    0.7.1
DendroPy                      4.5.2
docutils                      0.15.2
dpath                         2.1.5
dropbox                       11.36.0
epiweeks                      2.1.4
evofr                         0.1.18
exceptiongroup                1.1.1
fasteners                     0.17.3
fastjsonschema                2.16.3
fastprogress                  1.0.3
filechunkio                   1.8
filelock                      3.11.0
fonttools                     4.39.3
frozenlist                    1.3.3
fsspec                        2023.4.0
ftputil                       5.0.4
gitdb                         4.0.10
GitPython                     3.1.31
google-api-core               2.11.0
google-api-python-client      2.85.0
google-auth                   2.17.3
google-auth-httplib2          0.1.0
google-cloud-core             2.3.2
google-cloud-storage          2.8.0
google-crc32c                 1.1.2
google-resumable-media        2.4.1
googleapis-common-protos      1.57.1
grpcio                        1.52.1
httplib2                      0.22.0
humanfriendly                 10.0
idna                          3.4
importlib-metadata            6.4.1
importlib-resources           5.12.0
iniconfig                     2.0.0
isal                          1.1.0
isodate                       0.6.1
jax                           0.4.8
jaxlib                        0.4.7
jaxopt                        0.5.5
Jinja2                        3.1.2
jmespath                      1.0.1
jsonschema                    3.2.0
jupyter_core                  5.3.0
kiwisolver                    1.4.4
logmuse                       0.2.6
markdown-it-py                2.2.0
MarkupSafe                    2.1.2
matplotlib                    3.7.1
mdurl                         0.1.0
ml-dtypes                     0.1.0
multidict                     6.0.4
multipledispatch              0.6.0
munkres                       1.1.4
nbformat                      5.8.0
networkx                      2.8.8
nextstrain-augur              21.1.0
nextstrain-cli                6.2.1
numpy                         1.24.2
numpyro                       0.11.0
oauth2client                  4.1.3
opt-einsum                    3.3.0
packaging                     23.1
pandas                        1.5.3
pangoLEARN                    2022-03-22
paramiko                      3.1.0
peppy                         0.35.5
phylo-treetime                0.9.6
Pillow                        9.5.0
pip                           23.1
plac                          1.3.5
platformdirs                  3.2.0
pluggy                        1.0.0
ply                           3.11
pooch                         1.7.0
prettytable                   3.7.0
protobuf                      4.21.12
psutil                        5.9.4
PuLP                          2.7.0
pyasn1                        0.4.8
pyasn1-modules                0.2.7
pycparser                     2.21
pyfastx                       0.8.4
Pygments                      2.15.0
PyJWT                         2.6.0
PyNaCl                        1.5.0
pyOpenSSL                     23.1.1
pyparsing                     3.0.9
pyrsistent                    0.19.3
pysftp                        0.2.9
PySocks                       1.7.1
pytest                        7.3.1
python-dateutil               2.8.2
python-irodsclient            1.1.6
python-lzo                    1.14
pytools                       2020.1
pytz                          2023.3
pyu2f                         0.1.5
PyYAML                        5.4.1
ratelimiter                   1.2.0
requests                      2.28.2
reretry                       0.11.8
rich                          13.3.4
rsa                           4.7.2
s3fs                          2023.4.0
s3transfer                    0.6.0
scipy                         1.10.1
setuptools                    67.6.1
setuptools-scm                7.1.0
six                           1.16.0
slacker                       0.14.0
smart-open                    6.3.0
smmap                         3.0.5
snakemake                     7.25.0
stone                         3.3.1
stopit                        1.1.2
tabulate                      0.9.0
throttler                     1.2.1
tomli                         2.0.1
toposort                      1.10
tqdm                          4.65.0
traitlets                     5.9.0
typing_extensions             4.5.0
ubiquerg                      0.6.2
unicodedata2                  15.0.0
uritemplate                   4.1.1
urllib3                       1.26.15
veracitools                   0.1.3
wcmatch                       8.3
wcwidth                       0.2.6
wheel                         0.40.0
wrapt                         1.15.0
xopen                         1.7.0
yarl                          1.8.2
yte                           1.5.1
zipp                          3.15.0
zstandard                     0.19.0

@corneliusroemer
Copy link
Member Author

Could it be due to differences in version of jax(lib)? Left working, right not working.

jax    0.4.8     | jax     0.4.1
jaxlib 0.4.7     | jaxlib  0.3.22

@corneliusroemer
Copy link
Member Author

Can't upgrade in my env because of some grpc-cpp and libgrpc clash. Some packages require grpc-cpp but the conda-forge recipe for that is behind libgrpc - yet libgrpc constrains grpc-cpp.

@corneliusroemer
Copy link
Member Author

This seems to be the problem, usher requires libprotobuf >=3.20.2,<3.21.0a0 which means one can use at most grpc-cpp 1.47.1 but that limits the jax version one can use. I guess I can uninstall usher, but heads up to @AngieHinrichs that this could be worth bumping.

@corneliusroemer
Copy link
Member Author

So now that I've uninstalled usher and updated jax to 0.4.8 it also runs with my ambient environment.

Lesson could be that evofr requires more stringent minimal version constraints for jax @marlinfiggins

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
No open projects
Nextstrain planning (archived)
Status: New
Milestone
No milestone
Development

No branches or pull requests
1 participant
@corneliusroemer








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/nextstrain/forecasts-ncov/issues/38

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy