Skip to content

General issue: Cannot upgrade database #4034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
JeffJerseyCow opened this issue Aug 7, 2020 · 4 comments
Closed

General issue: Cannot upgrade database #4034

JeffJerseyCow opened this issue Aug 7, 2020 · 4 comments
Labels
question Further information is requested

Comments

@JeffJerseyCow
Copy link

I've been following the guidance on how to configure CodeQL, create a database and analyze with a query pack. But I always end up with the following error:

A fatal error occurred: This CodeQL database is not compatible with the QL library /home/chris/projects/codeql/codeql-repo/python/ql/src/Security/CWE-502/UnsafeDeserialization.ql uses. You may need to run 'codeql database upgrade testDb'.

It doesn't matter what version of CodeQL I'm using, how many times I update the database or the queries I use. It's always the same error.

It's the case for the CPP, JavaScript and Python queries.

My directory that holes everything CodeQL is here.

$pwd
/home/chris/projects/codeql

The CodeQL version is 2.2.4 (I've also tried it with earlier versions) and is moved to a codeql-cli directory.

$curl -LO https://github.com/github/codeql-cli-binaries/releases/download/v2.2.4/codeql-linux64.zip
$unzip codeql-linux64.zip 
$mv codeql codeql-cli

I use the latest queries extracted to codeql-repo.

$ git clone https://github.com/github/codeql.git codeql-repo

I create the database and put it in dukDatabase.

codeql database create dukDatabase --language=cpp --command=make --source-root=../duktape

Then I try to analyse with a query, or query pack and it always fails.

codeql database analyze dukDatabase/ codeql-repo/cpp/ql/src/codeql-suites/cpp-lgtm.qls --format=sarifv2 --output=duk.sarifv2 --search-path=codeql-repo/

I always get an error complaining about not finding dependencies, but I add --search-path to resolve it. But I always get the error about an incompatible database.

I've followed the guidance to update the database, it just says it's already updated.

It doesn't matter what version of codeql I used or queries...it never works.

Any help is appreciated.
@JeffJerseyCow JeffJerseyCow added the question Further information is requested label Aug 7, 2020
@RasmusWL
Copy link
Member

Hi @JeffJerseyCow , sorry to hear you're having trouble with upgrading the database 😐

I reproduced your setup with codeql-cli and codeql-repo folders, but was not able to reproduce your upgrading problems.

Can you create a dummy DB like this

$ mkdir src-test
$ echo "print(42)" > src-test/wat.py
$ ./codeql-cli/codeql database create --language python --source-root src-test/ test-db

and show the output from running

$ ./codeql-cli/codeql database upgrade -vv test-db

@carbureted
Copy link

carbureted commented Aug 19, 2020 •
edited
Loading

I'm having the same issue (also with a C++ codebase) on OSX 10.14.6. Results of the above commands are below - looks like some dependencies might be missing?

codeql database create --language python --source-root src-test/ test-db
Initializing database at ./test-db.
Running command [codeql/python/tools/autobuild.sh] in ./src-test.
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash md5 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type md5
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash sha1 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type sha1
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash sha224 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type sha224
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash sha256 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type sha256
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash sha384 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type sha384
[2020-08-19 16:26:44] [build-err] ERROR:root:code for hash sha512 was not found.
[2020-08-19 16:26:44] [build-err] Traceback (most recent call last):
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 147, in <module>
[2020-08-19 16:26:44] [build-err]     globals()[__func_name] = __get_hash(__func_name)
[2020-08-19 16:26:44] [build-err]   File "/usr/local/Cellar/python@2/2.7.15_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/hashlib.py", line 97, in __get_builtin_constructor
[2020-08-19 16:26:44] [build-err]     raise ValueError('unsupported hash type ' + name)
[2020-08-19 16:26:44] [build-err] ValueError: unsupported hash type sha512
[2020-08-19 16:26:44] [build] Calling python3 python/tools/get_venv_lib.py
[2020-08-19 16:26:44] [build] Calling python3 -S python/tools/python_tracer.py -v -z all -c ./test-db/working/trap_cache -p /usr/local/lib/python3.8/site-packages -R ./src-test
[2020-08-19 16:26:45] [build] [INFO] Python version 3.8.5
[2020-08-19 16:26:45] [build] [INFO] Python extractor version 5.11
[2020-08-19 16:26:46] [build] [INFO] [4] Extracted file ./src-test/wat.py in 9ms
[2020-08-19 16:26:46] [build] [INFO] Processed 1 modules in 1.04s

@RasmusWL
Copy link
Member

Hi @carbureted , although those are error messages, it doesn't look critical, and it seems like the database was created succesfully. What problems are you seeing when running queries against it? (such as ./codeql-cli/codeql query run --database test-db codeql-repo/python/ql/examples/snippets/backticks.ql)

@aeisenberg aeisenberg mentioned this issue Apr 12, 2021
Merged
@jketema
Copy link
Contributor

jketema commented May 22, 2025

Closing. It's not likely that the user will still answer after 5 years. If this is still a problem, please open a new issue.

@jketema jketema closed this as completed May 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@carbureted @RasmusWL @JeffJerseyCow @jketema
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy