Skip to content

fix pgml conflicts with plpython #931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
fix pgml conflicts with plpython
pgml is not compatible with plpython, if using both pgml and plpython in the
same session, postgresql will crash.

minimum reproducible code:

```sql
SELECT pgml.embed('intfloat/e5-small', 'hi mom');

create or replace function pyudf()
returns int as
$$
return 0
$$ language 'plpython3u';
```

the call stack:

```
 Stack trace of thread 161970:
 #0  0x00007efc1429edb8 PyImport_Import (libpython3.9.so.1.0 + 0x9edb8)
 #1  0x00007efc1429f125 PyImport_ImportModule (libpython3.9.so.1.0 + 0x9f125)
 #2  0x00007efb04b0f496 n/a (plpython3.so + 0x10496)
 #3  0x00007efb04b1039d plpython3_validator (plpython3.so + 0x1139d)
 #4  0x0000559d0cdbc5c2 OidFunctionCall1Coll (postgres + 0x6465c2)
 #5  0x0000559d0c9d68bb ProcedureCreate (postgres + 0x2608bb)
 #6  0x0000559d0ca5030c CreateFunction (postgres + 0x2da30c)
 #7  0x0000559d0ce1c730 n/a (postgres + 0x6a6730)
 #8  0x0000559d0cc5a030 standard_ProcessUtility (postgres + 0x4e4030)
 #9  0x0000559d0cc545ed n/a (postgres + 0x4de5ed)
 #10 0x0000559d0cc546e7 n/a (postgres + 0x4de6e7)
 #11 0x0000559d0cc54beb PortalRun (postgres + 0x4debeb)
 #12 0x0000559d0cc55249 n/a (postgres + 0x4df249)
 #13 0x0000559d0cc576f0 PostgresMain (postgres + 0x4e16f0)
 #14 0x0000559d0cbc3e9c n/a (postgres + 0x44de9c)
 #15 0x0000559d0cbc50aa PostmasterMain (postgres + 0x44f0aa)
 #16 0x0000559d0c8ce7d2 main (postgres + 0x1587d2)
 #17 0x00007efc18427cd0 n/a (libc.so.6 + 0x27cd0)
 #18 0x00007efc18427d8a __libc_start_main (libc.so.6 + 0x27d8a)
 #19 0x0000559d0c8cee15 _start (postgres + 0x158e15)
```

this is because PostgreSQL is using dlopen(RTLD_GLOBAL). this will parse some
of symbols into the previous opened .so file, but the others will use a
relative offset in pgml.so, and will cause a null-pointer crash.

this commit hide all symbols except the UDF symbols (ends with `_wrapper`) and
the magic symbols (`_PG_init` `Pg_magic_func`). so dlopen(RTLD_GLOBAL) will
parse the symbols to the correct position.
  • Loading branch information
Sasasu committed Aug 18, 2023
commit 16d32d82dc9350e3125261185293fe79fd7fc211
6 changes: 3 additions & 3 deletions pgml-extension/.cargo/config
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[build]
# Postgres symbols won't be available until runtime
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup"]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup,-fuse-ld=lld"]

[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup,-fuse-ld=/usr/bin/mold"]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup,-fuse-ld=lld"]

[target.aarch64-unknown-linux-gnu]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup,-fuse-ld=/usr/bin/mold"]
rustflags = ["-C", "link-args=-Wl,-undefined,dynamic_lookup,-fuse-ld=lld"]
12 changes: 12 additions & 0 deletions pgml-extension/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,16 @@ fn main() {
println!("cargo:rustc-link-search=/opt/homebrew/opt/openblas/lib");
println!("cargo:rustc-link-search=/opt/homebrew/opt/libomp/lib");
}

// PostgreSQL is using dlopen(RTLD_GLOBAL). this will parse some
// of symbols into the previous opened .so file, but the others will use a
// relative offset in pgml.so, and will cause a null-pointer crash.
//
// hid all symbol to avoid symbol conflicts.
//
// append mode (link-args) only works with clang ld (lld)
println!(
"cargo:link-args=-Wl,--version-script={}/ld.map",
std::env::current_dir().unwrap().to_string_lossy(),
);
}
8 changes: 8 additions & 0 deletions pgml-extension/ld.map
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
global:
Pg_magic_func;
_PG_init;
*_wrapper;
local:
*;
};
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy