Skip to content

[Bug]: paddle_ops cpu 编译报错 #10199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
1 task done
anyu5733 opened this issue Mar 19, 2025 · 6 comments
Open
1 task done

[Bug]: paddle_ops cpu 编译报错 #10199

anyu5733 opened this issue Mar 19, 2025 · 6 comments
Assignees
@ZHUI
Labels
bug Something isn't working stale

Comments

@anyu5733
Copy link

软件环境

- paddlepaddle:3.0.0.dev20250318
- paddlepaddle-gpu: 
- paddlenlp: 3.0.0b4.post20250318

重复问题

  • I have searched the existing issues

错误描述

/data/server/test/PaddleNLP/csrc/cpu/src/stop_generation_multi_ends.cc:18:10: fatal error: helper.h: No such file or directory
   18 | #include "helper.h"
      |          ^~~~~~~~~~
compilation terminated.
/data/server/test/PaddleNLP/csrc/cpu/src/stop_generation_multi_ends.cc compile failed, command '/opt/gcc-9.4.0/bin/g++' failed with exit code 1
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/stop_generation_multi_ends.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc:18:1: error: ‘git’ does not name a type; did you mean ‘digit’?
   18 | git adtemplate <typename T>
      | ^~~
      | digit
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc: In function ‘void AvxWeightOnly(const Tensor&, const Tensor&, bool, std::string, paddle::Tensor&)’:
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc:98:5: error: ‘AvxCompute’ was not declared in this scope
   98 |     AvxCompute<T>(x,
      |     ^~~~~~~~~~
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc:98:17: error: expected primary-expression before ‘>’ token
   98 |     AvxCompute<T>(x,
      |                 ^
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc:109:5: error: ‘AvxCompute’ was not declared in this scope
  109 |     AvxCompute<T>(x,
      |     ^~~~~~~~~~
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc:109:17: error: expected primary-expression before ‘>’ token
  109 |     AvxCompute<T>(x,
      |                 ^
/data/server/test/PaddleNLP/csrc/cpu/src/avx_weight_only.cc compile failed, command '/opt/gcc-9.4.0/bin/g++' failed with exit code 1
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/avx_weight_only.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/xft_greedy_search.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/set_value_by_flags.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/token_penalty_multi_scores.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/save_with_output.o is compiled
/data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/xft_transformer.o is compiled
[2025-03-19 15:16:59,365] [    INFO] spawn.py:60 - g++ -pthread -B /data/server/app/miniconda3/compiler_compat -fno-strict-overflow -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /data/server/app/miniconda3/include -fPIC -O2 -isystem /data/server/app/miniconda3/include -pthread -B /data/server/app/miniconda3/compiler_compat -shared -Wl,-rpath,/data/server/app/miniconda3/lib -Wl,-rpath-link,/data/server/app/miniconda3/lib -L/data/server/app/miniconda3/lib -Wl,-rpath,/data/server/app/miniconda3/lib -Wl,-rpath-link,/data/server/app/miniconda3/lib -L/data/server/app/miniconda3/lib /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/save_with_output.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/avx_weight_only.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/set_value_by_flags.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/stop_generation_multi_ends.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/token_penalty_multi_scores.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/xft_greedy_search.o /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/xft_transformer.o -L/data/server/app/miniconda3/lib/python3.12/site-packages/paddle/base -L/data/server/test/PaddleNLP/csrc/cpu/xFasterTransformer/build -L/data/server/app/miniconda3/lib/python3.12/site-packages/paddle/libs -L/data/server/app/miniconda3/lib/python3.12/site-packages/paddle/base -Wl,-R/data/server/app/miniconda3/lib/python3.12/site-packages/paddle/libs -Wl,-R/data/server/app/miniconda3/lib/python3.12/site-packages/paddle/base -l:libxfastertransformer.so -l:libxft_comm_helper.so -o build/paddlenlp_ops/lib.linux-x86_64-cpython-312/paddlenlp_ops.so -l:libpaddle.so
g++: error: /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/avx_weight_only.o: No such file or directory
g++: error: /data/server/test/PaddleNLP/csrc/cpu/build/paddlenlp_ops/lib.linux-x86_64-cpython-312/stop_generation_multi_ends.o: No such file or directory
error: command '/opt/gcc-9.4.0/bin/g++' failed with exit code 1


依照教程，PaddleNLP/csrc/cpu/setup.sh执行最后一步“python ./src/setup_cpu.py install --user”报错，之前xFasterTransformer已成功编译


缺少helper.h文件 以及 git adtemplate <typename T>  AvxCompute<T> 缺失

稳定复现步骤 & 代码

python ./src/setup_cpu.py install --user
@anyu5733 anyu5733 added the bug Something isn't working label Mar 19, 2025
@ZHUI
Copy link
Collaborator

ZHUI commented Mar 20, 2025

辛苦 @bukejiyu 看看

@bukejiyu
Copy link
Contributor

#10218 可以按照pr修改试试

@ZHUI
Copy link
Collaborator

ZHUI commented Mar 20, 2025

pr merged。您可以尝试最新代码。

@anyu5733
Copy link
Author

anyu5733 commented Mar 20, 2025 •
edited by ZHUI
Loading

你好 感谢这么快的更新修复。
更新最新代码后。发现新问题。
与#9583 类似打包时错误。试用python 3.10.16 与python 3.12.9 均失败 setuptools 为 77.0.1
Package zlib-1.2.7-21.el7_9.x86_64 already installed and latest version
Package zlib-devel-1.2.7-21.el7_9.x86_64 already installed and latest version

能提供解决方案或完成编译的环境版本么？

错误详细信息如下：

[2025-03-20 17:58:04,233] [    INFO] bdist_egg.py:187 - creating stub loader for paddlenlp_ops.so
Received len(custom_op) = 7, using custom operator
[2025-03-20 17:58:04,248] [    INFO] util.py:485 - byte-compiling build/paddlenlp_ops/bdist.linux-x86_64/egg/paddlenlp_ops.py to paddlenlp_ops.cpython-310.pyc
[2025-03-20 17:58:04,250] [    INFO] dir_util.py:58 - creating build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,250] [    INFO] file_util.py:130 - copying paddlenlp_ops.egg-info/PKG-INFO -> build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,250] [    INFO] file_util.py:130 - copying paddlenlp_ops.egg-info/SOURCES.txt -> build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,250] [    INFO] file_util.py:130 - copying paddlenlp_ops.egg-info/dependency_links.txt -> build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,251] [    INFO] file_util.py:130 - copying paddlenlp_ops.egg-info/not-zip-safe -> build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,251] [    INFO] file_util.py:130 - copying paddlenlp_ops.egg-info/top_level.txt -> build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO
[2025-03-20 17:58:04,251] [    INFO] bdist_egg.py:210 - writing build/paddlenlp_ops/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
[2025-03-20 17:58:04,251] [    INFO] dir_util.py:58 - creating dist
[2025-03-20 17:58:04,251] [    INFO] bdist_egg.py:459 - creating 'dist/paddlenlp_ops-1.0-py3.10-linux-x86_64.egg' and adding 'build/paddlenlp_ops/bdist.linux-x86_64/egg' to it
Traceback (most recent call last):
  File "/data/server/test/PaddleNLP_new/csrc/cpu/./src/setup_cpu.py", line 129, in <module>
    setup(
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/paddle/utils/cpp_extension/cpp_extension.py", line 247, in setup
    setuptools.setup(**attr)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
    self.run_command(cmd)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/dist.py", line 1096, in run_command
    super().run_command(command)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/command/install.py", line 109, in run
    self.do_egg_install()
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/command/install.py", line 167, in do_egg_install
    self.run_command('bdist_egg')
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 357, in run_command
    self.distribution.run_command(command)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/dist.py", line 1096, in run_command
    super().run_command(command)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
    cmd_obj.run()
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 233, in run
    make_zipfile(
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 474, in make_zipfile
    visit(z, dirname, files)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/site-packages/setuptools/command/bdist_egg.py", line 467, in visit
    z.write(path, p)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/zipfile.py", line 1788, in write
    with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/zipfile.py", line 1528, in open
    return self._open_to_write(zinfo, force_zip64=force_zip64)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/zipfile.py", line 1630, in _open_to_write
    return _ZipWriteFile(self, zinfo, zip64)
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/zipfile.py", line 1115, in __init__
    self._compressor = _get_compressor(zinfo.compress_type,
  File "/data/server/app/miniconda3/envs/nlp/lib/python3.10/zipfile.py", line 691, in _get_compressor
    return zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION, zlib.DEFLATED, -15)
ValueError: Invalid initialization option

@bukejiyu
Copy link
Contributor

setuptools 75.1.0
setuptools-scm 8.2.1
这是我的版本

@github-actions GitHub Actions
Copy link

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动，被标记为stale。

@github-actions github-actions bot added the stale label May 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ZHUI ZHUI

Labels
bug Something isn't working stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ZHUI @anyu5733 @bukejiyu
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy