Skip to content

DNS Challenge with Cloudflare fails #3305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dkhelms opened this issue Nov 6, 2023 · 25 comments
Open

DNS Challenge with Cloudflare fails #3305

dkhelms opened this issue Nov 6, 2023 · 25 comments
Labels

Comments

@dkhelms
Copy link

dkhelms commented Nov 6, 2023

  • Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
    • Yes
  • Are you sure you're not using someone else's docker image?
    • Yes
  • Have you searched for similar issues (both open and closed)?
    • Yes

Describe the bug
I try to use DNS Challenge with Cloudflare to get a cert but it doesn't work.

Nginx Proxy Manager Version
2.10.4

To Reproduce

  1. Go to SSL Certificates
  2. Click Add New SSL Certificate
  3. Choose Let's Encrypt
  4. Use DNS Challenge and Cloudflare as DNS Provider

Expected behavior
For a cert to be issued.

Screenshots

Operating System
I am using Ubuntu 22.04 with the newest version of Portainer

Additional context
Here are the errors:

Error: Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir certbot-dns-cloudflare==$(certbot --version | grep -Eo '[0-9](\.[0-9]+)+') cloudflare && deactivate
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-cloudflare/
ERROR: Could not find a version that satisfies the requirement certbot-dns-cloudflare==2.5.0 (from versions: none)
ERROR: No matching distribution found for certbot-dns-cloudflare==2.5.0

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)```
@dkhelms dkhelms added the bug label Nov 6, 2023
@SamBouwer
Copy link

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

@dkhelms
Copy link
Author

dkhelms commented Nov 14, 2023

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

Yes, I updated the key as well. The problem was the newest version of NPM. I had to go back to 2.9.14 to renew the cert and make everything start working again.

@Doutianbao
Copy link

@dkhelms maybe a stupid question, but did you update the API key in the example that is shown when you select the Cloudflare DNS provider from the dropdown list? If you did update that key with a valid key from your Cloudflare account, can you resolve api.cloudflare.com (for example with "nslookup api.cloudflare.com"?

Yes, I updated the key as well. The problem was the newest version of NPM. I had to go back to 2.9.14 to renew the cert and make everything start working again.

same bug on v2.10.4

@deadblue0910
Copy link

@dkhelms也许是一个愚蠢的问题,但是当您从下拉列表中选择 Cloudflare DNS 提供商时,您是否更新了示例中显示的 API 密钥?如果您确实使用 Cloudflare 帐户中的有效密钥更新了该密钥,您能否解析 api.cloudflare.com(例如使用“nslookup api.cloudflare.com”?

是的,我也更新了密钥。问题出在最新版本的 NPM 上。我必须返回 2.9.14 更新证书并使一切重新开始工作。

v2.10.4 上有同样的错误

Have you solved this problem?also v2.10.4

@dkhelms
Copy link
Author

dkhelms commented Jan 3, 2024

@dkhelms也许是一个愚蠢的问题,但是当您从下拉列表中选择 Cloudflare DNS 提供商时,您是否更新了示例中显示的 API 密钥?如果您确实使用 Cloudflare 帐户中的有效密钥更新了该密钥,您能否解析 api.cloudflare.com(例如使用“nslookup api.cloudflare.com”?

是的,我也更新了密钥。问题出在最新版本的 NPM 上。我必须返回 2.9.14 更新证书并使一切重新开始工作。

v2.10.4 上有同样的错误

Have you solved this problem?also v2.10.4

Sorry, but I never did.

@jc21
Copy link
Member

jc21 commented Jan 10, 2024

You are welcome to try the github-develop docker tag, it's bleeding edge and frankly, I need people to test more DNS providers that I don't use.

It has certbot v2.8.0 (previously was v2.5.0) and also means DNS plugins will be using v2.8.0 as well.

@moviemakr1620
Copy link

@jc21 How do you do that? right now mine is having the same issues and would try anything.

@Wamy-Dev
Copy link

same issue for me, using the docker container.

@MattXcz
Copy link

MattXcz commented Jan 15, 2024

+1 on this... using proxmox lxc, first try failed, second one finished without errors...

@hurz07
Copy link

hurz07 commented Feb 28, 2024

@jc21 - tried GitHub-develp tag, no change for me. Currently running v2.11.1.

I use duckdns.org and run nginxproxymanager in a docker container on synology using portainer.
I created a macvlan network and excluded IPv6.
Everything runs well except creating lets encrypt certificates with duckdns DNS-challenge.

The error message:

CommandError: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
ERROR: Could not find a version that satisfies the requirement certbot-dns-duckdns~=0.9 (from versions: none)
ERROR: No matching distribution found for certbot-dns-duckdns~=0.9

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Looking forward for hints or an solution :-)
Thank you in advance.

@MingfuYAN
Copy link

I solved the problem. My cause was that DNSSEC was not configured correctly, visit this site to see if DNSSEC is configured correctly.

I found this issue by looking directly at the log.

  1. docker exec -it NPM-container-name bash
  2. cat /tmp/letsencrypt-log/letsencrypt.log

@hurz07
Copy link

hurz07 commented Mar 10, 2024

I also tested it in a clean virtual machine with an "own" IP address and it worked.
Think my problem is the use of "macvlan" in docker.

Thank you for your nice work!

@jc21 - tried GitHub-develp tag, no change for me. Currently running v2.11.1.

I use duckdns.org and run nginxproxymanager in a docker container on synology using portainer. I created a macvlan network and excluded IPv6. Everything runs well except creating lets encrypt certificates with duckdns DNS-challenge.

The error message:

CommandError: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/certbot-dns-duckdns/
ERROR: Could not find a version that satisfies the requirement certbot-dns-duckdns~=0.9 (from versions: none)
ERROR: No matching distribution found for certbot-dns-duckdns~=0.9

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Looking forward for hints or an solution :-) Thank you in advance.

@Karstenve
Copy link

Karstenve commented May 30, 2024

I had the same issue and found a lot of open or stale issues around this repo. What I found is that when I tried to manually install the certbot-dns-cloudflare when executing a bash in the docker container, for some reason the container couldn't reach the appropriate packages. What I did is add "network-mode: host" to the docker compose file and after that I could manually install and get the certificate working. I hope this helps people. I'm not going to react to other issues so I hope people find this.

EDIT:
After some more experimenting I found out the npm container didn't have internet access. After looking into the DNS config I found out Tailscale had replaced the DNS info in /etc/resolv.conf which caused DNS issues in all my containers. After fixing this file by disabling the replacement done by tailscale I could rebuild the containers and the issue was fixed. Adding network-mode: host also works but it's not ideal as I needed a bridged network in this case.

@Waldorf3
Copy link

Waldorf3 commented Jun 10, 2024

Getting similar errors here. Suddenly certs stopped being renewed. When trying to renew manually getting this error:

Internal Error
CommandError: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py:107: PendingDeprecationWarning: 
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!   WARNING  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       !! You're seeing this warning because you've upgraded the Python package 'cloudflare' to version  !!
       !! 2.20.* via an automated upgrade without version pinning. Version 2.20.0 exists to catch any    !!
       !! of these upgrades before Cloudflare releases a new major release under the release number 3.x. !!
       !!                                                                                                !!
       !! Should you determine that you need to revert this upgrade and pin to v2.19.* it is recommended !!
       !! you do the following: pip install --upgrade cloudflare==2.19.* or equivilant.                  !!
       !!                                                                                                !!
       !! Or you can upgrade to v3.x. NOTE: Release 3.x will not be code-compatible or call-compatible   !!
       !! with previous releases. To see more about upgrading to next major version, please see:         !!
       !! https://github.com/cloudflare/python-cloudflare/discussions/191                                !!
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  self.cf = CloudFlare.CloudFlare(token=api_token)
/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py:107: PendingDeprecationWarning: 
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!   WARNING  !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
       !! You're seeing this warning because you've upgraded the Python package 'cloudflare' to version  !!
       !! 2.20.* via an automated upgrade without version pinning. Version 2.20.0 exists to catch any    !!
       !! of these upgrades before Cloudflare releases a new major release under the release number 3.x. !!
       !!                                                                                                !!
       !! Should you determine that you need to revert this upgrade and pin to v2.19.* it is recommended !!
       !! you do the following: pip install --upgrade cloudflare==2.19.* or equivilant.                  !!
       !!                                                                                                !!
       !! Or you can upgrade to v3.x. NOTE: Release 3.x will not be code-compatible or call-compatible   !!
       !! with previous releases. To see more about upgrading to next major version, please see:         !!
       !! https://github.com/cloudflare/python-cloudflare/discussions/191                                !!
       !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
  self.cf = CloudFlare.CloudFlare(token=api_token)
Error determining zone_id: 6003 Invalid request headers. Please confirm that you have supplied valid Cloudflare API credentials. (Did you copy your entire API token/key? To use Cloudflare tokens, you'll need the python package cloudflare>=2.3.1. This certbot is running cloudflare 2.20.0)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Don't know anything about updating python, this is a docker container, just pulled the latest available.

@Framdark
Copy link

I am also having this issue error logs say this

`Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-25" --agree-tos --email "email@gmail.com" --domains "*.domain.top,domain.top" --authenticator dns-cloudflare --dns-cloudflare-credentials "/etc/letsencrypt/credentials/credentials-25"
Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

at ChildProcess.exithandler (node:child_process:422:12)
at ChildProcess.emit (node:events:517:28)
at maybeClose (node:internal/child_process:1098:16)
at ChildProcess._handle.onexit (node:internal/child_process:303:5)`

@paradox1612
Copy link

I had the same issue, Need to reinstall pip and pip install cloudflare==2.19.*

https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

@Framdark
Copy link

Framdark commented Jun 29, 2024 via email

@caiocesarstx
Copy link

I had the same problem with Cloudflare plugin:

using NPM on Docker, with docker swarm managing it and using DNS to access the NPM instance.

The log is below:

2024-07-01 20:05:38,016:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 191, in find_all
cls._load_entry_point(entry_point, plugins)
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 203, in _load_entry_point
plugin_ep = PluginEntryPoint(entry_point)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 42, in init
self.plugin_cls: Type[interfaces.Plugin] = entry_point.load()
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/metadata/init.py", line 202, in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1206, in _gcd_import
File "", line 1178, in _find_and_load
File "", line 1149, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 9, in
import CloudFlare
ModuleNotFoundError: No module named 'CloudFlare'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/certbot/bin/certbot", line 8, in
sys.exit(main())
^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
return internal_main.main(cli_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1841, in main
plugins = plugins_disco.PluginsRegistry.find_all()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 193, in find_all
raise errors.PluginError(
certbot.errors.PluginError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
2024-07-01 20:05:38,016:ERROR:certbot._internal.log:The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.

@paradox1612
Copy link

you can do :
pip install --upgrade cloudflare==2.19.*

@Framdark
Copy link

Framdark commented Jul 1, 2024

I unfortunately can't use pip because I am running NPM in home assistant.

@caiocesarstx
Copy link

you can do : pip install --upgrade cloudflare==2.19.*

It worked, thanks!

[7/1/2024] [9:16:54 PM] [Certbot ] › ▶ start Installing cloudflare...
***** omited *******
Waiting 120 seconds for DNS changes to propagate
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/npm-52/fullchain.pem
Key is saved at: /etc/letsencrypt/live/npm-52/privkey.pem
This certificate expires on 2024-09-29.
These files will be updated when the certificate renews.
NEXT STEPS:

  • The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

@caiocesarstx
Copy link

I unfortunately can't use pip because I am running NPM in home assistant.

Is importante to say that, this works for me, but for the running container, if eventually needs to restart, probably will be an error on renew and need to do it again for other certificates.

@Shredder5262
Copy link

pip install cloudflare==2.19.*

this worked for me

@claffern
Copy link

I had the same issue, Need to reinstall pip and pip install cloudflare==2.19.*

https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

this worked for me!!!!

@Silicon51
Copy link

So, there's a chance that you have my case: both piHole and NPM as docker containers.
Due to some weird behavior of DNS resolver NPM container do not have access to internet so cannot request for cert.
For me it log errors like Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/cloudflare/
Also I have error Failed to check the reachability due to a communication error with site24x7.com nginx proxy when in version 2.12.3 I use option "Test Server Reachability" from tab SSL Certificates.
How to solve it?
add following to your NPM docker compose:

dns:
  - 172.19.0.4 <<pihole IP adress>>
  - 1.1.1.1
  - 8.8.8.8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy