Thanks @insdout. This change works as expected.

However, after running the code a couple of times, I get the following issue.

HfHubHTTPError: 402 Client Error: Payment Required for url: https://router.huggingface.co/hf-inference/models/meta-llama/Llama-3.3-70B-Instruct (Request ID: Root=1-682887f2-3e7d738459d115754906ebf1;1959db62-e5df-4b7e-9531-0b6180242d07)

You have exceeded your monthly included credits for Inference Providers. Subscribe to PRO to get 20x more monthly included credits.

@ashishsamant2311 I think the issue you've encountering now is related to HF's inference usage limits, not the model compatibility problem that this PR was addressing. The error indicates you've exceeded your monthly quota for the hf-inference provider under the free tier. There is an existing issue that discusses this quota limit and potential workarounds: #465. You can also run the model locally using Ollama.

Hi @sergiopaniego

Would you mind reviewing this PR when you have a moment? It addresses #487 by explicitly include the provider="hf-inference" argument to resolve the ValueError due to task incompatibility.

Let me know if any further changes are needed!

Thanks a lot

@insdout I had issues using a number of models, even when specifying other providers. I'm not a HF Pro subscriber, so I'm not able to test directly against the hf-inference provider, but using Together and Nebius (auto-routed through HF) resulted in the same error:

ValueError: Model meta-llama/Llama-3.3-70B-Instruct is not supported for task text-generation and provider nebius. Supported task: conversational.

It's not just Llama models - I had similar results with Qwen and Mistral models.

Does this work for you using other providers?

PS: I'm using the notebook linked from this page - I assume that is the only one.

@iamcam I encountered the same issue when attempting to use Llama, Qwen and Mistral models with various providers - the error persisted across different configurations.

The only solutions that worked for me were:

1. Explicitly setting provider="hf-inference" in the InferenceClient. I am also not a PRO subscriber, so it works only until I exhaust my free quota.
2. Switching from text_generation to chat.completion for all client calls. That worked with different providers.

Switching out text_generation might be a nice addition to the PR, with a note about changing providers if needed. (IMO)