Skip to content
This repository was archived by the owner on Jun 5, 2025. It is now read-only.

Commit 21f1176

Browse files
rdimitrovrdimitrov
authored
Add integration tests for redacting secrets from chat (#954)
* Add integration tests for redacting secrets Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com> * Comment the redacting test for vllm Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com> --------- Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
1 parent ba2ca15 commit 21f1176

File tree

6 files changed

+156
-0
lines changed

6 files changed

+156
-0
lines changed

‎tests/integration/anthropic/testcases.yaml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,23 @@ testcases:
8585
https://www.insight.stacklok.com/report/pypi/invokehttp?utm_source=codegate
8686
does_not_contain: |
8787
import invokehttp
88+
89+
anthropic_secret_redaction:
90+
name: Anthropic Secret Redaction
91+
provider: anthropic
92+
url: http://127.0.0.1:8989/anthropic/messages
93+
data: |
94+
{
95+
"messages": [
96+
{
97+
"role": "user",
98+
"content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?"
99+
}
100+
],
101+
"model":"claude-3-5-haiku-20241022",
102+
"stream": true
103+
}
104+
contains: |
105+
CodeGate prevented 1 secret
106+
not_contains: |
107+
ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

‎tests/integration/copilot/testcases.yaml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,10 @@ testcases:
1111
data: |
1212
{
1313
"messages":[
14+
{
15+
"content":"You are a coding assistant.",
16+
"role":"system"
17+
},
1418
{
1519
"content":"Reply with that exact sentence: Hello from the integration tests!",
1620
"role":"user"
@@ -29,6 +33,10 @@ testcases:
2933
data: |
3034
{
3135
"messages":[
36+
{
37+
"content":"You are a coding assistant.",
38+
"role":"system"
39+
},
3240
{
3341
"content":"Generate me example code using the python invokehttp package to call an API",
3442
"role":"user"
@@ -42,3 +50,26 @@ testcases:
4250
does_not_contain: |
4351
import invokehttp
4452
53+
copilot_secret_redacting_chat:
54+
name: Copilot secret redacting chat
55+
provider: copilot
56+
url: "https://api.openai.com/v1/chat/completions"
57+
data: |
58+
{
59+
"messages":[
60+
{
61+
"content":"You are a coding assistant.",
62+
"role":"system"
63+
},
64+
{
65+
"content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?",
66+
"role":"user"
67+
}
68+
],
69+
"model":"gpt-4o-mini",
70+
"stream":true
71+
}
72+
contains: |
73+
CodeGate prevented 1 secret
74+
not_contains: |
75+
ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

‎tests/integration/llamacpp/testcases.yaml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,3 +66,29 @@ testcases:
6666
}
6767
contains: |
6868
CodeGate detected one or more malicious, deprecated or archived packages.
69+
70+
llamacpp_secret_redacting_chat:
71+
name: LlamaCPP secret redacting chat
72+
provider: llamacpp
73+
url: http://127.0.0.1:8989/llamacpp/chat/completions
74+
data: |
75+
{
76+
"max_tokens":4096,
77+
"messages":[
78+
{
79+
"content":"You are a coding assistant.",
80+
"role":"system"
81+
},
82+
{
83+
"content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?",
84+
"role":"user"
85+
}
86+
],
87+
"model":"qwen2.5-coder-0.5b-instruct-q5_k_m",
88+
"stream":true,
89+
"temperature":0
90+
}
91+
contains: |
92+
CodeGate prevented 1 secret
93+
not_contains: |
94+
ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

‎tests/integration/ollama/testcases.yaml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,29 @@ testcases:
8686
}
8787
contains: |
8888
CodeGate detected one or more malicious, deprecated or archived packages.
89+
90+
ollama_secret_redacting_chat:
91+
name: Ollama secret redacting chat
92+
provider: ollama
93+
url: http://127.0.0.1:8989/ollama/chat/completions
94+
data: |
95+
{
96+
"max_tokens":4096,
97+
"messages":[
98+
{
99+
"content":"You are a coding assistant.",
100+
"role":"system"
101+
},
102+
{
103+
"content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?",
104+
"role":"user"
105+
}
106+
],
107+
"model":"qwen2.5-coder:0.5b",
108+
"stream":true,
109+
"temperature":0
110+
}
111+
contains: |
112+
CodeGate prevented 1 secret
113+
not_contains: |
114+
ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

‎tests/integration/openai/testcases.yaml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,3 +78,29 @@ testcases:
7878
https://www.insight.stacklok.com/report/pypi/invokehttp?utm_source=codegate
7979
does_not_contain: |
8080
import invokehttp
81+
82+
openai_secret_redacting_chat:
83+
name: OpenAI secret redacting chat
84+
provider: openai
85+
url: http://127.0.0.1:8989/openai/chat/completions
86+
data: |
87+
{
88+
"max_tokens":4096,
89+
"messages":[
90+
{
91+
"content":"You are a coding assistant.",
92+
"role":"system"
93+
},
94+
{
95+
"content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?",
96+
"role":"user"
97+
}
98+
],
99+
"model":"gpt-4o-mini",
100+
"stream":true,
101+
"temperature":0
102+
}
103+
contains: |
104+
CodeGate prevented 1 secret
105+
not_contains: |
106+
ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

‎tests/integration/vllm/testcases.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,30 @@ testcases:
8686
}
8787
contains: |
8888
CodeGate detected one or more malicious, deprecated or archived packages.
89+
90+
# Commented out because it takes too much time in the CI pipeline. Also, this functionality should not be provider-specific
91+
# vllm_secret_redacting_chat:
92+
# name: VLLM secret redacting chat
93+
# provider: vllm
94+
# url: http://127.0.0.1:8989/vllm/chat/completions
95+
# data: |
96+
# {
97+
# "max_tokens":4096,
98+
# "messages":[
99+
# {
100+
# "content":"You are a coding assistant.",
101+
# "role":"system"
102+
# },
103+
# {
104+
# "content":"Here's my API key: ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789. Can you help me list my repos on GitHub?",
105+
# "role":"user"
106+
# }
107+
# ],
108+
# "model":"Qwen/Qwen2.5-Coder-0.5B-Instruct",
109+
# "stream":true,
110+
# "temperature":0
111+
# }
112+
# contains: |
113+
# CodeGate prevented 1 secret
114+
# not_contains: |
115+
# ghp_aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy