Skip to content

Initial suspicious commands #917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Feb 5, 2025
Merged

Initial suspicious commands #917

merged 12 commits into from
Feb 5, 2025

Conversation

therealnb
Copy link
Contributor

@therealnb therealnb commented Feb 4, 2025

Initial implementation of the prototype in https://github.com/stacklok/research/blob/command-detection/command_detection/command_models.ipynb. Closes #844

Note that:

  • There has been little optimisation of the ANN
  • We need to work on getting better data
  • Performance may not be good enough - testing required

Accuracy: 0.88
Precision: 0.8823529411764706
Recall: 0.7894736842105263
F1 Score: 0.8333333333333333

Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
@therealnb therealnb requested review from ptelang, jhrozek and blkt February 4, 2025 15:53
Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
@yrobla
Copy link
Contributor

yrobla commented Feb 5, 2025

It's great to have that! Have we measure how much time does it normally add to the pipeline?

@therealnb
Copy link
Contributor Author

@yrobla there is some timing info in https://github.com/stacklok/research/blob/command-detection/command_detection/command_models.ipynb (some charts). This model takes about 0.03-0.05 seconds.

Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
Signed-off-by: nigel brown <nigel@stacklok.com>
@therealnb therealnb merged commit f0c387f into main Feb 5, 2025
4 checks passed
@therealnb therealnb deleted the codegate-844 branch February 5, 2025 11:39
rdimitrov added a commit that referenced this pull request Feb 5, 2025
therealnb pushed a commit that referenced this pull request Feb 5, 2025
@therealnb therealnb restored the codegate-844 branch February 5, 2025 12:51
@therealnb therealnb mentioned this pull request Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Spotting commands in the stream from coding assistants like cline
3 participants
pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy