with regex report

akamai · Aug 3, 2021 · 5fe686d · 5fe686d
1 parent 4256be6
commit 5fe686d
Show file tree

Hide file tree

Showing 3 changed files with 21 additions and 10 deletions.
diff --git a/Readme.md b/Readme.md
@@ -148,29 +148,30 @@ We just turned off all step exept the regex generation steps that we want to run
 
 Now again (from the docker)
 
+/!\ This step can take few hours ( ~2h on a 48CPU machine, 378GB RAM without using all its ressources)
+
 ```bash
-python main.py
+python main.py 
 ```
 
-Check the log on luda_output/logs/luda.log at the end you can see a small report
+Check the log on luda_output/logs/luda.log at the end you can see a small report in the log ( where you see how each signature evolved at each round)
 
 ```txt
-        N paths: 64
+        N cluster : 2
+        N paths: 44
         N benign in final test: 9486
         Benign number for retraining : 30
         N round: 10
 
         Cluster sig paths:
 
-        cluster_27_0 : (\.*+[^_])++ ---> [^bin]++[^\.]++\.bin
-cluster_12_15 : [^php]*+php ---> /\w\w++/gate\.php ---> /\w\w\w\w\d/gate\.php
-cluster_8_16 : ([^_]\w++)++ ---> [^php]++php ---> /\w++/PHP/\w++\.php
-cluster_17_4 : ([^_]\w++)++ ---> [^\.]*+\.php ---> /\w++(?:/kbpanel)?+/post\.php ---> [^php]*+\w\w\w/?+\w++/post\.php
+        cluster_27_0 : (\.*+[^_])++ ---> [^bin]*+[^\.]*+\.bin
+cluster_17_4 : ([^_]\w++)++ ---> [^\.]++\.php ---> (\w*+/)++post\.php ---> [^php]++\w\w\w/?+\w++/post\.php
 
 
         After final testing:
-        Cluster with 0 FP: {'cluster_8_16', 'cluster_17_4', 'cluster_27_0', 'cluster_12_15'}
-        Number of paths covered with 0 FP: 64
+        Cluster with 0 FP: {'cluster_17_4', 'cluster_27_0'}
+        Number of paths covered with 0 FP: 44
         Percentage of paths covered with 0 FP: 100.0 %
 
         ### FP Report ###
@@ -181,9 +182,17 @@ cluster_17_4 : ([^_]\w++)++ ---> [^\.]*+\.php ---> /\w++(?:/kbpanel)?+/post\.php
 
         Without:
 
-        ['cluster_12_15', 'cluster_8_16', 'cluster_27_0', 'cluster_17_4']
+        ['cluster_27_0', 'cluster_17_4']
 ```
 
+You also get a report showing basic info on the run. It's a csv stored in the "regex_folder" ( following the above config, it is luda_output/myregexes/report_myregexes.csv)
+
+|id|name        |regex_js                                            |regex_java                     |malicious|benign|round|example_malicious          |results_file             |input_file             |
+|------|------------|----------------------------------------------------|-------------------------------|---------|------|-----|---------------------------|-------------------------|-----------------------|
+|0     |cluster_17_4|(?=([^php]+))\1\w\w\w(?=(/?))\2(?=(\w+))\3/post\.php|[^php]++\w\w\w/?+\w++/post\.php|17       |61    |3    |/mupanel/post.php.         |results_cluster_17_4.json|input_cluster_17_4.json|
+|1     |cluster_27_0|(?=([^bin]*))\1(?=([^\.]*))\2\.bin                  |[^bin]*+[^\.]*+\.bin           |27       |30    |1    |/neat/serverphp/config.bin.|results_cluster_27_0.json|input_cluster_27_0.json|
+
+
 Congrats on your first LUDA run. You now have 2 (Java) regex that can be used malicious urls belonging to the clusters you found :)
 
 On the next part, we will dive into LUDA architecture to understand each of its components, understand what else you can do and possibly make 

diff --git a/src/regex/regex.py b/src/regex/regex.py
@@ -351,3 +351,4 @@ def remove_nan_value_from_list(_list):
                 continue
             result.append(el)
         return result
+
diff --git a/src/use_case/use_case_regex_generation.py b/src/use_case/use_case_regex_generation.py
@@ -47,6 +47,7 @@ def run(self, main_file, features_folder, cluster_list, benign_for_retrain=20,
         self.regex_object.run_with_benign_check(_cluster_dict=cluster_dict, benign_list=str_to_not_match,
                                                 benign_for_retrain=benign_for_retrain,
                                                 take_existing_result=take_existing_result, round_max=round_max)
+        self.regex_object.create_result_report()
 
     @staticmethod
     def load_df(main_file, features_folder):
Original file line number	Diff line number	Diff line change
Expand Up		@@ -351,3 +351,4 @@ def remove_nan_value_from_list(_list):
		continue
		result.append(el)
		return result