Commit edf42c36 authored by Christian Marius Lillelund's avatar Christian Marius Lillelund
Browse files

Restructed the dataset baseline tests

parent 2dfb42c5
Pipeline #44080 failed with stage
in 3 minutes and 12 seconds
feature,shap_values,feat_imp
3Ats,0.31289357,0.047934923
2Ats,0.29975817,0.043996885
7Ats,0.29632568,0.04681093
8Ats,0.2858649,0.04623301
4Ats,0.25710154,0.04035271
4Ex,0.25318545,0.038030393
1Ats,0.24223052,0.04620158
5Ats,0.23203886,0.044167936
10Ats,0.22442655,0.047305025
3Ex,0.21373507,0.038372926
6Ex,0.17370935,0.0344399
5Ex,0.17300294,0.033084366
2Ex,0.17278673,0.034701586
Needs,0.16645956,0.02923049
9Ats,0.14796755,0.04421669
Cluster,0.13645108,0.027709806
8Ex,0.12976792,0.047058653
BirthYear,0.12052875,0.026461434
6Ats,0.11824282,0.032616466
7Ex,0.11497209,0.03494578
1Ex,0.11381523,0.02706626
MeanEvaluation,0.104623675,0.025604129
Physics,0.097208135,0.026487732
NumberAts,0.08153105,0.02365695
NumberFalls,0.07433112,0.029914727
Gender,0.03533531,0.028777694
NumberExercises,0.028090222,0.021228254
9Ex,0.025323618,0.032417543
NumberCancels,0.00019256209,0.0009752363
MaxNumberCancelsWeek,0.0,0.0
MaxNumberTrainingWeek,0.0,0.0
MaxEvaluation,0.0,0.0
MinEvaluation,0.0,0.0
MinNumberCancelsWeek,0.0,0.0
MinNumberTrainingWeekMin,0.0,0.0
NumberTraining,0.0,0.0
NumberWeeksWithTraining,0.0,0.0
3Ats,0.30701742,0.048671
7Ats,0.28294212,0.048502482
2Ats,0.28034192,0.045678426
8Ats,0.25991207,0.045692477
4Ex,0.2514352,0.04040911
4Ats,0.23456395,0.040136963
1Ats,0.23401716,0.045880385
5Ats,0.23319668,0.044036645
10Ats,0.21698344,0.046996757
3Ex,0.20088047,0.03707639
6Ex,0.16476086,0.035580214
Needs,0.15506667,0.028816348
2Ex,0.15479976,0.031543493
5Ex,0.1503424,0.03286143
9Ats,0.13760921,0.04254617
Cluster,0.12645094,0.028721172
8Ex,0.11938239,0.047088034
6Ats,0.11608223,0.030648576
7Ex,0.11577342,0.0352803
1Ex,0.10754093,0.030380217
BirthYear,0.10448934,0.02480473
MeanEvaluation,0.0906614,0.026322493
Physics,0.089720756,0.02604762
NumberAts,0.08033991,0.024879964
NumberFalls,0.07903269,0.028732995
Gender,0.037929114,0.030026866
NumberExercises,0.021545557,0.0199459
9Ex,0.017604694,0.030014977
NumberCancels,0.00033660323,0.0026778653
Complete case 1 - without columns MLP K=0: binary_accuracy: 0.6070 - precision: 0.5915 - recall: 0.5638 - auc: 0.6050 MLP K=1: binary_accuracy: 0.5463 - precision: 0.5248 - recall: 0.4966 - auc: 0.5441 MLP K=2: binary_accuracy: 0.5719 - precision: 0.5444 - recall: 0.6174 - auc: 0.5740 MLP K=3: binary_accuracy: 0.5559 - precision: 0.5352 - recall: 0.5101 - auc: 0.5538 MLP K=4: binary_accuracy: 0.5623 - precision: 0.5423 - recall: 0.5168 - auc: 0.5602 XGB K=0: binary_accuracy: 0.6134 - precision: 0.5909 - recall: 0.6107 - auc: 0.6133 XGB K=1: binary_accuracy: 0.5399 - precision: 0.5175 - recall: 0.4966 - auc: 0.5380 XGB K=2: binary_accuracy: 0.5367 - precision: 0.5132 - recall: 0.5235 - auc: 0.5361 XGB K=3: binary_accuracy: 0.5272 - precision: 0.5032 - recall: 0.5302 - auc: 0.5273 XGB K=4: binary_accuracy: 0.5751 - precision: 0.5513 - recall: 0.5772 - auc: 0.5752 RF K=0: binary_accuracy: 0.5911 - precision: 0.5745 - recall: 0.5436 - auc: 0.5889 RF K=1: binary_accuracy: 0.5847 - precision: 0.5760 - recall: 0.4832 - auc: 0.5800 RF K=2: binary_accuracy: 0.5591 - precision: 0.5414 - recall: 0.4832 - auc: 0.5556 RF K=3: binary_accuracy: 0.5367 - precision: 0.5152 - recall: 0.4564 - auc: 0.5331 RF K=4: binary_accuracy: 0.5399 - precision: 0.5166 - recall: 0.5235 - auc: 0.5392 SVM K=0: binary_accuracy: 0.5623 - precision: 0.5385 - recall: 0.5638 - auc: 0.5624 SVM K=1: binary_accuracy: 0.5591 - precision: 0.5440 - recall: 0.4564 - auc: 0.5544 SVM K=2: binary_accuracy: 0.5974 - precision: 0.5657 - recall: 0.6644 - auc: 0.6005 SVM K=3: binary_accuracy: 0.5495 - precision: 0.5270 - recall: 0.5235 - auc: 0.5483 SVM K=4: binary_accuracy: 0.5495 - precision: 0.5233 - recall: 0.6040 - auc: 0.5520
Complete case 1 - without columns MLP K=0: Accuracy: 0.5401 - Precision: 0.5159 - Recall: 0.5561 MLP K=1: Accuracy: 0.5310 - Precision: 0.4983 - Recall: 0.5107 MLP K=2: Accuracy: 0.5371 - Precision: 0.5222 - Recall: 0.5283 MLP K=3: Accuracy: 0.5350 - Precision: 0.5088 - Recall: 0.5410 MLP K=4: Accuracy: 0.5371 - Precision: 0.5182 - Recall: 0.5257 SVM K=0: Accuracy: 0.5381 - Precision: 0.5121 - Recall: 0.5474 SVM K=1: Accuracy: 0.5086 - Precision: 0.4802 - Recall: 0.5235 SVM K=2: Accuracy: 0.5330 - Precision: 0.5020 - Recall: 0.5195 SVM K=3: Accuracy: 0.5340 - Precision: 0.5065 - Recall: 0.5324 SVM K=4: Accuracy: 0.5360 - Precision: 0.5070 - Recall: 0.5278 XGB K=0: Accuracy: 0.5563 - Precision: 0.5291 - Recall: 0.5432 XGB K=1: Accuracy: 0.5310 - Precision: 0.5023 - Recall: 0.5561 XGB K=2: Accuracy: 0.5218 - Precision: 0.4913 - Recall: 0.5044 XGB K=3: Accuracy: 0.5289 - Precision: 0.4998 - Recall: 0.5409 XGB K=4: Accuracy: 0.5228 - Precision: 0.4957 - Recall: 0.5129 RF K=0: Accuracy: 0.5726 - Precision: 0.5539 - Recall: 0.4699 RF K=1: Accuracy: 0.5553 - Precision: 0.5350 - Recall: 0.4676 RF K=2: Accuracy: 0.5472 - Precision: 0.5212 - Recall: 0.4612 RF K=3: Accuracy: 0.5421 - Precision: 0.5166 - Recall: 0.4353 RF K=4: Accuracy: 0.5462 - Precision: 0.5224 - Recall: 0.4461 SVM K=0: Accuracy: 0.5553 - Precision: 0.5269 - Recall: 0.5584 SVM K=1: Accuracy: 0.5198 - Precision: 0.4922 - Recall: 0.5215 SVM K=2: Accuracy: 0.5533 - Precision: 0.5224 - Recall: 0.5499 SVM K=3: Accuracy: 0.5553 - Precision: 0.5258 - Recall: 0.5539 SVM K=4: Accuracy: 0.5431 - Precision: 0.5151 - Recall: 0.5215
Complete case 2 - with one-hot-encoding MLP K=0: binary_accuracy: 0.5527 - precision: 0.5319 - recall: 0.5034 - auc: 0.5505 MLP K=1: binary_accuracy: 0.5367 - precision: 0.5141 - recall: 0.4899 - auc: 0.5346 MLP K=2: binary_accuracy: 0.5942 - precision: 0.5733 - recall: 0.5772 - auc: 0.5935 MLP K=3: binary_accuracy: 0.5335 - precision: 0.5092 - recall: 0.5570 - auc: 0.5346 MLP K=4: binary_accuracy: 0.5335 - precision: 0.5103 - recall: 0.4966 - auc: 0.5319 XGB K=0: binary_accuracy: 0.5495 - precision: 0.5270 - recall: 0.5235 - auc: 0.5483 XGB K=1: binary_accuracy: 0.5495 - precision: 0.5263 - recall: 0.5369 - auc: 0.5489 XGB K=2: binary_accuracy: 0.5495 - precision: 0.5267 - recall: 0.5302 - auc: 0.5486 XGB K=3: binary_accuracy: 0.5240 - precision: 0.5000 - recall: 0.5302 - auc: 0.5242 XGB K=4: binary_accuracy: 0.5335 - precision: 0.5099 - recall: 0.5168 - auc: 0.5328 RF K=0: binary_accuracy: 0.5623 - precision: 0.5441 - recall: 0.4966 - auc: 0.5593 RF K=1: binary_accuracy: 0.5655 - precision: 0.5496 - recall: 0.4832 - auc: 0.5617 RF K=2: binary_accuracy: 0.5815 - precision: 0.5616 - recall: 0.5503 - auc: 0.5800 RF K=3: binary_accuracy: 0.5240 - precision: 0.5000 - recall: 0.4698 - auc: 0.5215 RF K=4: binary_accuracy: 0.5495 - precision: 0.5303 - recall: 0.4698 - auc: 0.5459 SVM K=0: binary_accuracy: 0.5783 - precision: 0.5669 - recall: 0.4832 - auc: 0.5739 SVM K=1: binary_accuracy: 0.6006 - precision: 0.5882 - recall: 0.5369 - auc: 0.5977 SVM K=2: binary_accuracy: 0.5974 - precision: 0.5752 - recall: 0.5906 - auc: 0.5971 SVM K=3: binary_accuracy: 0.5815 - precision: 0.5549 - recall: 0.6107 - auc: 0.5828 SVM K=4: binary_accuracy: 0.5240 - precision: 0.5000 - recall: 0.4161 - auc: 0.5190
Complete case 2 - with one-hot-encoding MLP K=0: Accuracy: 0.5584 - Precision: 0.5311 - Recall: 0.5475 MLP K=1: Accuracy: 0.5553 - Precision: 0.5151 - Recall: 0.5494 MLP K=2: Accuracy: 0.5553 - Precision: 0.5344 - Recall: 0.4873 MLP K=3: Accuracy: 0.5614 - Precision: 0.5409 - Recall: 0.5559 MLP K=4: Accuracy: 0.5401 - Precision: 0.5213 - Recall: 0.5152 SVM K=0: Accuracy: 0.5442 - Precision: 0.5164 - Recall: 0.5582 SVM K=1: Accuracy: 0.5584 - Precision: 0.5310 - Recall: 0.5669 SVM K=2: Accuracy: 0.5340 - Precision: 0.5030 - Recall: 0.5303 SVM K=3: Accuracy: 0.5685 - Precision: 0.5421 - Recall: 0.5734 SVM K=4: Accuracy: 0.5442 - Precision: 0.5169 - Recall: 0.5345 XGB K=0: Accuracy: 0.5330 - Precision: 0.5045 - Recall: 0.5259 XGB K=1: Accuracy: 0.5330 - Precision: 0.5030 - Recall: 0.5128 XGB K=2: Accuracy: 0.5076 - Precision: 0.4759 - Recall: 0.4484 XGB K=3: Accuracy: 0.5310 - Precision: 0.5038 - Recall: 0.5021 XGB K=4: Accuracy: 0.5259 - Precision: 0.4969 - Recall: 0.4741 RF K=0: Accuracy: 0.5604 - Precision: 0.5364 - Recall: 0.4679 RF K=1: Accuracy: 0.5533 - Precision: 0.5304 - Recall: 0.4719 RF K=2: Accuracy: 0.5299 - Precision: 0.5021 - Recall: 0.4420 RF K=3: Accuracy: 0.5533 - Precision: 0.5338 - Recall: 0.4525 RF K=4: Accuracy: 0.5431 - Precision: 0.5190 - Recall: 0.4505 SVM K=0: Accuracy: 0.5746 - Precision: 0.5466 - Recall: 0.5690 SVM K=1: Accuracy: 0.5827 - Precision: 0.5606 - Recall: 0.5646 SVM K=2: Accuracy: 0.5614 - Precision: 0.5366 - Recall: 0.5281 SVM K=3: Accuracy: 0.5817 - Precision: 0.5628 - Recall: 0.5560 SVM K=4: Accuracy: 0.5706 - Precision: 0.5446 - Recall: 0.5367
Complete case 3 - with catboost-encoding MLP K=0: binary_accuracy: 0.5399 - precision: 0.5120 - recall: 0.7181 - auc: 0.5481 MLP K=1: binary_accuracy: 0.5272 - precision: 0.5038 - recall: 0.4497 - auc: 0.5236 MLP K=2: binary_accuracy: 0.5272 - precision: 0.5030 - recall: 0.5705 - auc: 0.5291 MLP K=3: binary_accuracy: 0.5176 - precision: 0.4930 - recall: 0.4698 - auc: 0.5154 MLP K=4: binary_accuracy: 0.5495 - precision: 0.5267 - recall: 0.5302 - auc: 0.5486 XGB K=0: binary_accuracy: 0.5176 - precision: 0.4932 - recall: 0.4899 - auc: 0.5163 XGB K=1: binary_accuracy: 0.5176 - precision: 0.4929 - recall: 0.4631 - auc: 0.5151 XGB K=2: binary_accuracy: 0.4633 - precision: 0.4354 - recall: 0.4295 - auc: 0.4617 XGB K=3: binary_accuracy: 0.5016 - precision: 0.4755 - recall: 0.4564 - auc: 0.4995 XGB K=4: binary_accuracy: 0.5463 - precision: 0.5226 - recall: 0.5436 - auc: 0.5462 RF K=0: binary_accuracy: 0.5751 - precision: 0.5702 - recall: 0.4362 - auc: 0.5687 RF K=1: binary_accuracy: 0.5208 - precision: 0.4952 - recall: 0.3490 - auc: 0.5129 RF K=2: binary_accuracy: 0.5016 - precision: 0.4748 - recall: 0.4430 - auc: 0.4989 RF K=3: binary_accuracy: 0.5112 - precision: 0.4828 - recall: 0.3758 - auc: 0.5050 RF K=4: binary_accuracy: 0.5431 - precision: 0.5250 - recall: 0.4228 - auc: 0.5376 SVM K=0: binary_accuracy: 0.5559 - precision: 0.5357 - recall: 0.5034 - auc: 0.5535 SVM K=1: binary_accuracy: 0.5431 - precision: 0.5200 - recall: 0.5235 - auc: 0.5422 SVM K=2: binary_accuracy: 0.5751 - precision: 0.5465 - recall: 0.6309 - auc: 0.5776 SVM K=3: binary_accuracy: 0.5240 - precision: 0.5000 - recall: 0.4497 - auc: 0.5206 SVM K=4: binary_accuracy: 0.5399 - precision: 0.5172 - recall: 0.5034 - auc: 0.5383
Complete case 3 - with catboost-encoding MLP K=0: Accuracy: 0.5411 - Precision: 0.5101 - Recall: 0.5843 MLP K=1: Accuracy: 0.5249 - Precision: 0.5048 - Recall: 0.5387 MLP K=2: Accuracy: 0.5431 - Precision: 0.4981 - Recall: 0.5605 MLP K=3: Accuracy: 0.5371 - Precision: 0.5082 - Recall: 0.5521 MLP K=4: Accuracy: 0.5421 - Precision: 0.5198 - Recall: 0.5754 SVM K=0: Accuracy: 0.5330 - Precision: 0.5065 - Recall: 0.5344 SVM K=1: Accuracy: 0.5198 - Precision: 0.4912 - Recall: 0.5344 SVM K=2: Accuracy: 0.5198 - Precision: 0.4883 - Recall: 0.4979 SVM K=3: Accuracy: 0.5239 - Precision: 0.4957 - Recall: 0.5280 SVM K=4: Accuracy: 0.5472 - Precision: 0.5168 - Recall: 0.5515 XGB K=0: Accuracy: 0.5228 - Precision: 0.4921 - Recall: 0.4846 XGB K=1: Accuracy: 0.5056 - Precision: 0.4760 - Recall: 0.4998 XGB K=2: Accuracy: 0.5310 - Precision: 0.5021 - Recall: 0.5215 XGB K=3: Accuracy: 0.5320 - Precision: 0.5020 - Recall: 0.5257 XGB K=4: Accuracy: 0.5127 - Precision: 0.4828 - Recall: 0.4914 RF K=0: Accuracy: 0.5574 - Precision: 0.5400 - Recall: 0.4224 RF K=1: Accuracy: 0.5411 - Precision: 0.5166 - Recall: 0.4008 RF K=2: Accuracy: 0.5513 - Precision: 0.5327 - Recall: 0.3922 RF K=3: Accuracy: 0.5523 - Precision: 0.5320 - Recall: 0.4181 RF K=4: Accuracy: 0.5350 - Precision: 0.5069 - Recall: 0.4074 SVM K=0: Accuracy: 0.5635 - Precision: 0.5360 - Recall: 0.5626 SVM K=1: Accuracy: 0.5371 - Precision: 0.5109 - Recall: 0.5237 SVM K=2: Accuracy: 0.5472 - Precision: 0.5154 - Recall: 0.5520 SVM K=3: Accuracy: 0.5574 - Precision: 0.5291 - Recall: 0.5560 SVM K=4: Accuracy: 0.5442 - Precision: 0.5155 - Recall: 0.5194
Complete case 4 - with embeddings MLP K=0: binary_accuracy: 0.7700 - precision: 0.7619 - recall: 0.7517 - auc: 0.7691 MLP K=1: binary_accuracy: 0.7796 - precision: 0.7632 - recall: 0.7785 - auc: 0.7795 MLP K=2: binary_accuracy: 0.7923 - precision: 0.7958 - recall: 0.7584 - auc: 0.7908 MLP K=3: binary_accuracy: 0.7891 - precision: 0.7456 - recall: 0.8456 - auc: 0.7917 MLP K=4: binary_accuracy: 0.7891 - precision: 0.7456 - recall: 0.8456 - auc: 0.7917 XGB K=0: binary_accuracy: 0.7029 - precision: 0.7029 - recall: 0.6510 - auc: 0.7005 XGB K=1: binary_accuracy: 0.7061 - precision: 0.7021 - recall: 0.6644 - auc: 0.7042 XGB K=2: binary_accuracy: 0.6773 - precision: 0.6714 - recall: 0.6309 - auc: 0.6752 XGB K=3: binary_accuracy: 0.6901 - precision: 0.6733 - recall: 0.6779 - auc: 0.6895 XGB K=4: binary_accuracy: 0.7061 - precision: 0.7050 - recall: 0.6577 - auc: 0.7039 RF K=0: binary_accuracy: 0.6550 - precision: 0.6752 - recall: 0.5302 - auc: 0.6492 RF K=1: binary_accuracy: 0.7093 - precision: 0.7132 - recall: 0.6510 - auc: 0.7066 RF K=2: binary_accuracy: 0.6773 - precision: 0.6739 - recall: 0.6242 - auc: 0.6749 RF K=3: binary_accuracy: 0.7188 - precision: 0.7103 - recall: 0.6913 - auc: 0.7176 RF K=4: binary_accuracy: 0.7284 - precision: 0.7462 - recall: 0.6510 - auc: 0.7249 SVM K=0: binary_accuracy: 0.7508 - precision: 0.7518 - recall: 0.7114 - auc: 0.7490 SVM K=1: binary_accuracy: 0.7764 - precision: 0.7687 - recall: 0.7584 - auc: 0.7755 SVM K=2: binary_accuracy: 0.7827 - precision: 0.7755 - recall: 0.7651 - auc: 0.7819 SVM K=3: binary_accuracy: 0.7923 - precision: 0.7838 - recall: 0.7785 - auc: 0.7917 SVM K=4: binary_accuracy: 0.8019 - precision: 0.8042 - recall: 0.7718 - auc: 0.8005
Complete case 4 - with embeddings MLP K=0: Accuracy: 0.7594 - Precision: 0.7401 - Recall: 0.7608 MLP K=1: Accuracy: 0.7574 - Precision: 0.7328 - Recall: 0.7608 MLP K=2: Accuracy: 0.7584 - Precision: 0.7299 - Recall: 0.7566 MLP K=3: Accuracy: 0.7594 - Precision: 0.7377 - Recall: 0.7585 MLP K=4: Accuracy: 0.7655 - Precision: 0.7298 - Recall: 0.7716 SVM K=0: Accuracy: 0.7635 - Precision: 0.7456 - Recall: 0.7609 SVM K=1: Accuracy: 0.7614 - Precision: 0.7375 - Recall: 0.7673 SVM K=2: Accuracy: 0.7675 - Precision: 0.7512 - Recall: 0.7588 SVM K=3: Accuracy: 0.7563 - Precision: 0.7447 - Recall: 0.7413 SVM K=4: Accuracy: 0.7695 - Precision: 0.7464 - Recall: 0.7737 XGB K=0: Accuracy: 0.6893 - Precision: 0.6787 - Recall: 0.6487 XGB K=1: Accuracy: 0.6975 - Precision: 0.6746 - Recall: 0.6940 XGB K=2: Accuracy: 0.7025 - Precision: 0.6894 - Recall: 0.6726 XGB K=3: Accuracy: 0.6975 - Precision: 0.6913 - Recall: 0.6595 XGB K=4: Accuracy: 0.6964 - Precision: 0.6731 - Recall: 0.6899 RF K=0: Accuracy: 0.7330 - Precision: 0.7368 - Recall: 0.6768 RF K=1: Accuracy: 0.7249 - Precision: 0.7311 - Recall: 0.6595 RF K=2: Accuracy: 0.7157 - Precision: 0.7209 - Recall: 0.6467 RF K=3: Accuracy: 0.7249 - Precision: 0.7410 - Recall: 0.6464 RF K=4: Accuracy: 0.7289 - Precision: 0.7395 - Recall: 0.6575 SVM K=0: Accuracy: 0.7178 - Precision: 0.7143 - Recall: 0.6703 SVM K=1: Accuracy: 0.6822 - Precision: 0.6719 - Recall: 0.6400 SVM K=2: Accuracy: 0.7005 - Precision: 0.6910 - Recall: 0.6597 SVM K=3: Accuracy: 0.6954 - Precision: 0.6924 - Recall: 0.6443 SVM K=4: Accuracy: 0.6863 - Precision: 0.6778 - Recall: 0.6423
Complete case 5 - with counts MLP K=0: binary_accuracy: 0.5847 - precision: 0.5597 - recall: 0.5973 - auc: 0.5852 MLP K=1: binary_accuracy: 0.5942 - precision: 0.5724 - recall: 0.5839 - auc: 0.5938 MLP K=2: binary_accuracy: 0.5240 - precision: 0.5000 - recall: 0.5369 - auc: 0.5246 MLP K=3: binary_accuracy: 0.5431 - precision: 0.5185 - recall: 0.5638 - auc: 0.5441 MLP K=4: binary_accuracy: 0.5911 - precision: 0.5695 - recall: 0.5772 - auc: 0.5904 XGB K=0: binary_accuracy: 0.5847 - precision: 0.5621 - recall: 0.5772 - auc: 0.5843 XGB K=1: binary_accuracy: 0.5687 - precision: 0.5479 - recall: 0.5369 - auc: 0.5672 XGB K=2: binary_accuracy: 0.5016 - precision: 0.4777 - recall: 0.5034 - auc: 0.5017 XGB K=3: binary_accuracy: 0.5783 - precision: 0.5515 - recall: 0.6107 - auc: 0.5798 XGB K=4: binary_accuracy: 0.5974 - precision: 0.5865 - recall: 0.5235 - auc: 0.5941 RF K=0: binary_accuracy: 0.5815 - precision: 0.5692 - recall: 0.4966 - auc: 0.5776 RF K=1: binary_accuracy: 0.5591 - precision: 0.5447 - recall: 0.4497 - auc: 0.5541 RF K=2: binary_accuracy: 0.5272 - precision: 0.5034 - recall: 0.4966 - auc: 0.5258 RF K=3: binary_accuracy: 0.5847 - precision: 0.5664 - recall: 0.5436 - auc: 0.5828 RF K=4: binary_accuracy: 0.6006 - precision: 0.5984 - recall: 0.4899 - auc: 0.5956 SVM K=0: binary_accuracy: 0.5687 - precision: 0.5515 - recall: 0.5034 - auc: 0.5657 SVM K=1: binary_accuracy: 0.5751 - precision: 0.5548 - recall: 0.5436 - auc: 0.5736 SVM K=2: binary_accuracy: 0.5304 - precision: 0.5066 - recall: 0.5168 - auc: 0.5297 SVM K=3: binary_accuracy: 0.5495 - precision: 0.5282 - recall: 0.5034 - auc: 0.5474 SVM K=4: binary_accuracy: 0.5655 - precision: 0.5430 - recall: 0.5503 - auc: 0.5648
\ No newline at end of file
Complete case 5 - with counts MLP K=0: Accuracy: 0.5340 - Precision: 0.5212 - Recall: 0.5387 MLP K=1: Accuracy: 0.5472 - Precision: 0.5315 - Recall: 0.5044 MLP K=2: Accuracy: 0.5604 - Precision: 0.5283 - Recall: 0.5368 MLP K=3: Accuracy: 0.5614 - Precision: 0.5319 - Recall: 0.5432 MLP K=4: Accuracy: 0.5513 - Precision: 0.5183 - Recall: 0.5409 SVM K=0: Accuracy: 0.5371 - Precision: 0.5106 - Recall: 0.5365 SVM K=1: Accuracy: 0.5452 - Precision: 0.5166 - Recall: 0.5302 SVM K=2: Accuracy: 0.5574 - Precision: 0.5287 - Recall: 0.5476 SVM K=3: Accuracy: 0.5695 - Precision: 0.5449 - Recall: 0.5495 SVM K=4: Accuracy: 0.5574 - Precision: 0.5293 - Recall: 0.5519 XGB K=0: Accuracy: 0.5787 - Precision: 0.5524 - Recall: 0.5540 XGB K=1: Accuracy: 0.5574 - Precision: 0.5301 - Recall: 0.5236 XGB K=2: Accuracy: 0.5584 - Precision: 0.5330 - Recall: 0.5152 XGB K=3: Accuracy: 0.5472 - Precision: 0.5213 - Recall: 0.5151 XGB K=4: Accuracy: 0.5472 - Precision: 0.5195 - Recall: 0.5129 RF K=0: Accuracy: 0.5827 - Precision: 0.5723 - Recall: 0.4677 RF K=1: Accuracy: 0.5919 - Precision: 0.5778 - Recall: 0.5043 RF K=2: Accuracy: 0.6000 - Precision: 0.5892 - Recall: 0.4979 RF K=3: Accuracy: 0.5929 - Precision: 0.5858 - Recall: 0.4827 RF K=4: Accuracy: 0.5726 - Precision: 0.5609 - Recall: 0.4289 SVM K=0: Accuracy: 0.5563 - Precision: 0.5336 - Recall: 0.5129 SVM K=1: Accuracy: 0.5685 - Precision: 0.5451 - Recall: 0.5110 SVM K=2: Accuracy: 0.5756 - Precision: 0.5527 - Recall: 0.5217 SVM K=3: Accuracy: 0.5878 - Precision: 0.5741 - Recall: 0.5216 SVM K=4: Accuracy: 0.5827 - Precision: 0.5613 - Recall: 0.5216
\ No newline at end of file
......
Complete case using 5 clfs and 9 scalers
Complete case using 4 clfs and 9 scalers
Results for MLP, None:
Accuracy: 0.578
Precision: 0.612
Recall: 0.496
Accuracy: 0.619
Precision: 0.593
Recall: 0.64
Results for MLP, Standard:
Accuracy: 0.738
Precision: 0.732
Recall: 0.68
Accuracy: 0.761
Precision: 0.744
Recall: 0.774
Results for MLP, MinMax:
Accuracy: 0.775
Precision: 0.773
Recall: 0.721
Accuracy: 0.772
Precision: 0.749
Recall: 0.763
Results for MLP, MinMaxRange:
Accuracy: 0.768
Precision: 0.777
Recall: 0.71
Accuracy: 0.766
Precision: 0.749
Recall: 0.791
Results for MLP, Robust:
Accuracy: 0.746
Precision: 0.761
Recall: 0.706
Accuracy: 0.762
Precision: 0.73
Recall: 0.754
Results for MLP, MaxAbs:
Accuracy: 0.769
Precision: 0.772
Recall: 0.769
Accuracy: 0.771
Precision: 0.749
Recall: 0.767
Results for MLP, QuantileTransformer:
Accuracy: 0.762
Precision: 0.784
Recall: 0.733
Accuracy: 0.777
Precision: 0.748
Recall: 0.778
Results for MLP, QuantileTransformerNorm:
Accuracy: 0.77
Precision: 0.769
Recall: 0.733
Accuracy: 0.771
Precision: 0.757
Recall: 0.774
Results for SVM, None:
Accuracy: 0.534
Precision: 0.519
Recall: 0.294
Accuracy: 0.552
Precision: 0.524
Recall: 0.563
Results for SVM, Standard:
Accuracy: 0.683
Precision: 0.684
Recall: 0.63
Accuracy: 0.718
Precision: 0.714
Recall: 0.67
Results for SVM, MinMax:
Accuracy: 0.776
Precision: 0.779
Recall: 0.743
Accuracy: 0.78
Precision: 0.771
Recall: 0.761
Results for SVM, MinMaxRange:
Accuracy: 0.741
Precision: 0.749
Recall: 0.696
Accuracy: 0.768
Precision: 0.757
Recall: 0.75
Results for SVM, Robust:
Accuracy: 0.717
Precision: 0.72
Recall: 0.664
Accuracy: 0.739
Precision: 0.737
Recall: 0.696
Results for SVM, MaxAbs:
Accuracy: 0.776
Precision: 0.778
Recall: 0.745
Accuracy: 0.78
Precision: 0.772
Recall: 0.759
Results for SVM, QuantileTransformer:
Accuracy: 0.774
Precision: 0.773
Recall: 0.747
Accuracy: 0.788
Precision: 0.788
Recall: 0.752
Results for SVM, QuantileTransformerNorm:
Accuracy: 0.774
Precision: 0.773
Recall: 0.747
Accuracy: 0.788
Precision: 0.788
Recall: 0.752
Results for RF, None:
Accuracy: 0.709
Precision: 0.706
Recall: 0.67
Accuracy: 0.734
Precision: 0.741
Recall: 0.673
Results for RF, Standard:
Accuracy: 0.71
Precision: 0.708
Recall: 0.67
Accuracy: 0.733
Precision: 0.737
Recall: 0.677
Results for RF, MinMax:
Accuracy: 0.707
Precision: 0.705
Recall: 0.666
Accuracy: 0.733
Precision: 0.739
Recall: 0.672
Results for RF, MinMaxRange:
Accuracy: 0.709
Precision: 0.707
Recall: 0.668
Accuracy: 0.734
Precision: 0.74
Recall: 0.675
Results for RF, Robust:
Accuracy: 0.708
Precision: 0.705
Recall: 0.668
Accuracy: 0.731
Precision: 0.735
Recall: 0.672
Results for RF, MaxAbs:
Accuracy: 0.707
Precision: 0.705
Recall: 0.666
Accuracy: 0.733
Precision: 0.739
Recall: 0.672
Results for RF, QuantileTransformer:
Accuracy: 0.711
Precision: 0.711
Recall: 0.666
Accuracy: 0.735
Precision: 0.744
Recall: 0.67
Results for RF, QuantileTransformerNorm:
Accuracy: 0.711
Precision: 0.711
Recall: 0.666
Accuracy: 0.735
Precision: 0.744
Recall: 0.67
Results for XGB, None:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, Standard:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, MinMax:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, MinMaxRange:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, Robust:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, MaxAbs:
Accuracy: 0.694
Precision: 0.693
Recall: 0.654
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, QuantileTransformer:
Accuracy: 0.695
Precision: 0.694
Recall: 0.656
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Results for XGB, QuantileTransformerNorm:
Accuracy: 0.695
Precision: 0.694
Recall: 0.656
Results for CB, None:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, Standard:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, MinMax:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, MinMaxRange:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, Robust:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, MaxAbs:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, QuantileTransformer:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Results for CB, QuantileTransformerNorm:
Accuracy: 0.713
Precision: 0.711
Recall: 0.672
Accuracy: 0.689
Precision: 0.679
Recall: 0.649
Fall case using 5 clfs and 9 scalers
Fall case using 4 clfs and 9 scalers
Results for MLP, None:
Accuracy: 0.888
Precision: 0.828
Recall: 0.507
Accuracy: 0.84
Precision: 0.569
Recall: 0.718
Results for MLP, Standard:
Accuracy: 0.888
Precision: 0.853
Recall: 0.52
Results for MLP, MinMax:
Accuracy: 0.888
Precision: 0.839
Recall: 0.516
Results for MLP, MinMaxRange:
Accuracy: 0.887
Precision: 0.846
Recall: 0.524
Results for MLP, Robust:
Accuracy: 0.889
Precision: 0.849
Recall: 0.52
Results for MLP, MaxAbs:
Accuracy: 0.889
Precision: 0.838
Recall: 0.512
Results for MLP, QuantileTransformer:
Accuracy: 0.888
Precision: 0.84
Recall: 0.517
Results for MLP, QuantileTransformerNorm:
Accuracy: 0.888
Precision: 0.834
Recall: 0.524
########################################
Results for SVM, None:
Accuracy: 0.856
Precision: 0.857
Recall: 0.313
Results for SVM, Standard:
Accuracy: 0.887
Precision: 0.886
Recall: 0.482
Results for SVM, MinMax:
Accuracy: 0.887
Precision: 0.879
Recall: 0.485
Results for SVM, MinMaxRange:
Accuracy: 0.885
Precision: 0.88
Recall: 0.473
Results for SVM, Robust:
Accuracy: 0.879
Precision: 0.868
Recall: 0.446
Results for SVM, MaxAbs:
Accuracy: 0.887
Precision: 0.879
Recall: 0.485
Results for SVM, QuantileTransformer:
Accuracy: 0.887
Precision: 0.877
Recall: 0.488
Results for SVM, QuantileTransformerNorm:
Accuracy: 0.887
Precision: 0.877
Recall: 0.488
########################################
Results for RF, None:
Accuracy: 0.882
Precision: 0.783
Recall: 0.545
Results for RF, Standard:
Accuracy: 0.883
Precision: 0.786
Recall: 0.545
Results for RF, MinMax:
Accuracy: 0.883
Precision: 0.787
Recall: 0.546
Results for RF, MinMaxRange:
Accuracy: 0.883
Precision: 0.785
Recall: 0.546
Results for RF, Robust:
Accuracy: 0.882
Precision: 0.785
Recall: 0.544
Results for RF, MaxAbs:
Accuracy: 0.883
Precision: 0.788
Recall: 0.545
Results for RF, QuantileTransformer:
Accuracy: 0.882
Precision: 0.784
Recall: 0.546
Results for RF, QuantileTransformerNorm:
Accuracy: 0.882
Precision: 0.784
Recall: 0.546
########################################
Results for XGB, None:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, Standard:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, MinMax:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, MinMaxRange:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, Robust:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, MaxAbs:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, QuantileTransformer:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
Results for XGB, QuantileTransformerNorm:
Accuracy: 0.891
Precision: 0.87
Recall: 0.518
########################################
Results for CB, None:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, Standard:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, MinMax:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, MinMaxRange:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, Robust:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, MaxAbs:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, QuantileTransformer:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
Results for CB, QuantileTransformerNorm:
Accuracy: 0.892
Precision: 0.871
Recall: 0.521
########################################
#!/usr/bin/env python
import numpy as np
import config as cfg
from tools import file_reader, preprocessor
from sklearn.model_selection import train_test_split
import xgboost as xgb
from sklearn.ensemble import RandomForestClassifier
from tools import file_reader, file_writer, preprocessor
from sklearn.preprocessing import StandardScaler